Senior Data Software Engineer

EPAM Systems Inc; Brasil; Remote; BRL 180,000 - 240,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off and sick leave

Upskilling, reskilling and certific c a

LinkedIn Learning access

Global career opportunities

Job summary

EPAM Systems Inc is seeking a Senior Data Software Engineer to design scalable data solutions that support critical business decisions. You will build resilient data pipelines, implement ingestion from diverse sources, and ensure code quality through tests and reviews.

You will collaborate with cross-functional teams in an agile environment, optimize data movement, and contribute to data architecture improvements while upholding governance and reliability standards.

Qualifications

• 3+ years of relevant professional experience.
• Strong hands-on experience with PySpark and Spark SQL for large-scale data workflows.
• Experience building and managing Lakeflow pipelines and Jobs for data orchestration.
• Proficiency with Auto Loader and JDBC/CDC ingestion methods (DB2, SQL Server).
• Experience with Delta Lake and Unity Catalog for data storage and governance.
• Data quality assurance and source reconciliation experience.
• Unit testing with pytest.
• Git and deployment workflow knowledge.
• Upper-intermediate English (B2) or higher.

Responsibilities

• Design and build scalable data pipelines that support reliable data movement across multiple systems.
• Develop and maintain data ingestion processes from a variety of source systems.
• Write clean, efficient, and well-tested code following strong software engineering practices.
• Collaborate with cross-functional teams to understand data requirements and translate them into technical solutions.
• Monitor data pipeline performance and proactively address issues to maintain system reliability.
• Participate in code reviews to uphold quality standards and share knowledge across the team.
• Troubleshoot and resolve data quality issues, ensuring accuracy and consistency across datasets.
• Contribute to the ongoing improvement of data architecture and engineering standards.
• Document technical processes and solutions to support knowledge sharing and team onboarding.
• Work within an agile environment, participating in sprint planning and other team ceremonies.

Skills

English (B2+)

Team collaboration

Problem solving

Analytical thinking

Tools

PySpark

Spark SQL

Lakeflow

Auto Loader

JDBC/CDC

Delta Lake

Unity Catalog

Git

pytest

Databricks

Asset Bundles

Job description

We are looking for a Senior Data Software Engineer to join our team. This role is ideal for someone who enjoys designing resilient data solutions that support critical business decision-making. You'll work closely with a team that values clean, well-tested code and strong engineering practices. If you're motivated by solving complex data challenges at scale, this position offers plenty of room to grow and make an impactResponsibilitiesDesign and build scalable data pipelines that support reliable data movement across multiple systemsDevelop and maintain data ingestion processes from a variety of source systemsWrite clean, efficient, and well-tested code following strong software engineering practicesCollaborate with cross-functional teams to understand data requirements and translate them into technical solutionsMonitor data pipeline performance and proactively address issues to maintain system reliabilityParticipate in code reviews to uphold quality standards and share knowledge across the teamTroubleshoot and resolve data quality issues, ensuring accuracy and consistency across datasetsContribute to the ongoing improvement of data architecture and engineering standardsDocument technical processes and solutions to support knowledge sharing and team onboardingWork within an agile environment, participating in sprint planning and other team ceremoniesRequirementsA minimum of 3 years of relevant professional experienceStrong hands-on experience with PySpark and Spark SQL for building and processing large-scale data workflowsExperience building and managing Lakeflow pipelines and Jobs for data orchestrationProficiency with Auto Loader and JDBC/CDC ingestion methods, working with sources such as DB2 and SQL ServerSolid experience with Delta Lake and Unity Catalog for managing data storage and governanceBackground ensuring data quality and performing source reconciliation to maintain accurate, trustworthy datasetsExperience writing unit tests using pytest to validate code reliability and correctnessWorking knowledge of Git and Asset Bundles for version control and deployment managementUpper-intermediate English language skills or higher (B2)Nice to haveExperience with DataStage migration projectsBackground in Dynamics 365/Dataverse data extractionDatabricks certificationWe offerInternational projects with top brandsWork with global teams of highly skilled, diverse peersHealthcare benefitsEmployee financial programsPaid time off and sick leaveUpskilling, reskilling and certification coursesUnlimited access to the LinkedIn Learning library and 22,000+ coursesGlobal career opportunitiesVolunteer and community involvement opportunitiesEPAM Employee GroupsAward-winning culture recognized by Glassdoor, Newsweek and LinkedInEPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.