Senior Data Software Engineer

EPAM Systems Inc; México; A distancia; MXN 900.000 - 1.200.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Healthcare benefits

Paid time off

Upskilling and certification courses

LinkedIn Learning access

Descripción de la vacante

EPAM Systems Inc. seeks a Senior Data Software Engineer to architect scalable data pipelines and data solutions powering critical business decisions.

You will collaborate with a disciplined team focused on clean, well-tested code and robust engineering practices, tackling large-scale data problems with Lakeflow, Delta Lake, and Databricks. You will design ingestion workflows, perform data quality checks, participate in code reviews, and contribute to architecture evolution within an agile

Formación

• At least 3 years of hands-on data engineering experience.
• Strong experience with PySpark and Spark SQL for large-scale data workflows.
• Experience building and overseeing Lakeflow pipelines and Jobs for data orchestration.
• Proficient with Auto Loader and JDBC/CDC ingestion for various sources.
• Experience with Delta Lake and Unity Catalog for storage and governance.
• Experience writing unit tests with pytest to ensure code reliability.
• Familiarity with Git and deployment workflows.

Responsabilidades

• Architect and construct scalable data pipelines across multiple platforms.
• Build and sustain data ingestion workflows from diverse sources.
• Write clean, high-performing, well-tested code.
• Collaborate with cross-functional teams to shape data solutions.
• Monitor and optimize data pipeline performance and reliability.
• Participate in code reviews to maintain quality.
• Diagnose and fix data quality issues to ensure accuracy across datasets.
• Contribute to evolving data architecture practices and standards.
• Create documentation for technical workflows and solutions.
• Operate within an agile framework and participate in sprint planning.

Conocimientos

PySpark

Spark SQL

Data pipelines

Unit testing with pytest

Git

English (B2+)

Delta Lake

Unity Catalog

DataStage migration

Herramientas

Auto Loader

JDBC/CDC ingestion

Delta Lake

Unity Catalog

Databricks

Lakeflow

Descripción del empleo

We're on the lookout for a Senior Data Software Engineer to come aboard our team. This position is a great match for someone who takes pleasure in architecting durable data solutions that power important business choices. You'll collaborate closely with a group that prioritizes clean, thoroughly tested code and disciplined engineering approaches. For anyone energized by tackling large-scale data problems, this role provides ample opportunity for both growth and meaningful contributionResponsibilitiesArchitect and construct scalable data pipelines that enable dependable data flow across multiple platformsBuild and sustain data ingestion workflows pulling from a range of different source systemsProduce clean, high-performing, thoroughly tested code grounded in solid software engineering principlesPartner with teams across different functions to understand data needs and shape them into working technical solutionsKeep watch over data pipeline performance, stepping in ahead of time to resolve issues and preserve system dependabilityJoin in on code review sessions to help maintain quality benchmarks and spread knowledge throughout the teamDiagnose and fix data quality problems, safeguarding accuracy and consistency across all datasetsHelp drive continuous enhancement of data architecture practices and engineering standardsCreate documentation covering technical workflows and solutions to aid knowledge sharing and new team member onboardingOperate within an agile framework, taking part in sprint planning sessions and other recurring team activitiesRequirementsAt least 3 years of relevant hands-on professional backgroundSolid practical expertise with PySpark and Spark SQL for constructing and processing large-scale data workflowsBackground building and overseeing Lakeflow pipelines and Jobs to support data orchestrationSkilled in using Auto Loader along with JDBC/CDC ingestion approaches, working with sources like DB2 and SQL ServerStrong background with Delta Lake and Unity Catalog for handling data storage and governance needsExperience maintaining data quality standards and performing source reconciliation to ensure dependable, accurate datasetsBackground writing unit tests through pytest to confirm code correctness and reliabilityFamiliarity with Git and Asset Bundles for managing version control and deployment workflowsUpper-intermediate English language skills or higher (B2)Nice to haveBackground working on DataStage migration initiativesExperience extracting data from Dynamics 365/DataverseHolding a Databricks certificationWe offerInternational projects with top brandsWork with global teams of highly skilled, diverse peersHealthcare benefitsEmployee financial programsPaid time off and sick leaveUpskilling, reskilling and certification coursesUnlimited access to the LinkedIn Learning library and 22,000+ coursesGlobal career opportunitiesVolunteer and community involvement opportunitiesEPAM Employee GroupsAward-winning culture recognized by Glassdoor, Newsweek and LinkedInEPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.