Lead Data Engineer: Scalable Pipelines & Lakehouse

EPAM Systems Inc; México; Remote; MXN 900,000 - 1,300,000; Full time

Benefits offered by this job

Healthcare benefits

Global career opportunities

Learning & development programs

Paid time off

Job summary

EPAM Systems Inc in Mexico is seeking a Lead Data Software Engineer to build robust data frameworks that drive critical organizational decisions. You will design scalable pipelines, ingest data from diverse sources, and deliver clean, well-tested code within an agile team.

The role emphasizes leadership, code reviews, and data quality, with exposure to Delta Lake, Unity Catalog, Lakeflow, and Databricks. Strong PySpark, Spark SQL, and English communication are essential.

Qualifications

• Minimum 5 years of hands-on data engineering experience.
• At least 1 year of leading development teams.
• Strong experience with large-scale data processing and modern data platforms.

Responsibilities

• Design and assemble scalable data pipelines across platforms.
• Ingest data from multiple source systems with robust ETL/ELT processes.
• Deliver clean, tested code built on solid software engineering foundations.
• Collaborate with cross-functional teams to translate data requirements into solutions.
• Monitor pipeline performance and ensure reliability; participate in code reviews.
• Address data quality issues and drive improvements to data architectures and governance.
• Produce technical documentation for onboarding and knowledge transfer.

Skills

PySpark

Spark SQL

Delta Lake

Unity Catalog

Lakeflow

Auto Loader

JDBC/CDC

Databricks

Git

pytest

Databricks certification

Job description

EPAM Systems Inc in Mexico is seeking a Lead Data Software Engineer to build robust data frameworks that drive critical organizational decisions. You will design scalable pipelines, ingest data from diverse sources, and deliver clean, well-tested code within an agile team.

The role emphasizes leadership, code reviews, and data quality, with exposure to Delta Lake, Unity Catalog, Lakeflow, and Databricks. Strong PySpark, Spark SQL, and English communication are essential.