Lead Data Engineer: Databricks Spark Architect & Coach

EPAM Systems, Inc.; México; A distancia; MXN 1.200.000 - 1.900.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Healthcare benefits

Paid time off and sick leave

Upskilling, reskilling and certificate

LinkedIn Learning access

Descripción de la vacante

EPAM Systems, Inc. is seeking a Lead Data Engineer to shape scalable data ingestion, transformation, and governed data products on Databricks. You will blend hands-on delivery with technical leadership to enable self-service analytics and AI-enabled engineering.

Responsibilities include building batch and streaming pipelines with Spark, delivering end-to-end data products, and mentoring engineers while enforcing best practices across medallion layers and observability standards.

Formación

• 5+ years of data engineering experience with production-grade platforms.
• Lead-level technical leadership to drive standards and mentor engineers.
• End-to-end delivery across ingestion, transformation, semantic layers, and reporting.
• Advanced Apache Spark and PySpark expertise for scalable pipelines.
• Advanced SQL skills for transformation, modeling, and performance tuning.
• Upper-Intermediate English proficiency (B2) for leading technical discussions.

Responsabilidades

• Design scalable data ingestion and transformation frameworks on Databricks.
• Define and enforce Databricks best practices for medallion layers.
• Build batch and streaming pipelines using Spark and PySpark.
• Deliver end-to-end data products from ingestion to governed layers.
• Develop trusted analytical datasets and semantic models for reporting.
• Implement data quality, lineage, monitoring, and observability standards.
• Lead PoCs and evaluate emerging Databricks capabilities.
• Mentor engineers and drive reusable frameworks across teams.

Conocimientos

Databricks

Apache Spark

PySpark

SQL

Data ingestion

Data transformation

Data quality

Data lineage

Observability

Leadership

Scala

Mentoring

Herramientas

Scala

Descripción del empleo

EPAM Systems, Inc. is seeking a Lead Data Engineer to shape scalable data ingestion, transformation, and governed data products on Databricks. You will blend hands-on delivery with technical leadership to enable self-service analytics and AI-enabled engineering.

Responsibilities include building batch and streaming pipelines with Spark, delivering end-to-end data products, and mentoring engineers while enforcing best practices across medallion layers and observability standards.