Senior Data Engineer - Spark & Databricks, Global Projects

EPAM Systems; México; On-site; MXN 600,000 - 900,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off and sick leave

Upskilling and certification courses

Global career opportunities

Job summary

EPAM Systems in Mexico is seeking a seasoned data engineer to design scalable ingestion and transformation frameworks for analytics, and to build batch, near real-time, and streaming pipelines using Databricks and Apache Spark.

You will lead data governance, mentor engineers, collaborate with cross-functional teams, and help deliver end-to-end data products while advancing platform standards across the organization.

Qualifications

• 3+ years of data engineering with modern data platforms.
• Strong leadership to drive engineering standards and mentor engineers.
• Proven end-to-end delivery of enterprise analytics solutions.
• Advanced Spark and PySpark for large-scale processing.
• Advanced Databricks in production, with performance tuning.
• Strong SQL for transformation logic and analytics datasets.
• Knowledge of Lakehouse and Medallion patterns.
• Hands-on CI/CD and Git-based SDLC.

Responsibilities

• Design scalable data ingestion and transformation frameworks for analytics use cases.
• Build batch, near real-time, and streaming pipelines using Databricks and Apache Spark.
• Define Databricks best practices for Medallion layers, performance, and operations.
• Deliver end-to-end data products from raw ingestion through governed consumption layers.
• Develop trusted analytical datasets and semantic models for reporting and self-service.
• Implement data quality, lineage, monitoring, and observability across pipelines.
• Lead proof-of-concepts and evaluate new capabilities within the Databricks ecosystem.
• Enable AI-assisted engineering patterns and reusable accelerators for faster delivery.
• Coordinate with engineering and analytics stakeholders to align on delivery plans.
• Mentor engineers and promote consistent platform standards across teams.

Skills

Data engineering

Leadership

Project delivery

Apache Spark

PySpark

Databricks

SQL

Lakehouse

Medallion patterns

CI/CD

Git workflows

Data governance

English (B2)

Tools

Git

CI/CD

Job description

EPAM Systems in Mexico is seeking a seasoned data engineer to design scalable ingestion and transformation frameworks for analytics, and to build batch, near real-time, and streaming pipelines using Databricks and Apache Spark.

You will lead data governance, mentor engineers, collaborate with cross-functional teams, and help deliver end-to-end data products while advancing platform standards across the organization.