Senior Data Platform Engineer: Scalable Pipelines & AI

EPAM Systems, Inc.; México; A distancia; MXN 900.000 - 1.300.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Healthcare benefits

Paid time off

Descripción de la vacante

EPAM Systems, Inc. seeks a Senior Data Software Engineer to design scalable data pipelines and platforms powering analytics, reporting, and AI-driven initiatives.

You will collaborate with Product, Engineering, and Business teams to deliver robust data solutions using modern cloud and big data technologies. Responsibilities include building batch and streaming pipelines, optimizing Databricks/Spark/Kafka, and implementing data quality, governance, and observability.

Formación

• 3+ years of experience in data engineering or a related field.
• Proficiency in SQL, Python, PySpark, Scala, and Java.
• Expertise in Apache Spark, Spark SQL, Kafka, and Delta Lake.
• Skills in Azure Databricks, Unity Catalog, and Databricks Asset Bundles (DAB).
• Familiarity with Azure Data Factory, ADLS Gen2, Azure Key Vault, Cosmos DB, and Monitor.
• Knowledge of Data Modeling, Data Quality, Data Governance, and PII Handling.
• Background in Medallion Architecture (Bronze/Silver/Gold), CDC, and SCD Type 1 & Type 2.
• Competency in Azure DevOps, Git, CI/CD, and Gradle.

Responsabilidades

• Design and develop scalable batch and streaming data pipelines.
• Build and optimize Databricks, Spark, and Kafka-based solutions.
• Implement data quality, governance, and observability standards.
• Develop data models and reporting foundations for analytics and AI initiatives.
• Partner with Product, Engineering, and Business teams to deliver data-driven solutions.
• Support enterprise reporting, dashboards, and self-service analytics platforms.
• Apply MCP, prompt engineering, and multiple LLMs in AI-assisted development workflows.

Conocimientos

Analytical thinking

Team collaboration

Problem solving

Herramientas

Azure Databricks

Unity Catalog

Databricks Asset Bundles (DAB)

Azure Data Factory

ADLS Gen2

Azure Key Vault

Azure Cosmos DB

Azure Monitor

Apache Spark

Spark SQL

Kafka

Delta Lake

Python

SQL

PySpark

Scala

Java

CI/CD

Gradle

Descripción del empleo

EPAM Systems, Inc. seeks a Senior Data Software Engineer to design scalable data pipelines and platforms powering analytics, reporting, and AI-driven initiatives.

You will collaborate with Product, Engineering, and Business teams to deliver robust data solutions using modern cloud and big data technologies. Responsibilities include building batch and streaming pipelines, optimizing Databricks/Spark/Kafka, and implementing data quality, governance, and observability.