Lead Data Platform Engineer for AI Analytics

EPAM Systems, Inc.; México; A distancia; MXN 1.200.000 - 1.800.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Health benefits

Paid time off

Upskilling & certification

Descripción de la vacante

EPAM Systems, Inc. seeks a Lead Data Software Engineer to design and optimize cloud-scale data pipelines for analytics and AI initiatives.

You will collaborate with Product, Engineering, and Business teams to deliver reliable data solutions using Azure Databricks, Spark, Kafka, and governance best practices. You will build scalable batch and streaming pipelines, implement data quality and observability, and contribute to data models and reporting foundations for analytics.

Formación

• 5+ years of data engineering experience.
• Proficient in SQL, Python, PySpark, Scala and Java.
• Deep expertise in Spark, Spark SQL, Kafka, Delta Lake.
• Azure Databricks, Unity Catalog, and DAB experience.
• Strong knowledge of Azure data services and governance.

Responsabilidades

• Design scalable batch and streaming data pipelines.
• Optimize Databricks, Spark, and Kafka solutions.
• Implement data quality, governance, and observability standards.
• Develop data models and analytics foundations for AI initiatives.
• Collaborate with Product, Engineering, and Business teams on data-driven solutions.
• Support enterprise reporting, dashboards, and self-service analytics.

Conocimientos

5+ years experience

SQL

Python

PySpark

Scala

Java

Spark

Kafka

Delta Lake

Azure Databricks

Unity Catalog

Databricks Asset Bundles

Data Modeling

Data Quality

Data Governance

PII Handling

Medallion Architecture

CDC

SCD Type 1 & Type 2

Azure DevOps

Git

CI/CD

Gradle

Herramientas

Azure Databricks

Unity Catalog

Databricks Asset Bundles (DAB)

Azure Data Factory

ADLS Gen2

Azure Key Vault

Azure Monitor

Cosmos DB

Data Visualization tools

Tableau/ThoughtSpot/Cognos

Descripción del empleo

EPAM Systems, Inc. seeks a Lead Data Software Engineer to design and optimize cloud-scale data pipelines for analytics and AI initiatives.

You will collaborate with Product, Engineering, and Business teams to deliver reliable data solutions using Azure Databricks, Spark, Kafka, and governance best practices. You will build scalable batch and streaming pipelines, implement data quality and observability, and contribute to data models and reporting foundations for analytics.