Lead Data Software Engineer

EPAM Systems, Inc.; México; A distancia; MXN 1.200.000 - 1.800.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Health benefits

Paid time off

Upskilling & certification

Descripción de la vacante

EPAM Systems, Inc. seeks a Lead Data Software Engineer to design and optimize cloud-scale data pipelines for analytics and AI initiatives.

You will collaborate with Product, Engineering, and Business teams to deliver reliable data solutions using Azure Databricks, Spark, Kafka, and governance best practices. You will build scalable batch and streaming pipelines, implement data quality and observability, and contribute to data models and reporting foundations for analytics.

Formación

• 5+ years of data engineering experience.
• Proficient in SQL, Python, PySpark, Scala and Java.
• Deep expertise in Spark, Spark SQL, Kafka, Delta Lake.
• Azure Databricks, Unity Catalog, and DAB experience.
• Strong knowledge of Azure data services and governance.

Responsabilidades

• Design scalable batch and streaming data pipelines.
• Optimize Databricks, Spark, and Kafka solutions.
• Implement data quality, governance, and observability standards.
• Develop data models and analytics foundations for AI initiatives.
• Collaborate with Product, Engineering, and Business teams on data-driven solutions.
• Support enterprise reporting, dashboards, and self-service analytics.

Conocimientos

5+ years experience

SQL

Python

PySpark

Scala

Java

Spark

Kafka

Delta Lake

Azure Databricks

Unity Catalog

Databricks Asset Bundles

Data Modeling

Data Quality

Data Governance

PII Handling

Medallion Architecture

CDC

SCD Type 1 & Type 2

Azure DevOps

Git

CI/CD

Gradle

Herramientas

Azure Databricks

Unity Catalog

Databricks Asset Bundles (DAB)

Azure Data Factory

ADLS Gen2

Azure Key Vault

Azure Monitor

Cosmos DB

Data Visualization tools

Tableau/ThoughtSpot/Cognos

Descripción del empleo

We are building cloud-scale data pipelines and platforms as a Lead Data Software Engineer to enable analytics, reporting, and AI-driven initiatives. You will collaborate with Product, Engineering, and Business teams to deliver reliable solutions using Azure Databricks, Spark, Kafka, and modern governance practices.ResponsibilitiesDesign scalable batch and streaming data pipelinesBuild and fine-tune Databricks, Spark, and Kafka-based solutionsImplement data quality, governance, and observability standardsDevelop data models and reporting foundations for analytics and AI initiativesCollaborate with Product, Engineering, and Business teams to deliver data-driven solutionsSupport enterprise reporting, dashboards, and self-service analytics platformsLeverage AI tools such as Claude Code, GitHub Copilot, and similar platforms for development, testing, and productivity improvementsApply MCP, prompt engineering, and multiple LLMs in AI-assisted development workflowsRequirementsProven track record with 5+ years of experience in data engineering or a related fieldHands-on proficiency in SQL, Python, PySpark, Scala, and JavaDeep expertise in Apache Spark, Spark SQL, Kafka, and Delta LakeStrong experience with Azure Databricks, Unity Catalog, and Databricks Asset Bundles (DAB)Solid familiarity with Azure Data Factory (ADF), ADLS Gen2, Azure Key Vault, Azure Cosmos DB, and Azure MonitorThorough knowledge of Data Modeling, Data Quality, Data Governance, and PII HandlingDemonstrated background in Medallion Architecture (Bronze/Silver/Gold), CDC, and SCD Type 1 & Type 2Practical competency in Azure DevOps, Git, CI/CD, and GradleEnglish proficiency at B2+ (Upper-Intermediate) levelNice to haveFamiliarity with Tableau, ThoughtSpot, Cognos, and Data Visualization Best PracticesKnowledge of Semantic Layer Concepts, AI/BI Dashboards, and Databricks GenieUnderstanding of scalable data platforms, analytics, and reporting solutionsWe offerInternational projects with top brandsWork with global teams of highly skilled, diverse peersHealthcare benefitsEmployee financial programsPaid time off and sick leaveUpskilling, reskilling and certification coursesUnlimited access to the LinkedIn Learning library and 22,000+ coursesGlobal career opportunitiesVolunteer and community involvement opportunitiesEPAM Employee GroupsAward-winning culture recognized by Glassdoor, Newsweek and LinkedInEPAM is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, age, sexual orientation, gender identity or expression, disability, protected veteran status, or any other characteristic protected by applicable law.