Lead Data Engineer with Databricks

W3Global; Ciudad de México; Presencial; MXN 1,640,000 - 2,187,000; Jornada completa

Descripción de la vacante

A leading tech company in Mexico City is seeking a Senior Data Engineer to design and maintain scalable ETL/ELT data pipelines. The ideal candidate will have 10 to 15 years of experience with Azure technologies, extensive knowledge in SQL and Python, and a strong understanding of data governance. Responsibilities include optimizing data processes, collaborating with stakeholders, and ensuring data compliance. This position offers opportunities for hands-on project involvement and mentorship in a dynamic environment.

Formación

• 10 to 15 years of professional experience as a Senior Data Engineer.
• Strong hands‑on experience with Azure Data Factory and Azure Databricks.
• Proficiency in Python and SQL for data processing and scripting.
• Solid understanding of ETL/ELT methodologies and best practices.
• Experience designing and managing data solutions in a cloud environment.

Responsabilidades

• Design, build, and maintain scalable ETL/ELT data pipelines.
• Write complex, high-performance SQL queries for data extraction and loading.
• Collaborate with data architects and analysts on data requirements.
• Enforce data governance and security best practices.

Conocimientos

ETL/ELT data pipelines

SQL queries and scripts

Data transformation logic in Python

Azure Data Factory

Snowflake

Data governance and security

Data Modeling

DevOps processes

Educación

Bachelor's degree in computer science or related field

Herramientas

Azure Databricks

PySpark

Scala

Descripción del empleo

Must

• Design, build, and maintain scalable and efficient ETL/ELT data pipelines using Azure Data Factory and Azure Databricks.
• Write complex, high-performance SQL queries and scripts for data extraction, transformation, and loading.
• Implement data transformation logic using Python and PySpark (or Scala) within the Databricks environment.
• Hands‑on experience with Snowflake.
• Ensure pipeline reliability, scalability, and performance through monitoring, alerting, and proactive maintenance.
• Collaborate closely with data architects, analysts, and business stakeholders to understand data requirements and deliver actionable solutions.
• Enforce data governance and security best practices, ensuring compliance with organizational and regulatory standards.
• Optimize data storage, compute, and processing resources in the Azure ecosystem to reduce costs and improve efficiency.
• Document data flows, processes, and technical designs clearly and effectively.
• Contribute to code reviews, technical discussions, and mentorship of junior engineers.
• Bachelor's degree in computer science, Engineering, Information Systems, or a related technical field.
• 10 to 15 years of professional experience as a Senior Data Engineer.
• Strong hands‑on experience with Azure Data Factory and Azure Databricks.
• Proficiency in Python and SQL for data processing and scripting.
• Working experience with PySpark and/or Scala in distributed data processing.
• Solid understanding of ETL/ELT methodologies and best practices.
• Experience designing and managing data solutions in a cloud environment (Azure preferred).
• Proficient knowledge in Data Modeling and Data Warehousing.
• Expertise in DevOps processes.