Senior Data Engineer: Azure Databricks & Delta Lake Expert

Mps Group; Xico; Presencial; MXN 600.000 - 900.000; Jornada completa

Descripción de la vacante

Mps Group is seeking a Senior Data Engineer to design and maintain scalable data pipelines in Azure, with hands-on Databricks, Unity Catalog, and ADF expertise. You will collaborate with architects and business stakeholders to deliver secure, governed data platforms and cloud-based analytics.

The role involves building Delta Lake architectures, implementing CI/CD for data products, and ensuring data quality, security, and compliance across enterprise workloads, while continuously improving

Formación

• 4–7+ years of data engineering or big data experience.
• Hands-on Databricks, Unity Catalog, and Azure Data Factory experience.
• Proficient with PySpark, Spark SQL, and Delta Lake.
• Solid understanding of data modeling and data warehousing concepts.
• Experience with Azure cloud services and governance.

Responsabilidades

• Design, develop, and optimize ETL/ELT pipelines using Azure Data Factory and Databricks.
• Build scalable data ingestion frameworks for structured and unstructured data sources.
• Develop data transformation processes using PySpark, Spark SQL, and Databricks notebooks.
• Implement and manage Unity Catalog for enterprise data governance, security, and access controls.
• Create and optimize Delta Lake architectures for analytics workloads.
• Monitor pipeline performance and troubleshoot production issues.
• Collaborate with data architects to implement enterprise data models and best practices.
• Ensure data quality, integrity, security, and compliance standards are met.
• Support CI/CD deployment processes and DevOps for data engineering.
• Work with business users to deliver scalable data solutions.

Conocimientos

Azure Databricks

PySpark

Spark SQL

Unity Catalog

Azure Data Factory

Delta Lake

Data modeling

ETL/ELT

Azure cloud

Git / CI/CD

Descripción del empleo

Mps Group is seeking a Senior Data Engineer to design and maintain scalable data pipelines in Azure, with hands-on Databricks, Unity Catalog, and ADF expertise. You will collaborate with architects and business stakeholders to deliver secure, governed data platforms and cloud-based analytics.

The role involves building Delta Lake architectures, implementing CI/CD for data products, and ensuring data quality, security, and compliance across enterprise workloads, while continuously improving