Azure Data Engineer: Pipelines, Lakehouses & SQL Power

Bluelightconsulting; Ciudad de México; A distancia; MXN 450.000 - 650.000; Jornada completa

Descripción de la vacante

Bluelightconsulting is seeking a data engineer to design and implement scalable ETL pipelines using Python (PySpark) in Azure Synapse. You will work across REST APIs, SQL databases, and CSV data sources to deliver reliable data flows.

You will partner with data architects to model star schemas, enforce data quality, and optimize pipelines to meet SLAs while ensuring security and governance.

Formación

• Bachelor’s degree in Computer Science, Information Technology, or related field; equivalent work experience, with certifications in data engineering/data science as a plus.
• Proven ETL data engineering experience with Python (PySpark) for data extraction, transformation and loading from REST APIs, SQL tables and CSV files.
• Proficiency with Azure Synapse resources including Notebooks, Pipelines, Linked Services and Azure Key Vault.
• Demonstrated ability to write complex SQL queries, optimize performance, and work with SparkSQL and MS SQL for ETL.
• Knowledge of data integration best practices and tools, plus Git (Azure DevOps).
• Strong problem-solving and analytical skills with clear communication; team-oriented.

Responsabilidades

• Develop and maintain ETL data pipelines using Python (PySpark) within Azure Synapse Notebooks and/or Pipelines.
• Apply data warehousing principles to design star schemas and dimensions in an MPP SWL Pool.
• Extract data from REST APIs, SQL tables and CSV files.
• Optimize data notebooks/pipelines for scalability and performance in Azure Synapse.
• Contribute to data fabric concepts like data lakes, lakehouses, and delta lakes.
• Collaborate with data architects to model data and schemas.
• Implement data quality checks and validation processes.
• Identify performance bottlenecks and ensure SLAs are met.
• Monitor ETL jobs, diagnose issues and ensure pipeline reliability.
• Maintain documentation of ETL processes and data flows.
• Work with cross-functional teams to support data initiatives and ensure data governance.

Conocimientos

Python

PySpark

Azure Synapse

SQL

REST APIs

Data warehousing

SparkSQL

Git

Power BI

Educación

Bachelor's degree in CS/IT

Azure Data Engineer cert (plus)

Herramientas

Azure Synapse Notebooks

Azure Synapse Pipelines

Azure Key Vault

Linked Services

Descripción del empleo

Bluelightconsulting is seeking a data engineer to design and implement scalable ETL pipelines using Python (PySpark) in Azure Synapse. You will work across REST APIs, SQL databases, and CSV data sources to deliver reliable data flows.

You will partner with data architects to model star schemas, enforce data quality, and optimize pipelines to meet SLAs while ensuring security and governance.