Azure Data Engineer — PySpark ETL & Synapse Expert

Bluelightconsulting; Puebla de Zaragoza; A distancia; MXN 600.000 - 900.000; Jornada completa

Descripción de la vacante

Bluelightconsulting seeks an ETL Data Engineer to design, develop, and maintain data pipelines using Python (PySpark) in Azure Synapse. You will work with data scientists and analysts to ensure accurate, timely data for insights and decision-making.

The role requires strong SQL/SparkSQL skills and experience with REST APIs, SQL tables, and CSV sources. Join a fast-paced environment focused on data quality, governance, and scalable data architectures.

Formación

• Bachelor’s degree or equivalent work experience in CS/IT.
• Azure Data Engineer certification is a plus.
• Strong ETL experience with Python (PySpark).
• Proficient in Azure Synapse Notebooks, Pipelines, and Key Vault.
• Experience writing complex SQL and SparkSQL; MS SQL experience.

Responsabilidades

• ETL data engineering: develop and maintain ETL pipelines using PySpark in Azure Synapse for efficient extraction, transformation, and loading.
• Data warehousing: design star schemas and efficient storage in an MPP SWL pool.
• Data source expertise: extract data from REST APIs, SQL tables, and CSV files.
• Azure Synapse expertise: design notebooks/pipelines for scalability and performance.
• Data fabric concepts: contribute to data lakes, lakehouses, and cataloging initiatives.
• Data modeling: collaborate to create business-aligned data models.
• Data quality: implement checks and validation processes.
• Performance tuning: identify bottlenecks and optimize ETL pipelines to SLAs.
• Monitoring and troubleshooting: monitor jobs and fix issues.
• Documentation: maintain comprehensive docs of data flows and transformations.
• Collaboration: work with cross-functional teams on data initiatives.
• Security and compliance: ensure data governance and privacy.

Conocimientos

Python (PySpark)

ETL data engineering

SQL queries

REST APIs

CSV files

Big data

Machine learning basics

Agile practices

Power BI

Tableau

SparkSQL

Azure Synapse analytics

Educación

Bachelor’s degree in Computer Science or IT

Azure Data Engineer certification

Herramientas

Azure Synapse Notebooks

Azure Synapse Pipelines

Azure Key Vault

Linked Services

Git (Azure DevOps)

Descripción del empleo

Bluelightconsulting seeks an ETL Data Engineer to design, develop, and maintain data pipelines using Python (PySpark) in Azure Synapse. You will work with data scientists and analysts to ensure accurate, timely data for insights and decision-making.

The role requires strong SQL/SparkSQL skills and experience with REST APIs, SQL tables, and CSV sources. Join a fast-paced environment focused on data quality, governance, and scalable data architectures.