Azure Data Engineer: ETL & Data Warehouse Expert

Bluelightconsulting; San Luis Potosí; A distancia; MXN 480.000 - 780.000; Jornada completa

Descripción de la vacante

Bluelight is a leading software consultancy seeking an ETL Data Engineer to design, develop, and maintain data integration processes using Python (PySpark) and Azure Synapse Analytics to ensure accurate analytical data for insights and decision-making.

You will collaborate with data scientists, analysts, and stakeholders in a fast-paced environment where quality and governance are valued, contributing to scalable data pipelines and robust data models.

Formación

• Proven ETL data engineering using Python (PySpark) and data extraction from REST APIs, SQL databases, and CSV files.
• Proficiency with Azure Synapse resources including Notebooks and Pipelines.
• Ability to write and optimize SQL queries in SparkSQL and MS SQL for data pipelines.

Responsabilidades

• ETL Data Engineering: Develop and maintain ETL processes using Python (PySpark) within Azure Synapse Notebooks/Pipelines.
• Data Warehousing: Design data storage structures in an MPP SQL pool with star schemas, facts, and dimensions.
• Data Source Expertise: Extract data from REST APIs, SQL tables, and CSV files; ensure data quality and performance.
• Azure Synapse Analytics Expertise: Optimize notebooks/pipelines for scalability and speed.
• Data Modeling: Collaborate with data architects to create compliant data models.
• Data Quality: Implement validation checks to ensure data accuracy and consistency.
• Performance Tuning: Identify bottlenecks and optimize ETL processes to meet SLAs.
• Monitoring and Troubleshooting: Monitor jobs, diagnose issues, and ensure pipeline reliability.
• Documentation: Maintain documentation of ETL processes, data flows, and transformations.
• Collaboration: Work with cross-functional teams on data requirements and governance.

Conocimientos

Python PySpark

Azure Synapse

SQL Querying

ETL Engineering

Data Integration

Git AzureDevOps

Data Modeling

Spark SQL

Power BI Tableau

Educación

Bachelor's degree

Azure Data Engineer cert (nice to have)

Herramientas

Azure Synapse Notebooks

Azure Synapse Pipelines

Linked Services

Azure Key Vault

Git

Descripción del empleo

Bluelight is a leading software consultancy seeking an ETL Data Engineer to design, develop, and maintain data integration processes using Python (PySpark) and Azure Synapse Analytics to ensure accurate analytical data for insights and decision-making.

You will collaborate with data scientists, analysts, and stakeholders in a fast-paced environment where quality and governance are valued, contributing to scalable data pipelines and robust data models.