Azure ETL Data Engineer (PySpark & Synapse)

Bluelightconsulting; Hermosillo; A distancia; MXN 480.000 - 720.000; Jornada completa

Descripción de la vacante

Bluelight seeks an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) within Azure Synapse Analytics. You will work with data scientists and analysts to ensure accurate, timely data delivery for insights and decision-making.

The ideal candidate has hands-on experience with PySpark, SQL, and Azure Synapse, and can build scalable data models, pipelines, and dashboards in a collaborative, fast-paced environment.

Formación

• ETL data engineering experience using Python (PySpark) for data extraction, transformation, and loading from REST APIs, SQL databases, and CSV files.
• Proficiency with Azure Synapse Analytics resources including Notebooks, Pipelines, Linked Services, and Azure Key Vault.
• Strong SQL skills (SparkSQL and MS SQL) for ETL, data modeling and performance optimization.
• Experience with version control systems, such as Git (Azure DevOps).
• Excellent communication and teamwork abilities in a fast-paced environment.

Responsabilidades

• ETL data engineering: develop and maintain ETL processes using Python (PySpark) in Azure Synapse notebooks/pipelines.
• Data warehousing: design data storage structures in an MPP SQL pool based on star schemas.
• Data source integration: extract data from REST APIs, SQL tables, and CSV files.
• Azure Synapse Analytics: optimize notebooks/pipelines for scalability and performance.
• Data modeling: collaborate to create data models and schemas per business needs.
• Data quality: implement checks and validation to ensure data accuracy.
• Monitoring: troubleshoot ETL jobs and ensure pipeline reliability.
• Documentation: maintain documentation of data flows and transformations.
• Collaboration: work with cross-functional teams to meet data requirements.

Conocimientos

Python (PySpark)

SQL

Big data

Analytical thinking

Communication

Educación

Bachelor’s degree in Computer Science or IT

Herramientas

Azure Synapse Analytics

Notebooks

Pipelines

Git (Azure DevOps)

Power BI

Descripción del empleo

Bluelight seeks an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) within Azure Synapse Analytics. You will work with data scientists and analysts to ensure accurate, timely data delivery for insights and decision-making.

The ideal candidate has hands-on experience with PySpark, SQL, and Azure Synapse, and can build scalable data models, pipelines, and dashboards in a collaborative, fast-paced environment.