Azure Data Engineer: ETL Mastery with PySpark & Synapse

Bluelightconsulting; Ecatepec de Morelos; A distancia; MXN 420.000 - 780.000; Jornada completa

Descripción de la vacante

Bluelight seeks an ETL Data Engineer to design, develop, and maintain data integration processes using Python (PySpark) within Azure Synapse Analytics. You will work closely with data scientists, analysts, and stakeholders to ensure accurate, accessible analytical data for decision-making in a fast-paced environment.

Responsibilities include building scalable ETL pipelines, modeling data in star schemas, and implementing data quality checks, governance, and performance tuning to meet SLAs.

Formación

• Proven ETL data engineering with Python (PySpark) for data extraction, transformation, loading from REST APIs, SQL tables, and CSV files.
• Proficient in Azure Synapse Analytics resources including Notebooks, Pipelines, Linked Services, and Key Vault.
• Strong SQL skills; experience with SparkSQL and MS SQL for data processing.
• Knowledge of data integration best practices and tools.
• Experience with Git/Azure DevOps for version control.
• Excellent communication and collaborative teamwork abilities.
• Familiarity with big data, ML, and data analysis is a plus.

Responsabilidades

• ETL Data Engineering: Develop and maintain ETL processes using Python (PySpark) within Azure Synapse Notebooks and Pipelines.
• Data Warehousing: Design and build star schemas, facts, and dimensions in an MPP SWL pool.
• Data Source Expertise: Extract data from REST APIs, SQL tables, and CSV files.
• Azure Synapse Analytics Expertise: Optimize notebooks/pipelines for scalability and performance.
• Data Fabric Concepts: Contribute to data lakes, lakehouses, delta lakes, and data cataloging.
• Data Modeling: Collaborate to create data models and schemas that meet business needs.
• Data Quality: Implement quality checks and validation processes.
• Performance Tuning: Resolve bottlenecks to meet SLAs.
• Monitoring and Troubleshooting: Monitor ETL jobs and fix issues.
• Documentation: Maintain comprehensive docs of ETL data flows and transformations.
• Collaboration: Work with cross-functional teams on data requirements.
• Security and Compliance: Ensure data security and governance.

Conocimientos

Python PySpark

SQL

Azure Synapse

Data modeling

Data warehousing

Data integration

Git Azure DevOps

Data quality

Data governance

Communication

Analytical

Teamwork

Educación

Bachelor’s degree in CS/IT

Herramientas

Azure Synapse Notebooks

Azure Synapse Pipelines

Azure Key Vault

Azure Data Engineer (cert)

Descripción del empleo

Bluelight seeks an ETL Data Engineer to design, develop, and maintain data integration processes using Python (PySpark) within Azure Synapse Analytics. You will work closely with data scientists, analysts, and stakeholders to ensure accurate, accessible analytical data for decision-making in a fast-paced environment.

Responsibilities include building scalable ETL pipelines, modeling data in star schemas, and implementing data quality checks, governance, and performance tuning to meet SLAs.