Azure Synapse ETL Engineer: Data Warehousing & Analytics

Bluelightconsulting; Mérida; A distancia; MXN 420.000 - 700.000; Jornada completa

Descripción de la vacante

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) within Azure Synapse Analytics. You will work closely with data scientists, analysts, and stakeholders to ensure accurate, timely data for analytics and decision-making in a fast-paced setting.

The role emphasizes building scalable ETL processes, implementing data quality checks, and collaborating across teams to enable data-driven insights while ensuring security and

Formación

• Bachelor's degree in Computer Science, Information Technology, or related field, or equivalent work experience.
• Proven experience in ETL data engineering using Python (PySpark) for data extraction, transformation, and loading from REST APIs, SQL databases, and CSV files.
• Proficiency in Azure Synapse Analytics resources including Notebooks, Pipelines, Linked Services, and Key Vault.
• Strong SQL skills (SparkSQL/MS SQL) and ability to optimize queries for performance.
• Knowledge of data integration best practices and tools.
• Experience with version control systems such as Git (Azure DevOps).
• Excellent problem-solving and analytical skills with attention to detail.
• Strong communication and collaboration skills; comfortable working in cross-functional teams.
• Familiarity with big data, machine learning, and data analysis is a plus.

Responsabilidades

• ETL Data Engineering: Develop and maintain ETL processes using Python (PySpark) within Azure Synapse Notebooks and/or Pipelines.
• Data Warehousing: Design data storage structures in an MPP SQL pool using star schemas.
• Data Source Expertise: Extract data from REST APIs, SQL tables, and CSV files.
• Azure Synapse Analytics Expertise: Optimize notebooks/pipelines for scalability and performance.
• Data Fabric Concepts: Contribute to data lakes, lakehouses, delta lakes, and cataloging.
• Data Modeling: Create data models and schemas with data architects.
• Data Quality: Implement validation checks to ensure data accuracy.
• Performance Tuning: Identify bottlenecks and optimize ETL workloads to meet SLAs.
• Monitoring and Troubleshooting: Monitor jobs and resolve issues to ensure reliability.
• Documentation: Maintain thorough documentation of data flows and transformations.
• Collaboration: Work with cross-functional teams on data requirements.
• Security and Compliance: Ensure data governance and privacy compliance.

Conocimientos

Python (PySpark)

Azure Synapse

SQL

Data warehousing

REST APIs

CSV files

Git / Azure DevOps

Data modeling

Data quality

Power BI / Tableau

Educación

Bachelor's degree in CS/IT or related field

Herramientas

Azure Synapse Notebooks

Azure Synapse Pipelines

Azure Key Vault

Descripción del empleo

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) within Azure Synapse Analytics. You will work closely with data scientists, analysts, and stakeholders to ensure accurate, timely data for analytics and decision-making in a fast-paced setting.

The role emphasizes building scalable ETL processes, implementing data quality checks, and collaborating across teams to enable data-driven insights while ensuring security and