Azure Data Engineer: ETL, Synapse & Data Warehousing

Bluelightconsulting; Piura; A distancia; PEN 136.286 - 238.501; Jornada completa

Descripción de la vacante

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data pipelines using Python (PySpark) within Azure Synapse Analytics. You will ensure accurate, timely data for analytics and collaborate with data scientists, analysts, and stakeholders.

Responsibilities include building scalable ETL processes, data warehousing concepts, data quality checks, and performance tuning across notebooks and pipelines.

Formación

• Bachelor's degree in Computer Science, Information Technology, or a related field; or equivalent work experience, with Azure Data Engineer certs as a plus.
• Proven experience in ETL data engineering using Python (PySpark) to extract, transform, and load from REST APIs, SQL tables, and CSV files.
• Proficiency with Azure Synapse Analytics resources including Notebooks, Pipelines, Linked Services, and Azure Key Vault.
• Experience writing complex SQL queries and working with SparkSQL and MS SQL for ETL.
• Knowledge of data integration best practices and tools.
• Familiarity with Git/Azure DevOps for version control.
• Strong problem-solving and analytical skills with attention to detail.
• Excellent communication and teamwork abilities.

Responsabilidades

• ETL Data Engineering: Develop and maintain ETL data pipelines using Python (PySpark) in Azure Synapse Analytics Notebooks and Pipelines.
• Data Warehousing: Design and build data storage structures using star schemas, facts, and dimensions in an MPP SWL Pool.
• Data Source Expertise: Extract data from REST APIs, SQL database tables, and CSV files.
• Azure Synapse Analytics Expertise: Optimize notebooks/pipelines for scalability and performance.
• Data Fabric Concepts: Contribute to data lakes, lakehouses, delta lakes, and data cataloging.
• Data Modeling: Work with data architects to create aligned data models and schemas.
• Data Quality: Implement data quality checks and validation processes.
• Performance Tuning: Identify bottlenecks and optimize ETL jobs to meet SLAs.
• Monitoring and Troubleshooting: Monitor ETL jobs, diagnose issues, and implement fixes.
• Documentation: Maintain documentation of ETL processes, data flows, and transformations.
• Collaboration: Work with cross-functional teams to understand data requirements.
• Security and Compliance: Ensure data security and privacy compliance.

Conocimientos

Python (PySpark)

ETL data engineering

SQL / SparkSQL

Data modeling

Data quality & validation

Monitoring & troubleshooting

Communication & collaboration

Educación

Bachelor's degree in Computer Science / IT or related field

Herramientas

Azure Synapse Analytics

Power BI

Tableau

Git / Azure DevOps

Descripción del empleo

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data pipelines using Python (PySpark) within Azure Synapse Analytics. You will ensure accurate, timely data for analytics and collaborate with data scientists, analysts, and stakeholders.

Responsibilities include building scalable ETL processes, data warehousing concepts, data quality checks, and performance tuning across notebooks and pipelines.