Azure Data Engineer: ETL & Data Warehousing Pro

Bluelightconsulting; Antofagasta; A distancia; CLP 18.000.000 - 36.000.000; Jornada completa

Descripción de la vacante

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data pipelines using Python (PySpark) within Azure Synapse Analytics. You will work with data scientists, analysts, and stakeholders to deliver high-quality data for insights and decision-making in a fast-paced environment.

The role emphasizes data modeling, data quality checks, and performance tuning of ETL notebooks and pipelines, with collaboration across cross-functional teams and a focus on security and governance.

Formación

• Bachelor’s degree in Computer Science, Information Technology, or equivalent; certifications in data engineering or data science are a plus.

Responsabilidades

• ETL Data Engineering: Develop and maintain ETL data pipelines using Python (PySpark) within Azure Synapse Analytics Notebooks/Pipelines.
• Data Warehousing: Design and build data storage structures in an MPP SWL Pool with star schemas, facts, and dimensions.
• Data Source Expertise: Extract data from REST APIs, SQL tables, and CSV files.
• Azure Synapse Analytics Expertise: Design and optimize notebooks/pipelines for scalability and performance.
• Data Fabric Concepts: Contribute to data lakes, lakehouses, delta lakes, and data cataloging initiatives.
• Data Modeling: Collaborate with data architects to create appropriate data models and schemas.
• Data Quality: Implement checks and validation to ensure accuracy and consistency.
• Performance Tuning: Identify bottlenecks and optimize ETL notebooks/pipelines to meet SLAs.
• Monitoring and Troubleshooting: Monitor ETL jobs, diagnose issues, and implement reliable solutions.
• Documentation: Maintain documentation of ETL processes, data flows, and transformations.
• Collaboration: Work with cross-functional teams on data requirements.
• Security and Compliance: Ensure data security and governance compliance.

Conocimientos

Python (PySpark)

SQL querying

Data modeling

Data warehousing

Data quality checks

Problem solving

Communication skills

Educación

Bachelor’s degree in CS/IT or equivalent

Herramientas

Azure Synapse Analytics

Notebooks

Pipelines

Linked Services

Azure Key Vault

Git/Azure DevOps

Power BI

Tableau

Descripción del empleo

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data pipelines using Python (PySpark) within Azure Synapse Analytics. You will work with data scientists, analysts, and stakeholders to deliver high-quality data for insights and decision-making in a fast-paced environment.

The role emphasizes data modeling, data quality checks, and performance tuning of ETL notebooks and pipelines, with collaboration across cross-functional teams and a focus on security and governance.