Azure Data Engineer: ETL, PySpark & Synapse

Bluelightconsulting; Bogotá; A distancia; COP 60.000.000 - 120.000.000; Jornada completa

Descripción de la vacante

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data integration processes using PySpark within Azure Synapse Analytics. You will work with data scientists and analysts to ensure accurate, accessible analytical data and scalable pipelines.

The role emphasizes data quality, governance, and collaboration across cross-functional teams in a fast-paced environment.

Formación

• Bachelor’s degree in Computer Science, Information Technology, or related field; or equivalent work experience.
• Proven experience in ETL data engineering with Python (PySpark) for data extraction, transformation, and loading from REST APIs, SQL tables, and CSV files.
• Proficiency in Azure Synapse Analytics resources including Notebooks, Pipelines, Linked Services, and Azure Key Vault.
• Strong SQL skills with SparkSQL and MS SQL for data extraction, transformation, and load.
• Experience with Git/Azure DevOps and data integration best practices.
• Excellent problem-solving, analytical, and communication skills.

Responsabilidades

• ETL data engineering: develop and maintain ETL processes using PySpark within Azure Synapse.
• Data warehousing: design storage structures with star schemas and facts/dimensions.
• Data source expertise: extract data from REST APIs, SQL tables, and CSVs.
• Azure Synapse Analytics: design and optimize notebooks/pipelines for performance.
• Data fabric concepts: contribute to data lakes, lakehouses, delta lakes, and cataloging.
• Data modeling: collaborate with data architects on data models.
• Data quality: implement checks to ensure accuracy and consistency.
• Performance tuning: identify bottlenecks and optimize ETL pipelines.
• Monitoring: monitor jobs, diagnose issues, ensure reliability.
• Documentation: maintain docs of ETL processes and data flows.
• Collaboration: work with cross-functional teams on data requirements.
• Security and compliance: ensure data governance and privacy standards.

Conocimientos

Python (PySpark)

Analytical thinking

Communication

Data modeling

Data quality

Performance tuning

Data integration

Educación

Bachelor’s degree in Computer Science or related field

Azure Data Engineer certification (optional)

Herramientas

Azure Synapse Analytics

Notebooks

Pipelines

Linked Services

Azure Key Vault

Git (Azure DevOps)

SparkSQL

MS SQL

Power BI

Tableau

Descripción del empleo

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data integration processes using PySpark within Azure Synapse Analytics. You will work with data scientists and analysts to ensure accurate, accessible analytical data and scalable pipelines.

The role emphasizes data quality, governance, and collaboration across cross-functional teams in a fast-paced environment.