Azure Data Engineer: ETL & Synapse Specialist

Bluelightconsulting; Sur; A distancia; COP 78.120.000 - 122.760.000; Jornada completa

Descripción de la vacante

Bluelightconsulting in Colombia is seeking an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) and Azure Synapse Analytics. The role collaborates with data scientists and analysts to ensure accurate, timely, and accessible analytical data for decision making.

You will implement data warehousing concepts, build scalable notebooks and pipelines, and monitor data quality and performance while contributing to data governance and security.

Formación

• Bachelor’s degree in Computer Science, Information Technology, or related field (or equivalent work experience); Azure Data Engineer certs are a plus.
• Experience with ETL data engineering using Python (PySpark) for data extraction, transformation, and loading from REST APIs, SQL databases, and CSV files.
• Proficiency with Azure Synapse Analytics resources including Notebooks, Pipelines, Linked Services, and Azure Key Vault.
• Ability to write complex SQL queries and work with SparkSQL and MS SQL for data extraction, transformation, and loading.
• Knowledge of data integration best practices and tools; experience with version control (Git, Azure DevOps).
• Strong problem-solving and analytical skills; excellent written and verbal communication; team orientation.

Responsabilidades

• ETL Data Engineering: Develop and maintain ETL pipelines using Python (PySpark) within Azure Synapse (Notebooks/Pipelines) for efficient data extraction, transformation, and loading.
• Data Warehousing: Design data storage structures in an MPP SQL pool with star schemas, facts, and dimensions.
• Data Source Expertise: Extract data from REST APIs, SQL tables, and CSV files.
• Azure Synapse Analytics Expertise: Optimize notebooks/pipelines for scalability and performance.
• Data Fabric Concepts: Support data lakes, lakehouses, delta lakes, and data cataloging.
• Data Modeling: Collaborate to create data models and schemas meeting business needs.
• Data Quality: Implement data quality checks and validation processes.
• Performance Tuning: Identify bottlenecks and improve ETL performance to meet SLAs.
• Monitoring and Troubleshooting: Monitor jobs, diagnose issues, and ensure reliability.
• Documentation: Maintain comprehensive documentation of data flows and transformations.
• Collaboration: Work with cross-functional teams and support data-related initiatives.
• Security and Compliance: Ensure data governance and privacy standards.

Conocimientos

Strong problem-solving skills

Excellent communication skills

Team collaboration

Educación

Bachelor’s degree in Computer Science or related field

Herramientas

Python (PySpark)

Azure Synapse Analytics

SQL (SparkSQL, MS SQL)

Git / Azure DevOps

Power BI / Tableau

Descripción del empleo

Bluelightconsulting in Colombia is seeking an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) and Azure Synapse Analytics. The role collaborates with data scientists and analysts to ensure accurate, timely, and accessible analytical data for decision making.

You will implement data warehousing concepts, build scalable notebooks and pipelines, and monitor data quality and performance while contributing to data governance and security.