Azure Data Engineer: ETL, PySpark & Synapse Expert

Bluelightconsulting; Tacna; A distancia; PEN 60.000 - 90.000; Jornada completa

Descripción de la vacante

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data integration processes using Python (PySpark) within Azure Synapse Analytics. You will work with data scientists, analysts, and stakeholders to deliver clean, reliable data for insights and decision-making in a fast-paced environment.

Responsibilities include building ETL pipelines, data warehousing, data modeling, and ensuring data quality and security.

Formación

• Bachelor's degree in Computer Science, Information Technology, or related field; or equivalent experience with related certifications.
• Proven ETL data engineering experience using Python (PySpark) with REST APIs, SQL databases and CSV files.
• Proficiency with Azure Synapse Analytics resources including Notebooks and Pipelines.
• Strong SQL skills (SparkSQL and MS SQL) for data extraction and transformation.
• Experience with Git/Azure DevOps or similar version control.
• Strong problem-solving and communication skills; team collaboration.
• Familiarity with big data, ML, and data visualization tools (Power BI/Tableau) is a plus.

Responsabilidades

• ETL Data Engineering: Develop and maintain ETL data pipelines using Python (PySpark) in Azure Synapse Analytics to ensure reliable data flows.
• Data Warehousing: Design and build scalable data storage structures in an MPP SQL environment with star schemas.
• Data Source Expertise: Extract data from REST APIs, SQL databases, and CSV files.
• Azure Synapse Analytics: Optimize notebooks and pipelines for performance and scalability.
• Data Fabric Concepts: Contribute to data lakes, lakehouses, delta lakes, and data cataloging initiatives.
• Data Modeling: Collaborate with data architects to create useful data models and schemas.
• Data Quality: Implement data quality checks and validations.
• Performance Tuning: Identify bottlenecks and optimize ETL notebooks/pipelines to meet SLAs.
• Monitoring and Troubleshooting: Monitor jobs, diagnose issues, and implement fixes.
• Documentation: Maintain comprehensive docs for ETL processes, data flows, and transformations.
• Collaboration: Work with cross-functional teams to understand data requirements and support data initiatives.
• Security and Compliance: Ensure data governance and privacy standards are met.

Conocimientos

Python (PySpark)

SQL (SparkSQL/MS SQL)

Data Modeling

Data Warehousing

ETL Design

Big Data Concepts

Communication

Problem-solving

Educación

Bachelor’s degree in Computer Science / IT or related field

Herramientas

Azure Synapse Analytics

Azure Notebooks/Pipelines

Git / Azure DevOps

Power BI

Tableau

Descripción del empleo

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data integration processes using Python (PySpark) within Azure Synapse Analytics. You will work with data scientists, analysts, and stakeholders to deliver clean, reliable data for insights and decision-making in a fast-paced environment.

Responsibilities include building ETL pipelines, data warehousing, data modeling, and ensuring data quality and security.