Azure Data Engineer: ETL, Synapse & Spark

Bluelightconsulting; Valparaíso; A distancia; CLP 83.955.223 - 121.268.656; Jornada completa

Descripción de la vacante

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) within Azure Synapse. You will collaborate with data scientists and analysts to ensure accurate, timely analytical data for decision making.

The role emphasizes data quality, scalable data notebooks/pipelines, and close collaboration with cross-functional teams in a fast-paced environment. Strong SQL, Azure Synapse, and visualization tool experience are essential.

Formación

• Experience with ETL data engineering using Python (PySpark) to extract, transform, and load from REST APIs, SQL databases, and CSV files.
• Proficiency with Azure Synapse Analytics resources including Notebooks and Pipelines.
• Strong SQL skills, including SparkSQL and MS SQL, for data extraction and transformation.
• Knowledge of data integration best practices and data governance concepts.
• Experience with version control systems such as Git (Azure DevOps).
• Familiarity with data visualization tools (Power BI, Tableau) and Agile methodologies.

Responsabilidades

• ETL Data Engineering: develop and maintain ETL processes using Python (PySpark) in Azure Synapse Notebooks and Pipelines.
• Data Warehousing: design data storage structures using star schemas, facts and dimensions in an MPP SWL pool.
• Data Source Expertise: extract data from REST APIs, SQL tables, and CSV files.
• Azure Synapse Analytics Expertise: design and optimize notebooks and pipelines for performance and scale.
• Data Modeling: collaborate with data architects to create schemas aligned with business needs.
• Monitoring and Troubleshooting: monitor ETL jobs, diagnose issues, and ensure pipeline reliability.
• Documentation and Collaboration: maintain docs and work with cross-functional teams on data initiatives.
• Security and Compliance: ensure data governance and privacy standards are met.

Conocimientos

ETL data engineering

Python (PySpark)

SQL querying

Data modeling

Data governance

Azure Synapse

Communication skills

Problem solving

Agile methodologies

Educación

Bachelor’s degree in Computer Science or related field; or equivalent

Herramientas

Azure Synapse Notebooks

Azure Synapse Pipelines

Git (Azure DevOps)

Azure Key Vault

Power BI

Tableau

Linked Services

Descripción del empleo

Bluelight is seeking an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) within Azure Synapse. You will collaborate with data scientists and analysts to ensure accurate, timely analytical data for decision making.

The role emphasizes data quality, scalable data notebooks/pipelines, and close collaboration with cross-functional teams in a fast-paced environment. Strong SQL, Azure Synapse, and visualization tool experience are essential.