Azure Data Engineer: ETL, Analytics & Cloud Warehousing

Bluelightconsulting; La Plata; A distancia; ARS 1.800.000 - 3.000.000; Jornada completa

Descripción de la vacante

Bluelight is seeking an ETL Data Engineer in Argentina to design, develop, and maintain data integration pipelines using Python (PySpark) and Azure Synapse Analytics. You will ensure accurate, accessible analytical data for stakeholders, collaborating with data scientists and analysts in a fast-paced environment.

You will implement data quality checks, optimize pipelines for performance, and document data flows while adhering to governance and security standards.

Formación

• Experience using Python (PySpark) for ETL from REST APIs, SQL databases and CSV files.
• Proficiency with Azure Synapse resources including Notebooks, Pipelines, Linked Services and Key Vault.
• Strong SQL and data modeling skills; ability to design star schemas and scalable data warehouses.

Responsabilidades

• ETL Data Engineering: Develop and maintain ETL data pipelines using Python (PySpark) within Azure Synapse Analytics Notebooks, and/or Azure Synapse Analytics Pipelines, to ensure efficient data extractions, transformation, and loading.
• Data Warehousing: Design and build effective data storage structures in a Massively Parallel Processing (MPP) SWL Pool.
• Data Source Expertise: Extract data from REST APIs, SWL database tables, and CSV files.
• Azure Synapse Analytics Expertise: Design and optimize data notebooks/pipelines for scalability and performance.
• Data Fabric Concepts: Contribute to data lakes, lakehouses, delta lakes, and data cataloging.
• Data Modeling: Create data models and schemas with business requirements in mind.
• Data Quality: Implement data quality checks and validation processes.
• Performance Tuning: Optimize ETL data notebooks/pipelines to meet SLAs.
• Monitoring and Troubleshooting: Monitor ETL jobs and resolve issues for reliability.
• Documentation: Maintain comprehensive documentation of ETL processes, data flows, and transformations.
• Collaboration: Work with cross-functional teams to understand data requirements.
• Security and Compliance: Ensure data security and privacy.

Conocimientos

ETL Data Engineering

Python (PySpark)

Azure Synapse Analytics

SQL Queries

Data Warehousing

Git (Azure DevOps)

Data Modeling

Data Quality

Azure Notebooks / Pipelines

SparkSQL

Educación

Bachelor’s degree in Computer Science or related field

Azure Data Engineer certification (optional)

Herramientas

Azure Synapse Notebooks

Azure Synapse Pipelines

Linked Services

Azure Key Vault

MS SQL

REST APIs

CSV Files

Power BI

Tableau

Descripción del empleo

Bluelight is seeking an ETL Data Engineer in Argentina to design, develop, and maintain data integration pipelines using Python (PySpark) and Azure Synapse Analytics. You will ensure accurate, accessible analytical data for stakeholders, collaborating with data scientists and analysts in a fast-paced environment.

You will implement data quality checks, optimize pipelines for performance, and document data flows while adhering to governance and security standards.