Azure Data Engineer - PySpark & Synapse ETL

Visa Hunt; Mexico; Remote; PHP 4,869,000 - 7,304,000; Full time

Job summary

Bluelight is expanding its data engineering team and seeks an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) within Azure Synapse Analytics.

You will work closely with data scientists and analysts, building scalable data models, ensuring data quality, and delivering reliable data for analytics across projects in a fast-paced environment.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field; or equivalent work experience, with Azure Data Engineer certs a plus.
• Proven experience in ETL data engineering with Python (PySpark) to extract, transform, and load from REST APIs, SQL databases, and CSV files.
• Proficiency with Azure Synapse Analytics resources including Notebooks, Pipelines, Linked Services, and Key Vault.
• Strong SQL skills with SparkSQL and MS SQL in data extraction, transformation, and loading.
• Familiarity with data integration best practices, tools, and version control (Git/Azure DevOps).
• Excellent problem-solving and analytical abilities; strong communication and teamwork.

Responsibilities

• ETL Data Engineering: develop and maintain ETL processes using Python (PySpark) within Azure Synapse Notebooks and Pipelines.
• Data Warehousing: design data storage structures using star schema concepts for scalable analytics.
• Data Source Integration: extract data from REST APIs, SQL tables, and CSV files.
• Azure Synapse Analytics: optimize notebooks and pipelines for performance and scalability.
• Data Modeling: collaborate with data architects to create robust data models.
• Data Quality: implement data quality checks and validations.
• Monitoring and Troubleshooting: monitor ETL jobs, diagnose issues, and ensure reliability.
• Documentation: maintain thorough documentation of data workflows and transformations.
• Security and Compliance: uphold data governance and privacy standards in all processes.

Skills

Python (PySpark)

SQL

Azure Synapse Analytics

ETL

REST APIs

SparkSQL

MS SQL

Power BI

Tableau

Git (Azure DevOps)

Data Warehousing

Communication skills

Education

Bachelor’s degree in Computer Science or related field

Tools

Azure Synapse Notebooks

Azure Synapse Pipelines

Linked Services

Azure Key Vault

Job description

Bluelight is expanding its data engineering team and seeks an ETL Data Engineer to design, develop, and maintain data integration pipelines using Python (PySpark) within Azure Synapse Analytics.

You will work closely with data scientists and analysts, building scalable data models, ensuring data quality, and delivering reliable data for analytics across projects in a fast-paced environment.