Azure Data Engineer (ID: 4069)

Zohorecruit; Amsterdam; On-site; EUR 90,000 - 120,000; Full time

Job summary

Zohorecruit seeks a senior data engineer to design, build, and maintain enterprise-scale data pipelines in Azure. You will leverage ADF, Databricks, and PySpark to deliver robust ETL/ELT solutions and ensure data quality across business domains.

Collaborating with cross-functional teams, you will implement CI/CD pipelines in Azure DevOps, optimize performance, and ensure security/governance standards while staying current with cloud data engineering best practices.

Qualifications

• 8-10 years of professional data engineering or cloud data role.
• Hands-on Azure Data Factory for orchestration and movement.
• Hands-on Azure Databricks for scalable processing.
• Strong Python and PySpark programming skills.
• Proficient in SQL and handling large datasets.
• Experience with ETL/ELT pipelines and data security practices.

Responsibilities

• Design and develop scalable data pipelines using Azure Data Factory and Azure Databricks.
• Develop PySpark applications for processing large data volumes.
• Write clean, maintainable Python-based data solutions.
• Build and manage CI/CD workflows in Azure DevOps.
• Perform data extraction, transformation, loading, validation across sources.

Skills

Azure Data Factory

Azure Databricks

PySpark

Python

SQL

Data Engineering

CI/CD

Tools

Azure DevOps

Job description

• Design, develop, and maintain complex and scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks.
• Build and optimize large-scale data processing solutions using PySpark and Databricks.
• Develop robust ETL/ELT solutions in Python for data extraction, transformation, and loading.
• Collaborate with cross-functional teams to understand data requirements and deliver efficient data engineering solutions.
• Implement and maintain CI/CD pipelines using Azure DevOps for automated testing, integration, and deployment.
• Develop, test, and maintain reliable data solutions aligned with technical and business requirements.
• Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
• Work with large datasets and use SQL for data querying and data management.
• Ensure data solutions follow applicable security, compliance, and engineering best practices.
• Stay updated with the latest trends, technologies, and best practices in Azure cloud data engineering.

What You Bring to the Table:

• 8-10 years of overall professional experience in data engineering, cloud data engineering, or related technology roles.
• Strong hands-on experience with Azure Data Factory for data orchestration, movement, and transformation.
• Strong experience with Azure Databricks for developing and managing scalable data processing solutions.
• Strong programming skills in Python.
• Hands-on experience with PySpark for large-scale data processing.
• Good knowledge of SQL and experience working with large datasets.
• Experience developing and maintaining ETL/ELT data pipelines.
• Understanding of data security, compliance, and cloud data engineering best practices.
• Strong problem-solving and analytical skills.
• Experience working independently as well as collaboratively within cross-functional teams.

You Should Possess the Ability To:

• Design and develop scalable data pipelines using Azure Data Factory and Azure Databricks.
• Develop efficient PySpark applications for processing and transforming large volumes of data.
• Write clean, maintainable, and reusable Python-based data engineering solutions.
• Build and manage automated CI/CD workflows in Azure DevOps.
• Perform data extraction, transformation, loading, validation, and integration across different data sources.
• Optimize data pipelines and Databricks workloads for performance and scalability.
• Monitor pipelines, identify failures or bottlenecks, and implement appropriate solutions.
• Troubleshoot data processing and pipeline-related issues effectively.
• Write SQL queries for data analysis, transformation, and validation.
• Work with stakeholders to understand requirements and translate them into technical data solutions.
• Follow appropriate data security, governance, compliance, and development standards.
• Document technical solutions, data processes, pipelines, and deployment procedures.
• Adapt to new Azure services, cloud data engineering technologies, and industry best practices.

What We Bring to the Table:

• An opportunity to work on enterprise-scale Azure cloud data engineering projects.
• Exposure to Azure Data Factory, Azure Databricks, PySpark, Python, Azure DevOps, and SQL.
• Opportunities to design and optimize large-scale data pipelines and processing frameworks.
• A collaborative environment involving cross-functional technical and business teams.
• Opportunities to contribute to cloud data modernization, automation, and CI/CD initiatives.

Let’s Connect

Want to discuss this opportunity in more detail? Feel free to reach out.