Azure Data Engineer: Build Scalable Pipelines with Python

Stafide; Amsterdam; On-site; EUR 65,000 - 90,000; Full time

Job summary

Stafide is seeking a data engineer to design, develop, and maintain scalable data pipelines on Azure. You will transform data using Python and orchestrate workflows with Azure Data Factory while leveraging Azure Databricks and Spark for processing.

You will integrate data from multiple sources via APIs, monitor pipelines, and troubleshoot issues in an Agile environment, ensuring reliable, scalable data platforms for analytics and reporting.

Qualifications

• 4–6 years of professional experience in data engineering, Azure data solutions, or related technical roles.
• Hands-on experience with Microsoft Azure data engineering services.
• Strong programming knowledge in Python for data processing and transformation.
• Experience developing and maintaining data pipelines using Azure Data Factory.
• Practical experience with Azure Databricks and Apache Spark.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Microsoft Azure data services.
• Develop data processing and transformation solutions using Python.
• Build, configure, and manage data workflows using Azure Data Factory (ADF).
• Develop and optimize data processing solutions using Azure Databricks and Apache Spark.
• Integrate data from multiple sources using APIs and other data integration techniques.
• Monitor data pipelines, troubleshoot failures, and resolve data processing issues.
• Collaborate with cross-functional teams to understand data requirements and deliver effective technical solutions.
• Follow coding standards, version control practices, testing procedures, and Agile ways of working.
• Maintain technical documentation and support data platform improvements.

Skills

Azure Data Factory

Azure Databricks

Python

APIs

Data pipelines

Azure data services

Apache Spark

Tools

Azure Data Factory

Azure Databricks

Apache Spark

Job description

Stafide is seeking a data engineer to design, develop, and maintain scalable data pipelines on Azure. You will transform data using Python and orchestrate workflows with Azure Data Factory while leveraging Azure Databricks and Spark for processing.

You will integrate data from multiple sources via APIs, monitor pipelines, and troubleshoot issues in an Agile environment, ensuring reliable, scalable data platforms for analytics and reporting.