Azure Data Engineer (ID: 4098)

Stafide; Amsterdam; On-site; EUR 65,000 - 90,000; Full time

Job summary

Stafide is seeking a data engineer to design, develop, and maintain scalable data pipelines on Azure. You will transform data using Python and orchestrate workflows with Azure Data Factory while leveraging Azure Databricks and Spark for processing.

You will integrate data from multiple sources via APIs, monitor pipelines, and troubleshoot issues in an Agile environment, ensuring reliable, scalable data platforms for analytics and reporting.

Qualifications

• 4–6 years of professional experience in data engineering, Azure data solutions, or related technical roles.
• Hands-on experience with Microsoft Azure data engineering services.
• Strong programming knowledge in Python for data processing and transformation.
• Experience developing and maintaining data pipelines using Azure Data Factory.
• Practical experience with Azure Databricks and Apache Spark.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Microsoft Azure data services.
• Develop data processing and transformation solutions using Python.
• Build, configure, and manage data workflows using Azure Data Factory (ADF).
• Develop and optimize data processing solutions using Azure Databricks and Apache Spark.
• Integrate data from multiple sources using APIs and other data integration techniques.
• Monitor data pipelines, troubleshoot failures, and resolve data processing issues.
• Collaborate with cross-functional teams to understand data requirements and deliver effective technical solutions.
• Follow coding standards, version control practices, testing procedures, and Agile ways of working.
• Maintain technical documentation and support data platform improvements.

Skills

Azure Data Factory

Azure Databricks

Python

APIs

Data pipelines

Azure data services

Apache Spark

Tools

Azure Data Factory

Azure Databricks

Apache Spark

Job description

• Design, develop, and maintain scalable data pipelines using Microsoft Azure data services.
  
• Develop data processing and transformation solutions using Python.
  
• Build, configure, and manage data workflows using Azure Data Factory (ADF).
  
• Develop and optimize data processing solutions using Azure Databricks and Apache Spark.
  
• Integrate data from multiple sources using APIs and other data integration techniques.
  
• Work with Azure Databricks and Azure data services to process, transform, and prepare data for analytics and reporting.
  
• Implement data ingestion, transformation, and loading processes to support business requirements.
  
• Monitor data pipelines, troubleshoot failures, and resolve data processing issues.
  
• Optimize pipeline performance, reliability, and scalability.
  
• Collaborate with cross-functional teams to understand data requirements and deliver effective technical solutions.
  
• Follow coding standards, version control practices, testing procedures, and Agile ways of working.
  
• Maintain technical documentation and support data platform improvements.
  

What You Bring to the Table

• 4-6 years of professional experience in data engineering, Azure data solutions, or related technical roles.
  
• Hands-on experience with Microsoft Azure data engineering services.
  
• Strong programming knowledge in Python for data processing and transformation.
  
• Experience developing and maintaining data pipelines using Azure Data Factory.
  
• Practical experience with Azure Databricks and Apache Spark.
  
• Experience integrating data from external systems using APIs.
  
• Understanding of data ingestion, transformation, ETL/ELT processes, and data integration concepts.
  
• Knowledge of data pipeline monitoring, troubleshooting, and performance optimization.
  
• Familiarity with Agile methodologies and collaborative development practices.
  
• Strong analytical, problem-solving, and communication skills.

You Should Possess the Ability to :

• Develop reliable and scalable data pipelines using Python, Azure Data Factory, and Azure Databricks.
  
• Extract, transform, and load data from different sources into target data platforms.
  
• Build API-based integrations to ingest data from external applications and systems.
  
• Process and transform large datasets using Databricks and Spark.
  
• Troubleshoot pipeline failures, data quality issues, and performance bottlenecks.
  
• Optimize data processing workflows to improve performance and reliability.
  
• Collaborate with technical teams and business stakeholders to translate requirements into data engineering solutions.
  
• Follow established development standards and participate in Agile development activities.
  
• Document data pipelines, integration processes, and technical solutions.

What We Bring to the Table

• An opportunity to work on Azure-based data engineering projects.
  
• Exposure to Python development, cloud data integration, and large-scale data processing.
  
• Opportunities to contribute to data pipeline development and optimization.
  
• Collaboration with technical teams in an Agile working environment.