Senior Data Engineer: Scalable Pipelines & Data Governance

Stafide; Amsterdam; On-site; EUR 70,000 - 90,000; Full time

Benefits offered by this job

Opportunity to work on large-scale cloud data platforms

Collaborative environment

Challenging projects

Job summary

A leading data solutions company in Amsterdam is seeking a Data Engineer with over 8 years of experience in architecting and implementing scalable data pipelines using Databricks and Azure Data Factory. The role involves optimizing data ingestion processes, ensuring data quality, and collaborating with cross-functional teams. Ideal candidates will possess advanced Python scripting skills and a strong understanding of data governance principles. This position offers an exciting opportunity to work on large-scale cloud data platforms.

Qualifications

• 8+ years of experience in data engineering or big data environments.
• Strong hands-on experience with Databricks and PySpark.
• Practical experience with Azure Data Factory for data pipeline development.
• Advanced Python scripting skills.
• Solid understanding of data governance principles.
• Experience implementing data quality frameworks.
• Hands-on experience with data lineage tracking.
• Experience with large-scale data processing frameworks.

Responsibilities

• Design, build, and maintain scalable data pipelines using Databricks, PySpark, and Azure Data Factory.
• Develop and optimize data ingestion, transformation, and loading processes.
• Implement data governance frameworks.
• Build data quality validation processes.
• Establish data lineage tracking mechanisms.
• Develop and maintain Python scripts for data workflows.
• Collaborate with data scientists and analysts.
• Optimize data pipeline performance within Azure.

Skills

Databricks

PySpark

Azure Data Factory

Python scripting

Data governance

Data quality frameworks

Data lineage tracking

Job description

A leading data solutions company in Amsterdam is seeking a Data Engineer with over 8 years of experience in architecting and implementing scalable data pipelines using Databricks and Azure Data Factory. The role involves optimizing data ingestion processes, ensuring data quality, and collaborating with cross-functional teams. Ideal candidates will possess advanced Python scripting skills and a strong understanding of data governance principles. This position offers an exciting opportunity to work on large-scale cloud data platforms.