Azure Data Engineer - Pipelines, Databricks & PySpark

Ubique Systems

Amsterdam

On-site

EUR 70,000 - 100,000

Full time

4 hours ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Ubique Systems in Amsterdam is seeking a data engineer to design, build and maintain data pipelines in Azure cloud. You will work with PySpark, Databricks and Python to deliver scalable ETL solutions and optimize performance.

You will collaborate with cross-functional teams, implement CI/CD with Azure DevOps, and ensure data security and quality according to standards. Prior experience with Azure Data Factory is essential.

Qualifications

Experience with Azure Data Factory to orchestrate data movement and transformation.
Proficiency in Python and PySpark for big data processing.
Hands-on with Azure Databricks for building scalable data pipelines.
Familiarity with SQL for querying and managing large datasets.

Responsibilities

Design, develop, and maintain complex data pipelines using Azure Data Factory and Azure Databricks.
Build and optimize large-scale data processing solutions using PySpark and Databricks.
Collaborate with cross-functional teams to understand data requirements and deliver efficient data solutions.
Implement CI/CD pipelines in Azure DevOps for automated testing, integration, and deployment of data solutions.
Develop, test, and maintain robust data solutions in Python for ETL processes.
Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
Ensure data security and compliance with company standards and best practices.
Stay updated with the latest trends and best practices in cloud data engineering and Azure services.