data engineer for cloud data platforms

HireHi; Amsterdam; On-site; EUR 70,000 - 100,000; Full time

Benefits offered by this job

Salary plus bonus

Lease car

Laptop and phone

Career development plan

European work option 4 weeks/year

Knowledge-sharing community

Job summary

Devoteam, an AI-driven technology consultancy, seeks a Data Engineer with 2+ years of Databricks experience to design and deploy scalable data pipelines in Amsterdam.

You will extract data from diverse sources, transform with Spark, and load into data lakes and warehouses while upholding governance and security practices. Collaboration with data scientists and BI teams is essential.

Qualifications

• University degree in CS or engineering or ML/AI specialization.

Responsibilities

• Design, develop and deploy robust, scalable data pipelines using Databricks.
• Extract data from databases, APIs and streaming platforms.
• Transform and cleanse data using Spark and load into data lakes and warehouses.
• Optimize pipeline performance for efficiency and cost.
• Use Databricks SQL, Delta Lake, Databricks Runtime, and Databricks Workflows to orchestrate workflows.
• Collaborate with data scientists, analysts, and BI teams; ensure data governance and quality.

Skills

Python

Scala

SQL

Analytical thinking

Problem solving

Education

Bachelor's or Master's in CS/Engineering

Tools

Databricks

PySpark

Spark SQL

Delta Lake

Databricks Runtime

Databricks Workflows

Azure

AWS

GCP

Job description

Devoteam is an AI-driven technology consultancy with more than 11,000 technology experts across 25 countries in Europe and the Middle East. The company partners with Google, Microsoft, AWS, and ServiceNow and specializes in Cloud, Cyber, Data, and AI in the Netherlands.

Задачи

• Design, develop, and deploy robust and scalable data pipelines using Databricks;
• Extract data from databases, APIs, and streaming platforms;
• Transform and cleanse data using Spark and load it into data lakes, data warehouses, and other target systems;
• Optimize pipeline performance for efficiency and cost-effectiveness;
• Use Databricks SQL, Delta Lake, Databricks Runtime, and Databricks Workflows to orchestrate data workflows;
• Integrate Databricks with Azure, AWS, or GCP cloud services for storage, compute, and security;
• Contribute to data models and architectures for data lakes and data warehouses;
• Implement data quality checks and validation procedures throughout data pipelines;
• Follow data governance policies and best practices;
• Collaborate with data scientists, analysts, business intelligence specialists, and business stakeholders;
• Provide technical expertise on Databricks and data engineering best practices;
• Stay up to date with Databricks features, cloud technologies, and data engineering trends;
• Continuously improve solutions and deliver innovative results for clients.

Требования

• Hold a university degree in Computer Science, (Business) Engineering, or a specialized master's in Machine Learning or AI;
• Have 2+ years of experience as a Data Engineer with demonstrated expertise in Databricks;
• Have hands-on experience with at least one major cloud platform: Azure, AWS, or GCP;
• Understand cloud data services and infrastructure;
• Have strong programming skills in Python or Scala;
• Have experience with data manipulation libraries such as PySpark and Spark SQL;
• Understand data warehousing principles, ETL processes, data modeling techniques, and database systems;
• Have advanced SQL skills for data querying, transformation, and analysis;
• Have excellent communication and interpersonal skills;
• Have strong analytical and problem-solving abilities with a proactive approach to data-related challenges;
• Nice to have: Proficiency in Dutch.

Условия

• Excellent primary and secondary employment benefits, including salary, bonus, lease car, laptop, and phone;
• Personalized career development plan with training and certifications;
• Opportunities to grow as a thought leader through knowledge sharing, blogs, and keynote sessions;
• The option to work from within Europe for 4 weeks per year;
• Internal knowledge-sharing community with sessions including Tech Thursdays and KISSes.