Senior Data Engineer

LanceSoft Europe; Amsterdam; On-site; EUR 90,000 - 130,000; Full time

Job summary

LanceSoft Europe is seeking an experienced data engineer in Amsterdam to design and operate production-grade data pipelines, integrating data from lakes and warehouses on cloud platforms.

The role requires deep hands-on experience with Spark, Airflow, PySpark, Python and SQL, plus strong SQL and data modelling skills. You will work with BigQuery/GCP and have exposure to Azure or AWS in a collaborative team.

Qualifications

• At least 8 years of relevant experience in data engineering, cloud engineering, data analytics or a closely related technical field.
• Proven hands-on experience designing and building complex production-grade data pipelines using ETL patterns.
• Strong programming skills in Python and SQL, with practical experience using PySpark for distributed data processing.
• Hands-on experience with Apache Spark and workflow orchestration tools such as Apache Airflow.
• Experience with cloud data platforms, preferably Google Cloud Platform and BigQuery, and working knowledge of Azure or AWS.
• Strong understanding of data modelling, normalization and denormalization, data integration, data lakes and data warehouses.
• Experience with relational and non-relational data stores, including technologies such as MS SQL, Hive and dbt, plus object storage such as Amazon S3.
• Experience with containerization, orchestration and automated delivery tooling, including Docker, Kubernetes and CI/CD pipelines.
• Experience managing and optimizing distributed systems and clusters for large-scale batch processing.
• Good command of software engineering practices, including modular design, testing, code review, documentation, security and production support.

Skills

Data engineering

Cloud engineering

Data analytics

ETL patterns

Python programming

SQL programming

Distributed data processing

Software engineering practices

Data modelling

Data warehousing

CI/CD pipelines

Docker

Kubernetes

PySpark

Tools

Apache Spark

Airflow

Google Cloud Platform

BigQuery

Azure

AWS

dbt

MS SQL

Hive

S3

Docker

Kubernetes

Job description

- At least 8 years of relevant experience in data engineering, cloud engineering, data analytics or a closely related technical field.

- Proven hands-on experience designing and building complex production-grade data pipelines using ETL and ETL patterns.

- Strong programming skills in Python and SQL, with practical experience using PySpark for distributed data processing.

- Hands-on experience with Apache Spark and workflow orchestration tools such as Apache Airflow.

- Experience with cloud data platforms, preferably Google Cloud Platform and BigQuery, and working knowledge of Azure or AWS.

- Strong understanding of data modelling, normalization and denormalization, data integration, data lakes and data warehouses.

- Experience with relational and non-relational data stores, including technologies such as MS SQL, Hive and dbt, plus object storage such as Amazon S3.

- Experience with containerization, orchestration and automated delivery tooling, including Docker, Kubernetes and CI/CD pipelines.

- Experience managing and optimizing distributed systems and clusters for large-scale batch processing.

- Good command of software engineering practices, including modular design, testing, code review, documentation, security and production support.