Senior Big Data Engineer - Python & Spark Pipelines

La Fosse; Brussel Hoofdstad; On-site; EUR 65,000 - 95,000; Full time

Job summary

La Fosse in Belgium is seeking a Big Data Engineer to develop, integrate, and optimise large-scale data flows in close collaboration with data scientists. You will design robust data pipelines using Hadoop, Spark, Hive, Airflow, and modern data technologies, performing batch ETL and API integration.

The role emphasizes hands-on data cleansing and orchestration, with experience in cloud architectures (AWS/Azure) and full-time availability. This position is based in Belgium.

Qualifications

• Experience with real-time architectures and event streaming infrastructure is advantageous.

Responsibilities

• Collaborate with data scientists to develop effective technical solutions.
• Perform data cleansing, preparation and integration.
• Design and build robust data pipelines.
• Manage component orchestration.
• Handle batch Spark ETL, including wrangling, joining, and aggregation.

Skills

Collaboration

Analytical

Organised

Problem Solving

Data cleansing

Pipeline design

System integration

Tools

Hadoop

Spark

Hive

Airflow

Docker

Kubernetes

Python

API development

AWS

Azure

Job description

La Fosse in Belgium is seeking a Big Data Engineer to develop, integrate, and optimise large-scale data flows in close collaboration with data scientists. You will design robust data pipelines using Hadoop, Spark, Hive, Airflow, and modern data technologies, performing batch ETL and API integration.

The role emphasizes hands-on data cleansing and orchestration, with experience in cloud architectures (AWS/Azure) and full-time availability. This position is based in Belgium.