Big Data Engineer (Spark)

Nebul Bv; Netherlands; Hybrid; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Hybrid work near The Hague & Amsterdam

Equity participation

Competitive compensation

Job summary

Nebul is seeking an Apache Spark Specialist to design, deploy, and manage Nebul’s accelerated, fully-managed Spark proposition on Europe’s sovereign AI cloud, enabling customers to extract value from data at scale with top-tier performance and security.

You will architect scalable Spark environments optimized for GPU acceleration and distributed compute, integrate with the Nebul stack, and partner with product and engineering to shape the offering and support early customer deployments.

Qualifications

• 4–7 years of experience with Apache Spark in production.
• Deep knowledge of Spark internals, tuning, and shuffle management.
• Experience deploying Spark in Kubernetes/on-prem/cloud clusters.
• Strong scripting and automation (Python/Scala).
• Familiar with cloud security and IaC practices.

Responsibilities

• Architect, deploy, and operate scalable Spark environments on Nebul's cloud.
• Optimize Spark workloads for GPU-accelerated, distributed performance.
• Define security, monitoring, governance, and data protection practices.
• Collaborate with product, engineering, and customers to shape the offering.
• Develop CI/CD deployment models for reliability and repeatability.
• Document architectures and performance benchmarks.

Skills

Apache Spark

Kubernetes

Python/Scala

GPU acceleration

Databricks / Lakehouse

Security & governance

Terraform IaC

Tools

Databricks

Delta Lake

Airflow

Argo

Job description

about us

About Nebul
At Nebul, we’re building Europe’s sovereign AI cloud — trusted, secure, and purpose-built for the next generation of intelligent infrastructure.

We’re looking for an Apache Spark Specialist to take a leading role in designing, deploying, and managing Nebul’s accelerated, fully-managed Spark proposition — enabling our customers to extract value from data at scale with top-tier performance and security.
You will be responsible for architecting Spark environments optimized for GPU acceleration and distributed compute, ensuring seamless integration with Nebul’s sovereign cloud stack. This role combines deep technical ownership with solution innovation: from deploying high-performance clusters, to shaping the product experience and supporting early customer implementations.

This is not a maintenance role — it is an opportunity to build and evolve a flagship data capability from the ground up.

What You’ll Do

• Architect, deploy, and operate scalable Apache Spark environments on Nebul’s sovereign AI cloud.
• Design and optimize Spark workloads for GPU-accelerated and distributed performance.
• Define and implement best practices for security, monitoring, governance, and data protection.
• Partner closely with product, engineering, and customer teams to shape our managed Spark offering.
• Evaluate and integrate complementary technologies (e.g., Delta Lake, Lakehouse components, tooling).
• Support early customer pilots and translate feedback into roadmap improvements.
• Develop automation and CI/CD deployment models to ensure reliability, repeatability, and efficiency.
• Document architectures, operational procedures, and performance benchmarks.

What We’re Looking For

• 4–7 years of experience working with Apache Spark in production environments.
• Strong deep-dive knowledge of Spark internals: performance tuning, partition strategies, caching, and shuffle management.
• Hands-on deployment experience in Kubernetes , cloud infrastructure, or on-prem clusters.
• Solid understanding of distributed data platforms (e.g., Databricks, EMR, Hadoop, Lakehouse architectures).
• Strong scripting and automation skills (Python / Scala preferred).
• Ability to translate client needs into technical architectures and operational models.
• Familiarity with cloud-security principles and infrastructure-as-code practices.

Bonus Points

• Experience with GPU acceleration for Spark , RAPIDS, or ML workloads.
• Exposure to high-security or regulated environments (government, critical industry).
• Knowledge of observability stacks (Prometheus, Grafana) and orchestration (Airflow, Argo).
• Contributions to open-source or performance engineering work.
• Background in data engineering or MLOps.

Why Nebul

• Build a sovereign European alternative to hyperscalers — with real impact.
• Lead the creation of a key product capability powering AI and data innovation.
• Collaborate with world-class engineers across cloud, security, and high-performance compute.
• Hybrid environment near The Hague and Amsterdam.
• Competitive compensation, growth opportunities, and equity participation in a critical market.

Requirements

• Based in the Netherlands (commutable to Leiden).
• Valid EU work permit (no sponsorship currently available).
• Fluent in English (Dutch not required).