Senior Data Engineer

Impala Search Ltd; Nordrhein-Westfalen; Hybrid; EUR 90.000 - 130.000; Vollzeit

Zusammenfassung

Impala Search Ltd in Hamburg, Germany, is seeking a Senior Data Engineer to design and implement real-time data pipelines powering AI systems at scale. You will own feature engineering infrastructure and collaborate with Data Scientists and Backend Engineers to deliver robust data platforms.

You will work on TB-scale datasets, leverage Apache Flink, Kafka and Kubernetes, and help shape the overall data architecture while relocating to Hamburg with hybrid onsite work.

Qualifikationen

• 5+ years of commercial Data Engineering experience.
• Proficient in Apache Flink, Kafka, and Java.
• Experience with AWS, Airflow, dbt, Terraform and Kubernetes in production.

Aufgaben

• Build and scale real-time streaming data pipelines supporting production ML systems.
• Develop and enhance feature engineering infrastructure used by Data Scientists.
• Improve data quality through monitoring, validation and governance.
• Optimise large-scale ETL and streaming workloads for performance and reliability.
• Collaborate closely with Data Science and Backend Engineering teams.
• Build ingestion frameworks for new and evolving data sources.
• Help shape the future architecture of the company's cloud data platform.
• Take ownership of technical decisions and platform scalability.

Kenntnisse

Apache Flink

Kafka

Java

Go/Python

AWS

Airflow

dbt

Terraform

Kubernetes

Data Lakes

Feature Stores

Lakehouse

Real-time streaming

Jobbeschreibung

Our client is building the intelligence layer behind one of the world's largest rewarded advertising platforms, helping 770+ million users discover and engage with new mobile apps every year. Their machine learning systems make 200+ million decisions daily, processing 100,000+ predictions per second from a 1PB+ behavioural data platform. Backed by a $100 million strategic investment, they're expanding the engineering team responsible for the real-time data infrastructure powering next-generation AI systems.

You'll join as a Senior Data Engineer, helping build one of the industry's highest-scale ML data platforms. You'll engineer real-time pipelines, optimise distributed data systems, and enable intelligent decision-making across infrastructure handling 100,000+ predictions every second.

Key Responsibilities

• Build and scale real-time streaming data pipelines supporting production ML systems.
• Develop and enhance feature engineering infrastructure used by Data Scientists.
• Improve data quality through monitoring, validation and governance.
• Optimise large-scale ETL and streaming workloads for performance and reliability.
• Collaborate closely with Data Science and Backend Engineering teams.
• Build ingestion frameworks for new and evolving data sources.
• Help shape the future architecture of the company's cloud data platform.
• Take ownership of technical decisions and platform scalability.

Qualifications

• 5+ years of commercial Data Engineering experience building production data platforms.
• Strong experience with Apache Flink, Kafka, and real-time streaming architectures.
• Professional Java development experience, with Go or Python considered a bonus.
• Experience processing TB-scale datasets and high-throughput event streams.
• Hands-on experience with AWS, Airflow, dbt, Terraform, and Kubernetes.
• Experience working with Machine Learning or Data Science teams in production environments.
• Understanding of Data Lakes, Feature Stores, Lakehouse architecture, and modern data modelling.
• Comfortable owning architecture and working across multiple engineering teams.

Location: Hamburg, Germany (Hybrid, 3 days onsite) with full relocation support available.

If you're excited by the opportunity to build the data platform behind AI systems serving hundreds of millions of users worldwide, while owning architecture and solving genuinely complex engineering challenges at scale, we'd love to hear from you.