Senior Data Platform Engineer – Real-Time Pipelines

Kpler; Scotland; On-site; GBP 120,000 - 180,000; Full time

Job summary

Kpler is seeking a Staff Engineer in the UK to lead data architecture and real-time pipelines. You will drive end-to-end delivery, mentor engineers, and raise the bar on operational excellence across the data platform.

You bring 10+ years in data or software engineering, deep Java knowledge, streaming experience with Kafka/Flink, and cloud expertise across AWS/GCP/Azure, plus collaboration with product and domain stakeholders.

Qualifications

• 10+ years of experience in data or software engineering with DevOps exposure.
• Strong backend/data/distributed systems foundations, with significant Java experience.
• Proven streaming experience with Kafka or Flink.
• Ability to align business priorities with technical delivery across teams.

Responsibilities

• Design, build and maintain real-time and batch data pipelines.
• Improve reliability and observability, and own operational excellence of pipelines.
• Own cloud-based infrastructure (build and run).
• Manage data products with diverse stakeholders.
• Provide strategic and technical leadership, mentoring other engineers.
• Understand complex domain with many cross-team dependencies.
• Own features end-to-end.

Skills

Data/Software Eng

Java

Kafka

Flink

Cloud AWS/GCP/Azure

DevOps exposure

English proficiency

Tools

Spark

Terraform

Pulumi

Job description

Kpler is seeking a Staff Engineer in the UK to lead data architecture and real-time pipelines. You will drive end-to-end delivery, mentor engineers, and raise the bar on operational excellence across the data platform.

You bring 10+ years in data or software engineering, deep Java knowledge, streaming experience with Kafka/Flink, and cloud expertise across AWS/GCP/Azure, plus collaboration with product and domain stakeholders.