Lead Streaming Platform Engineer

ON.com; Greater London; On-site; GBP 65,000 - 90,000; Full time

Job summary

ON.com is looking for a skilled Streaming Platform Engineer to join our Data Streaming Platform team in Greater London. This hybrid role combines platform, software, and data engineering responsibilities, focusing on building and maintaining a high-performance real-time data streaming platform.

The successful candidate will work with technologies such as Apache Kafka, Flink, and cloud services. If you are passionate about scalable systems and data-driven solutions, we invite you to apply.

Qualifications

• Strong production experience with Apache Kafka and its ecosystem.
• Experience building real-time data pipelines for ML and analytics.
• Hands-on experience with cloud platforms and orchestration tools.

Responsibilities

• Design and maintain infrastructure for data streaming platform.
• Implement data pipelines and optimize stream processing applications.
• Collaborate with teams to enhance data integration and quality.

Skills

Apache Kafka

Apache Flink

Spark Streaming

Python

Kubernetes

Docker

Terraform

Tools

AWS

GCP

Azure

New Relic

Prometheus

Grafana

Job description

We are seeking a highly skilled and motivated Streaming Platform Engineer to join the Data Streaming Platform team. This unique hybrid role combines platform, software, and data engineering to build, scale, and maintain our high-performance real‑time data streaming platform. The ideal candidate has a passion for architecting robust, scalable systems to enable data‑driven products and services at massive scale.

Your Mission

• Design, build, and maintain the core infrastructure for our real‑time data streaming platform, ensuring high availability, reliability, and low latency.
• Implement and optimize data pipelines and stream processing applications using Apache Kafka, Apache Flink, and Spark Streaming.
• Collaborate with software and data engineering teams to define event schemas, ensure data quality, and support integration of new services into the streaming ecosystem.
• Develop and maintain automation and tooling for platform provisioning, configuration management, and CI/CD pipelines.
• Champion the development of self‑service tools and workflows that empower engineers to manage their own streaming data needs, reducing friction and accelerating development.
• Monitor platform performance, troubleshoot issues, and implement observability solutions (metrics, logging, tracing) to ensure the platform’s health and stability.
• Stay up‑to‑date with the latest advancements in streaming and distributed systems technologies and propose innovative solutions to technical challenges.

Qualifications

• Strong production experience with Apache Kafka and its ecosystem (e.g., Confluent Cloud, Kafka Streams, Kafka Connect). Solid understanding of distributed systems and event‑driven architectures.
• Experience building and optimizing real‑time data pipelines for ML, analytics, and reporting, leveraging Apache Flink, Spark Structured Streaming, and low‑latency OLAP systems like Apache Pinot.
• Hands‑on experience with major cloud platforms (AWS, GCP, or Azure), Kubernetes, Docker, Terraform, CI/CD (GitHub Actions), and observability tools (New Relic, Prometheus, Grafana).
• Proficiency in at least one programming language: Python, TypeScript, Java, Scala, or Go.
• Familiarity with data platform concepts, including data lakes and data warehouses.

About the Team

You will be part of a talented and diverse team of data engineers, data scientists, and product managers focused on revolutionizing the use of stream‑processing across the organization. We’re building innovative data solutions to optimize internal processes, enhance customer experiences, and drive business growth.

On is an Equal Opportunity Employer. We are committed to creating a work environment that is fair and inclusive, where all decisions related to recruitment, advancement, and retention are free of discrimination.