Data Engineer: Real-Time Pipelines & Platform Reliability

ITV plc; Hounslow; Hybrid; GBP 70,000 - 110,000; Full time

Job summary

ITV plc is seeking a Data Engineer within the BDE team to contribute to the Synapse platform, ITV’s operational data engine. We are on an ambitious journey to transition our data landscape to make it fit for future needs which collect and measure user behaviour, ensuring high-quality data powers ITV's core viewer experiences.

This role is roughly balanced across three pillars: 50% feature building, 25% platform & service health, 25% BAU & support.

Qualifications

• Strong experience building data pipelines and streaming solutions.
• Proficient in SQL and at least one of Python, Java or Go.
• Familiar with Terraform and containerized services.
• Experience with GCP and multi-project architectures.

Responsibilities

• Pipeline Engineering: Build and maintain high-throughput streaming and batch pipelines on Dataflow and BigQuery, with Python for orchestration and tooling, utilising Go for the low-latency viewer-facing services, such as continue watching.
• Infrastructure as Code: Deploy and manage services using a lego brick Terraform methodology, with the flexibility to write custom pipelines when standard modules don't fit the use case.
• Testing & Quality: Write unit and integration tests as part of our Definition of Done, and help us move beyond after-the-event validation toward proactive, integrated testing, including load testing ahead of big live events.
• Observability: Set up metrics and alerting for what you ship, using native logging and monitoring dashboards defined in Terraform, alongside Grafana & Prometheus for deeper system metrics.
• Security: Implement and manage fine-grained access through GCP IAM to ensure data integrity and compliance.
• Support & Rota: Take part in the team’s support and on-call rota, triaging support issues and bugs as they are reported. Critical issues with our viewer-facing services are picked up straight away, with everything else prioritised alongside planned engineering work.

Skills

SQL

Python

Java

Go

GCP

Data streaming

BigQuery

Dataflow

Airflow

Terraform

Kubernetes

Tools

Terraform

BigQuery

Dataflow

Airflow/Composer

GKE

Job description

ITV plc is seeking a Data Engineer within the BDE team to contribute to the Synapse platform, ITV’s operational data engine. We are on an ambitious journey to transition our data landscape to make it fit for future needs which collect and measure user behaviour, ensuring high-quality data powers ITV's core viewer experiences.

This role is roughly balanced across three pillars: 50% feature building, 25% platform & service health, 25% BAU & support.