Senior Data Systems Engineer - Global PB-Scale Pipelines

Wayve; Greater London; On-site; GBP 110,000 - 160,000; Full time

Benefits offered by this job

Hybrid work policy

Job summary

Wayve in London is hiring a Senior Software Engineer, Data to build the Data Transfer Hub, a globally distributed system that will move petabytes of sensor data per day from partner networks. You will own end-to-end data pipelines, collaboration with partners, and on-call rotations in a fast-changing environment.

You will work with Flyte, Kafka, Terraform and Python on Azure to scale data exchange for Wayve's AI training at scale.

Qualifications

• Experience building and operating large-scale data transfer pipelines.
• Strong software engineering skills.
• Experience delivering and maintaining production systems.
• Experience with workflow orchestration for parallelised jobs (Flyte preferred; Airflow/Prefect also relevant).
• Experience building reliable, observable data systems (retry strategies, backfills, data integrity checks, lifecycle/state tracking, alerting).
• Strong cloud platform skills (Azure preferred; AWS/GCP a plus).
• Proficiency in Python and Terraform.

Responsibilities

• Build and operate large-scale data transfer pipelines for video, LiDAR and sensor data from global partners at PB/day scale.
• Contribute to the technical design of the Data Transfer Hub across UX, reliability, throughput and cost.
• Write infrastructure-as-code in Terraform and pipeline logic in Python.
• Build reliable operational foundations, including retries, backfills, data integrity checks and dashboards.
• Collaborate with product and partner teams to solve real customer problems.

Skills

Data pipelines

Python

Terraform

Azure

Kafka/Flyte

Observability & monitoring

Cloud data transfer

Tools

Flyte

Kafka

Azure

Terraform

Job description

Wayve in London is hiring a Senior Software Engineer, Data to build the Data Transfer Hub, a globally distributed system that will move petabytes of sensor data per day from partner networks. You will own end-to-end data pipelines, collaboration with partners, and on-call rotations in a fast-changing environment.

You will work with Flyte, Kafka, Terraform and Python on Azure to scale data exchange for Wayve's AI training at scale.