Senior Data Engineer, PB-scale Transfer Hub

AI Startups UK; Greater London; Hybrid; GBP 100,000 - 150,000; Full time

Benefits offered by this job

Hybrid work model

Office in London

Job summary

Wayve in London is seeking a Senior Software Engineer, Data to help build the Data Transfer Hub, a mission-critical system that moves petabytes of sensor data daily from partners around the world. You will join a small, high-autonomy team delivering a scalable, hub-and-spoke data exchange infrastructure.

The role requires strong Python and Terraform skills, Flyte or similar workflow tools, and deep cloud experience with Azure.

Qualifications

• Proven experience building large-scale data transfer pipelines.
• Strong software engineering foundations.
• Production systems delivery & maintenance experience.
• Experience with workflow orchestration (Flyte preferred).
• Observable data systems with retry and observability.
• Azure cloud expertise; AWS/GCP a plus.
• Python and Terraform proficiency.
• Experience with blob storage and large file transfers.
• Product-minded, customer-focused, ownership-driven.
• Comfort in fast-changing environments.

Responsibilities

• Build and operate large-scale data transfer pipelines from partners.
• Contribute to Data Transfer Hub design and trade-offs.
• Implement pipelines with Flyte and Kafka; use Azure storage.
• Develop hub-and-spoke data transfer model at scale.
• Write infrastructure as code in Terraform; Python pipelines.
• Establish reliability: retries, backfills, data checks.
• Work on cross-cloud networking with Azure as primary.
• Participate in on-call rotation.
• Collaborate with product, commercial, and partners.
• Adapt to evolving requirements while upholding standards.

Skills

Large-scale data transfer pipelines

Azure cloud

Python

Terraform

Kafka

Flyte

Observability & reliability

Blob storage / large file transfers

Data engineering

Ownership & product mindset

Tools

Flyte

Terraform

Python

Azure

Kafka

Job description

Wayve in London is seeking a Senior Software Engineer, Data to help build the Data Transfer Hub, a mission-critical system that moves petabytes of sensor data daily from partners around the world. You will join a small, high-autonomy team delivering a scalable, hub-and-spoke data exchange infrastructure.

The role requires strong Python and Terraform skills, Flyte or similar workflow tools, and deep cloud experience with Azure.