Senior Software Engineer, Data — Scale PB/Day Data Hub

Wayve; New York (NY); On-site; USD 120,000 - 186,000; Full time

Job summary

Wayve is seeking a Senior Software Engineer, Data to build the Data Transfer Hub, enabling petabyte-scale data movement from global partners and scalable AI model training for autonomousVehicles.

You will join a small, high-autonomy team, design and operate large-scale pipelines with Flyte, Kafka, and Azure, and own observability, reliability, and on-call rotations in a fast-changing, customer-focused environment.

Qualifications

• Proven experience building and operating large-scale data transfer or data processing pipelines (multi-TB/PB scale).
• Strong software engineering fundamentals and production systems ownership.
• Experience with workflow orchestration for parallel jobs (Flyte preferred; Airflow/Prefect also relevant).
• Experience with cloud platforms (Azure preferred; AWS/GCP a plus).
• Proficiency in Python and Terraform for infrastructure and data tooling.
• Experience with blob/storage and cloud-to-cloud data transfer.

Responsibilities

• Build and operate large-scale data transfer pipelines for video, LiDAR and sensor data from global partners at PB/day scale.
• Contribute to the Data Transfer Hub design, balancing UX, reliability, throughput, cost and observability.
• Build distributed pipelines using Flyte and Kafka; leverage Azure for storage and transfer infrastructure.
• Develop a globally distributed hub-and-spoke data transfer model with partners.
• Write infrastructure-as-code in Terraform and pipeline logic in Python.
• Establish reliable operational foundations: retries, backfills, data integrity checks, dashboards and alerts.
• Work on cloud-to-cloud networking with Azure as the primary platform.
• Participate in on-call rotation after establishment.
• Collaborate with product, commercial and partner teams to solve real customer problems.
• Adapt to evolving requirements while upholding engineering standards.

Skills

Python

Terraform

Azure

AWS

GCP

Distributed systems

Observability

Data transfer pipelines

On-call readiness

Tools

Flyte

Airflow

Kafka

Azure storage

Job description

Wayve is seeking a Senior Software Engineer, Data to build the Data Transfer Hub, enabling petabyte-scale data movement from global partners and scalable AI model training for autonomousVehicles.

You will join a small, high-autonomy team, design and operate large-scale pipelines with Flyte, Kafka, and Azure, and own observability, reliability, and on-call rotations in a fast-changing, customer-focused environment.