Data Ingestion Engineer - Spark Pipelines (Hybrid)

Wayve; Sunnyvale (CA); Hybrid; USD 140,000 - 210,000; Full time

Benefits offered by this job

Meaningful equity

Hybrid working

Learning budget

Job summary

Wayve is seeking an engineer to join the Ingestion Systems Team, focused on building data foundations for self-driving technology. You will work with high-volume Spark-based pipelines, optimize throughput, and ensure reliability across multi-step ingestion workflows.

You’ll debug data issues, collaborate with engineers, and help orchestrate complex data flows while learning and contributing in a fast-moving, hybrid environment.

Qualifications

• Strong production experience with Apache Spark and Python.
• Built, operated or debugged large-scale ingestion, ETL or distributed data-processing pipelines.
• Experience with Airflow, Flyte, Databricks, Delta Lake, Scala or Java would be an advantage.

Responsibilities

• Debug and unblock failing ingestion pipelines, stalled jobs and processing queues.
• Investigate corrupt, malformed or unexpected data from a variety of sources.
• Optimise Spark jobs and batch-processing pipelines for throughput, reliability and compute efficiency.
• Work alongside permanent engineers to prioritise critical datasets and reduce operational toil.
• Improve monitoring, retries and failure recovery across multi-step ingestion workflows.

Skills

Apache Spark

Python

ETL pipelines

Airflow

Flyte

Databricks

Delta Lake

Scala

Java

Job description

Wayve is seeking an engineer to join the Ingestion Systems Team, focused on building data foundations for self-driving technology. You will work with high-volume Spark-based pipelines, optimize throughput, and ensure reliability across multi-step ingestion workflows.

You’ll debug data issues, collaborate with engineers, and help orchestrate complex data flows while learning and contributing in a fast-moving, hybrid environment.