Software Engineer, Fleet Data Pipelines

Avala; San Francisco (CA); Hybrid; USD 130,000 - 190,000; Full time

Benefits offered by this job

Health care (medical, dental, vision)

FSAs

Unlimited PTO

Equity

Job summary

Avala is seeking a skilled Data Engineer to build the data infrastructure that ingests, processes, and manages petabytes of sensor data from autonomous vehicle and robotics fleets. The role focuses on designing scalable pipelines that transform raw data into high-quality training datasets for Physical AI.

Join a team working on Fleet Learning Infrastructure for Physical AI, emphasizing data quality, scalable storage, and collaboration with ML teams to deliver training-ready datasets.

Qualifications

• BS in Computer Science or equivalent experience.
• 3+ years experience with large-scale data processing systems.
• Proficiency in Python; experience with Spark, Kafka, or similar frameworks.
• Experience with cloud infrastructure (AWS, GCP, or Azure).
• Knowledge of video/image processing and sensor data formats.
• Familiarity with ML training data requirements and formats.

Responsibilities

• Design and implement large-scale data pipelines for fleet sensor data ingestion.
• Build systems to process video, LiDAR, radar, and IMU data streams.
• Develop auto-labeling pipelines using state estimation and 3D reconstruction.
• Create data quality monitoring and validation systems.
• Optimize storage and retrieval for petabyte-scale datasets.
• Collaborate with ML teams to deliver training-ready datasets.

Skills

Python

Spark

Kafka

Cloud platforms

Video processing

Sensor data formats

ML training data

Education

BS in Computer Science or equivalent

Job description

Posted 10 months ago San Francisco, CA full-time

About This Role

Build the data infrastructure that ingests, processes, and manages petabytes of sensor data from autonomous vehicle and robotics fleets. Design scalable pipelines that transform raw fleet data into high-quality training datasets for Physical AI.

Key Responsibilities

• Design and implement large-scale data pipelines for fleet sensor data ingestion
• Build systems to process video, LiDAR, radar, and IMU data streams
• Develop auto-labeling pipelines using state estimation and 3D reconstruction
• Create data quality monitoring and validation systems
• Optimize storage and retrieval for petabyte-scale datasets
• Collaborate with ML teams to deliver training-ready datasets

Requirements

• BS in Computer Science or equivalent experience
• 3+ years experience with large-scale data processing systems
• Proficiency in Python; experience with Spark, Kafka, or similar frameworks
• Experience with cloud infrastructure (AWS, GCP, or Azure)
• Knowledge of video/image processing and sensor data formats
• Familiarity with ML training data requirements and formats

What We Offer

Competitive salary and meaningful equity in the team building Fleet Learning Infrastructure for Physical AI

Full health, dental, and vision coverage, plus FSA and unlimited PTO

Work on the closed-loop data infrastructure that the next generation of autonomous systems learns from

A team that built Tesla Autopilot's data infrastructure, now building something bigger