Senior Data Engineer - Fleet Sensor Data & AI Infra

Avala; San Francisco (CA); Hybrid; USD 130,000 - 190,000; Full time

Benefits offered by this job

Health care (medical, dental, vision)

FSAs

Unlimited PTO

Equity

Job summary

Avala is seeking a skilled Data Engineer to build the data infrastructure that ingests, processes, and manages petabytes of sensor data from autonomous vehicle and robotics fleets. The role focuses on designing scalable pipelines that transform raw data into high-quality training datasets for Physical AI.

Join a team working on Fleet Learning Infrastructure for Physical AI, emphasizing data quality, scalable storage, and collaboration with ML teams to deliver training-ready datasets.

Qualifications

• BS in Computer Science or equivalent experience.
• 3+ years experience with large-scale data processing systems.
• Proficiency in Python; experience with Spark, Kafka, or similar frameworks.
• Experience with cloud infrastructure (AWS, GCP, or Azure).
• Knowledge of video/image processing and sensor data formats.
• Familiarity with ML training data requirements and formats.

Responsibilities

• Design and implement large-scale data pipelines for fleet sensor data ingestion.
• Build systems to process video, LiDAR, radar, and IMU data streams.
• Develop auto-labeling pipelines using state estimation and 3D reconstruction.
• Create data quality monitoring and validation systems.
• Optimize storage and retrieval for petabyte-scale datasets.
• Collaborate with ML teams to deliver training-ready datasets.

Skills

Python

Spark

Kafka

Cloud platforms

Video processing

Sensor data formats

ML training data

Education

BS in Computer Science or equivalent

Job description

Avala is seeking a skilled Data Engineer to build the data infrastructure that ingests, processes, and manages petabytes of sensor data from autonomous vehicle and robotics fleets. The role focuses on designing scalable pipelines that transform raw data into high-quality training datasets for Physical AI.

Join a team working on Fleet Learning Infrastructure for Physical AI, emphasizing data quality, scalable storage, and collaboration with ML teams to deliver training-ready datasets.