Lead Data Platform Engineer — Petabyte-Scale Media

Build AI; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Competitive pay

Medical, dental, and vision packages

Housing subsidy for SF or Shenzhen

Job summary

Build AI, based in San Francisco, seeks a lead for the data platform focused on on-device capture, scalable ingestion, and training-ready datasets. You will manage end-to-end data paths, codec decisions, and storage strategies for large-scale physical AI datasets.

You will work across teams in SF and Shenzhen, with emphasis on performance, cost, and reliability, delivering datasets to external research customers and partners.

Qualifications

• Must have owned a production media or sensor data path at real scale (petabyte-scale object storage, codecs, compression, uploads under flaky bandwidth).
• Experience with data paths used by large-scale video systems (e.g., Autopilot/Waymo-like) for training datasets.

Responsibilities

• Own data platform end-to-end: on-device capture, upload, object storage, training-ready shards.
• Balance compression, codecs, and storage for cost-effective 1080p30 hours of data.
• Ensure robust uploads with on-device buffering, batching, retries under unstable networks.
• Define dataset packaging, versioning, and delivery to external research customers.
• Collaborate with firmware teams to align ingest contracts across devices.

Skills

Production data path

Python

Systems programming

Linux

Tools

Object storage

Video codecs

Job description

Build AI, based in San Francisco, seeks a lead for the data platform focused on on-device capture, scalable ingestion, and training-ready datasets. You will manage end-to-end data paths, codec decisions, and storage strategies for large-scale physical AI datasets.

You will work across teams in SF and Shenzhen, with emphasis on performance, cost, and reliability, delivering datasets to external research customers and partners.