Lead Data Platform Engineer - Petabyte-Scale Media

Build AI

San Francisco (CA)

On-site

USD 170,000 - 290,000

Full time

11 days ago

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Competitive pay

Medical, dental, and vision packages

Housing subsidy

Relocation support

Wellness benefits

Daily lunch and dinner

Unlimited compute budget

Unlimited Codex and Claude credits

Travel

Job summary

Build AI is hiring a lead for the data platform to manage on-device capture, upload under flaky bandwidth, and training-ready shards for research customers. The role spans compression, storage-tier decisions, and end-to-end dataset packaging and delivery.

You will collaborate with Shenzhen firmware to standardize ingest contracts and help scale across sites worldwide. The team values engineers who can work across engineering and research, shipping data paths at petabyte scale and beyond in a

Qualifications

Strong software engineering
Python and at least one systems language
Linux
You measure cost and throughput, not whether the demo uploaded
Want to scale in-the-wild physical-labor video, not run a generic data org

Responsibilities

Own the data platform end-to-end: on-device capture, upload under flaky bandwidth, object storage, training-ready shards
Compression, codecs, and storage-tier trade-offs so 1080p30 hours stay cheap enough to keep collecting
Upload that survives bad networks: on-device buffering, batching, retries, a drop rate you can actually see
Object storage and training-shard formats. The hard problem is petabyte-scale media, not a warehouse
Own dataset packaging, versioning, and delivery to external research customers
Work with Shenzhen firmware so new devices speak one ingest contract, not a custom path per SKU
Make health, cost, and drop rate obvious as we add sites and countries