Technical Lead Manager, Scalable Training Data & Movement

OpenAI; California (MO); On-site; USD 180,000 - 260,000; Full time

Job summary

OpenAI is seeking a hands-on Technical Lead Manager to own datasets across its training infrastructure, shaping how training jobs read data, APIs, and storage contracts. You will lead as the technical owner for dataset reads, aligning researchers, framework owners, and storage teams around a durable platform.

The role calls for leading the design of a unified data access layer, defining APIs, ensuring reproducibility, and building reliable data paths across large-scale distributed training

Qualifications

• Experience designing and building dataset read platforms for multiple training frameworks.
• Ability to define APIs, storage-format expectations, and versioning for reproducible data access.
• Experience with stateful iterators, caching, and reliable data pipelines.

Responsibilities

• Design and build a unified dataset read platform for current and future training frameworks.
• Define dataset APIs, storage-format expectations, registration/versioning, and migration paths.
• Build reliability into the read path with stateful iteration, caching, fast restart, recovery, and clear operational contracts.
• Develop terminal and web-based visualizers to inspect multimodal and RL data late in the pipeline.
• Write and review production code in core data loading, service, caching, and reliability paths.
• Partner with teams working on training frameworks, reinforcement learning, multimodal models, storage, runtime, and cluster infrastructure.

Skills

Dataset infra

API design

Python & low-level systems

Rust/C++ exposure

Distributed training

Tools

Python

Rust/C++

Job description

OpenAI is seeking a hands-on Technical Lead Manager to own datasets across its training infrastructure, shaping how training jobs read data, APIs, and storage contracts. You will lead as the technical owner for dataset reads, aligning researchers, framework owners, and storage teams around a durable platform.

The role calls for leading the design of a unified data access layer, defining APIs, ensuring reproducibility, and building reliable data paths across large-scale distributed training