Technical Lead Manager, Data Movement & Training Runtime

OpenAI; San Francisco (CA); On-site; USD 295,000 - 445,000; Full time

Job summary

OpenAI is seeking a Technical Lead Manager to own dataset management in training infrastructure. This role involves designing a unified platform for data access and building tools for debugging and visualization.

Ideal candidates have experience with distributed training systems, API design, and debugging. Compensation ranges from $295K to $445K. OpenAI is committed to equal opportunity and providing reasonable accommodations.

Qualifications

• Experience in building and owning distributed training infrastructure.
• Familiarity with stateful iterators and caching mechanisms.
• Understanding of the failure modes in large distributed systems.

Responsibilities

• Design a unified dataset read platform for training frameworks.
• Define storage formats and versioning for data access.
• Build visualizers for data inspection and debugging.

Skills

Dataset management

API design

Debugging

Performance optimization

Python programming

Education

Experience in data infrastructure at large scale

Tools

Rust or C++ (useful)

Job description

OpenAI is seeking a Technical Lead Manager to own dataset management in training infrastructure. This role involves designing a unified platform for data access and building tools for debugging and visualization.

Ideal candidates have experience with distributed training systems, API design, and debugging. Compensation ranges from $295K to $445K. OpenAI is committed to equal opportunity and providing reasonable accommodations.