Software Engineer, Machine Learning Infrastructure

AI Breaking Wire; San Francisco, Northern (CA, KY); Hybrid; USD 170,000 - 250,000; Full time

Benefits offered by this job

Industry-leading compensation and/or:?

Unlimited PTO

Top-tier medical, dental, and vision

Catered lunches & on-site wellness

Job summary

OpenAI in San Francisco is seeking an experienced engineer to architect, build, and scale high-performance distributed training and inference infrastructure for massive LLMs. You will optimize GPU cluster utilization and memory management for multi-node training, partnering with research teams to accelerate experiments.

The role requires deep expertise in distributed systems, networking, and high-throughput computing, plus strong coding in C++, Python, and CUDA, with hands-on experience on large

Qualifications

• 3+ years of industry experience in distributed systems or related field.
• Deep expertise in distributed systems, networking, and high-throughput computing.
• Strong programming in C++, Python, and CUDA.

Responsibilities

• Architect, build, and scale high-performance distributed training and inference infrastructure for massive LLMs.
• Optimize GPU cluster utilization, network topologies, and memory management for multi-node training jobs.
• Collaborate with research teams to streamline experiment velocity and scale up model architectures.
• Troubleshoot complex distributed system bottlenecks, hardware failures, and performance regressions.

Skills

Distributed systems

Networking

High-throughput computing

C++

Python

CUDA

Kubernetes

Slurm

Education

B.S./M.S./Ph.D. in CS or related

Tools

Kubernetes

Slurm

Job description

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that artificial general intelligence benefits all of humanity. Our infrastructure team builds the massive distributed systems required to train and serve frontier models.

Responsibilities

• Architect, build, and scale high-performance distributed training and inference infrastructure for massive LLMs.


• Optimize GPU cluster utilization, network topologies, and memory management for multi-node training jobs.


• Collaborate with research teams to streamline experiment velocity and scale up next-generation model architectures.


• Troubleshoot complex distributed system bottlenecks, hardware failures, and performance regressions in production.

Requirements

• B.S., M.S., or Ph.D. in Computer Science or related technical field with 3+ years of industry experience.


• Deep expertise in distributed systems, networking (InfiniBand/RoCE), and high-throughput computing.


• Strong programming skills in C++, Python, and CUDA.


• Experience working with large-scale GPU clusters (NVIDIA A100/H100/B200) and orchestration tools (Kubernetes, Slurm).

Benefits

• Industry-leading compensation and equity packages.


• Unlimited PTO and flexible work arrangements.


• Top-tier medical, dental, and vision benefits.


• Catered daily lunches and on-site wellness programs.