Senior ML Infra Engineer - Scale GPU Clusters, Remote

AI Breaking Wire; San Francisco, Northern (CA, KY); Hybrid; USD 320,000 - 500,000; Full time

Benefits offered by this job

Equity

Medical/Dental/Vision coverage

Unlimited PTO

Wellness stipends

Job summary

OpenAI is seeking a Senior ML Infrastructure Engineer to architect and scale large-scale training and inference systems. You will work with researchers to optimize performance, throughput, and reliability across tens of thousands of GPUs.

The role requires deep expertise in distributed systems, CUDA, Python, C++, and modern orchestration tools, with collaboration across hardware vendors for next-gen accelerators.

Qualifications

• BS/MS/PhD in CS or related field.
• 5+ years of software engineering experience with distributed systems.
• Proficiency in C++, Python, CUDA, and modern cluster orchestration tools (Kubernetes, Ray).
• Deep understanding of networking (InfiniBand, RoCE) and distributed training frameworks Megatron-LM, DeepSpeed.

Responsibilities

• Architect, build, and scale distributed training and inference infrastructure.
• Diagnose and resolve complex performance bottlenecks in multi-node GPU clusters.
• Develop tools and automation for CI/CD, deployment, and monitoring of ML workloads.
• Collaborate with hardware vendors to prototype and integrate next-gen accelerator chips.

Skills

Python

C++

CUDA

Kubernetes

Distributed systems

Infrastructure

Education

B.S./M.S./Ph.D. in Computer Science or related field

Tools

Megatron-LM

DeepSpeed

Ray

Job description

OpenAI is seeking a Senior ML Infrastructure Engineer to architect and scale large-scale training and inference systems. You will work with researchers to optimize performance, throughput, and reliability across tens of thousands of GPUs.

The role requires deep expertise in distributed systems, CUDA, Python, C++, and modern orchestration tools, with collaboration across hardware vendors for next-gen accelerators.