GPT Inference Infrastructure Engineer

OpenAI; San Francisco (CA); On-site; USD 293,000 - 385,000; Full time

Job summary

OpenAI is seeking a Software Engineer for the GPT Infrastructure team to build platform services that qualify and optimize inference workloads across diverse hardware. You will implement long-running workflows that generate kernels, configure runtimes, and validate correctness on accelerator devices.

You will collaborate across model architecture, compilers, runtimes, and security boundaries to turn research prototypes into reliable infrastructure with clear interfaces and strong observability.

Qualifications

• Experience building distributed systems and production services.
• Proficiency in Python, C++, Go, or Rust.
• Experience designing APIs, orchestration systems, durable workflows, or backend services.
• Strong understanding of Linux, networking, storage, containers, and distributed architectures.
• Ability to reason about model execution across software and hardware.
• Experience using profiling, tracing, benchmarking, and measurement to guide decisions.
• Strong ownership and ability to collaborate across teams.

Responsibilities

• Design, build, and operate APIs and control-plane services for long-running campaigns.
• Build secure partner-side execution and evaluation software for accelerator hardware.
• Integrate workloads, hardware profiles, toolchains, runtimes, and backends into a repeatable platform.
• Develop evaluation systems for fidelity, latency, throughput, and cost efficiency.
• Automate kernel and runtime configuration generation and optimization.
• Diagnose performance and correctness issues across code, kernels, and hardware.
• Turn experiments into reliable product surfaces with clear interfaces.
• Collaborate with Research, Inference Engineering, Runtime, and Security teams.
• Drive architecture and execution across OpenAI systems and partner environments.

Skills

Distributed systems

Infrastructure platforms

Production services

Orchestration systems

Python

C++

Go

Rust

Job description

OpenAI is seeking a Software Engineer for the GPT Infrastructure team to build platform services that qualify and optimize inference workloads across diverse hardware. You will implement long-running workflows that generate kernels, configure runtimes, and validate correctness on accelerator devices.

You will collaborate across model architecture, compilers, runtimes, and security boundaries to turn research prototypes into reliable infrastructure with clear interfaces and strong observability.