TPU Compiler Engineer for High-Performance Inference

OpenAI, Inc.; San Francisco (CA); On-site; USD 140,000 - 210,000; Full time

Job summary

OpenAI is seeking an engineer to lead the optimization of state-of-the-art models on TPUs. You will own performance across the full forward pass, from model code through kernels to hardware execution, with broad ownership over bottlenecks and improvements.

You will collaborate with researchers and engineers to support new model architectures, build benchmarks, and validate correctness, applying quantization and other techniques to push latency and throughput while preserving model quality.

Qualifications

• Experience optimizing ML workloads on TPUs (training, inference, or both).
• Understanding of modern model architectures and their mapping to accelerator hardware.
• Ability to connect kernel behavior to full-model performance and work across stack layers.
• Strong software engineering skills with systematic profiling, debugging, and validation.

Responsibilities

• Optimize model execution on TPUs across the forward pass (model code, kernels, hardware).
• Profile workloads, identify bottlenecks, and improve latency, throughput, and hardware utilization.
• Develop and optimize kernels guided by surrounding model behavior.
• Explore quantization to improve inference efficiency while preserving quality.
• Build benchmarks and tools to measure performance and validate correctness.
• Collaborate with researchers and engineers to productionize performance improvements.

Skills

TPU optimization

Kernel optimization

Profiling

C++

Python

Performance engineering

Model inference

Quantization

Tools

TPU

Job description

OpenAI is seeking an engineer to lead the optimization of state-of-the-art models on TPUs. You will own performance across the full forward pass, from model code through kernels to hardware execution, with broad ownership over bottlenecks and improvements.

You will collaborate with researchers and engineers to support new model architectures, build benchmarks, and validate correctness, applying quantization and other techniques to push latency and throughput while preserving model quality.