ML Systems Engineer: Trainium Inference & Kernels

Slope; San Francisco (CA); On-site; USD 140,000 - 210,000; Full time

Job summary

OpenAI in San Francisco seeks an experienced Software Engineer to help bring inference workloads to AWS Trainium and build the software stack to run frontier models efficiently on the platform. This deeply technical, cross‑stack role covers kernels, compilers, and model execution.

You will develop high‑performance kernels, improve compiler support, and ensure scalable execution of the model forward pass on Trainium.

Qualifications

• 3+ years of relevant engineering experience in ML systems, compilers, kernels, runtimes, or performance engineering.
• Strong systems programming fundamentals and performance‑critical software experience.
• Experience with GPU, TPU, Trainium, or other accelerator architectures.
• Ability to reason about performance across stack layers from hardware to ML frameworks.
• Bonus: experience with AWS Trainium or the AWS Neuron SDK.
• Bonus: contributions to ML frameworks or compiler infrastructure like LLVM/MLIR/XLA/Triton.

Responsibilities

• Build and optimize OpenAI's inference stack for AWS Trainium.
• Develop high‑performance kernels for critical model operations and workloads.
• Extend and improve compiler support to efficiently target Trainium hardware.
• Build the systems necessary to execute and optimize the model forward pass on Trainium.
• Profile workloads and identify bottlenecks across kernels, compiler‑generated code, runtime, and model execution.
• Partner with inference and ML systems teams to bring new models and architectures onto Trainium.
• Work across the hardware/software boundary to unlock performance and capabilities from specialized AI accelerators.
• Own complex performance and systems problems end‑to‑end, from investigation through production deployment.

Skills

ML systems

compilers

kernels

runtimes

performance engineering

Job description

OpenAI in San Francisco seeks an experienced Software Engineer to help bring inference workloads to AWS Trainium and build the software stack to run frontier models efficiently on the platform. This deeply technical, cross‑stack role covers kernels, compilers, and model execution.

You will develop high‑performance kernels, improve compiler support, and ensure scalable execution of the model forward pass on Trainium.