ML Systems Engineer for Trainium Inference

AI Chopping Block; San Francisco, Northern (CA, KY); Hybrid; USD 180,000 - 280,000; Full time

Job summary

OpenAI is seeking a Software Engineer, Trainium, to bring OpenAI's inference workloads to AWS Trainium and build the software stack for efficient model execution.

This deeply technical role spans kernels, compilers, and model execution, addressing performance bottlenecks and enabling frontier-scale AI systems across the stack.

If you enjoy ML systems, compilers, and accelerator hardware challenges, this role fits you—self-directed and capable across abstraction layers.

Qualifications

• 3+ years of relevant engineering experience in ML systems, compilers, kernels, runtimes, or performance engineering.
• Experience with GPU/TPU/Trainium or other specialized accelerators.
• Ability to reason about performance across hardware, kernels, compilers and ML frameworks.
• Owning ambiguous problems end-to-end and learning new domains as needed.
• Bonus: experience with AWS Trainium or AWS Neuron SDK; contributions to ML frameworks or compiler infra.

Responsibilities

• Build and optimize OpenAI's inference stack for AWS Trainium.
• Develop high-performance kernels for critical model operations.
• Extend and improve compiler support to target Trainium hardware.
• Build systems to execute and optimize the model forward pass on Trainium.
• Profile workloads and identify bottlenecks across kernels, runtime, and model execution.
• Collaborate with inference and ML systems teams to bring models and architectures to Trainium.
• Work across hardware/software boundaries to unlock accelerator performance.

Skills

Systems programming

Performance engineering

ML systems

Kernel development

Compiler support

Tools

LLVM

MLIR

XLA

Triton

PyTorch

Job description

OpenAI is seeking a Software Engineer, Trainium, to bring OpenAI's inference workloads to AWS Trainium and build the software stack for efficient model execution.

This deeply technical role spans kernels, compilers, and model execution, addressing performance bottlenecks and enabling frontier-scale AI systems across the stack.

If you enjoy ML systems, compilers, and accelerator hardware challenges, this role fits you—self-directed and capable across abstraction layers.