ML Systems Engineer — Trainium Inference & Kernels

United States Digital Space LLC; United States; Remote; USD 140,000 - 210,000; Full time

Job summary

United States Digital Space LLC seeks a Software Engineer, Trainium, to advance inference workloads on AWS Trainium and build the software stack for frontier models. You will work across kernels, compilers, and model execution to optimize performance on Trainium.

The role involves cross-stack work, identifying bottlenecks and delivering scalable solutions, with a focus on high-performance kernel development and compiler enhancements for accelerator hardware.

Qualifications

• 3+ years of relevant engineering experience in ML systems, compilers, kernels, runtimes, or performance engineering.
• Strong systems programming fundamentals and performance-critical software development.
• Experience with GPU/TPU/Trainium or other accelerator architectures.
• Ability to reason about performance across multiple layers of the stack, from hardware to ML frameworks.
• Owning technically ambiguous problems end-to-end and learning new hardware/software domains as needed.
• Bonus: experience with AWS Trainium or AWS Neuron SDK.
• Bonus: contributions to ML frameworks such as PyTorch or JAX, compiler infra such as LLVM/MLIR/XLA/Triton.

Responsibilities

• Build and optimize the company's inference stack for AWS Trainium.
• Develop high-performance kernels for critical model operations and workloads.
• Extend and improve compiler support to efficiently target Trainium hardware.
• Build the systems necessary to execute and optimize the model forward pass on Trainium.
• Profile workloads and identify bottlenecks across kernels, compiler-generated code, runtime, and model execution.
• Partner with inference and ML systems teams to bring new models and architectures onto Trainium.
• Work across the hardware/software boundary to unlock performance and capabilities from specialized AI accelerators.
• Own complex performance and systems problems end-to-end, from investigation through production deployment.

Skills

ML systems

Kernels

Runtimes

Systems programming

Accelerator architectures

Ambiguity ownership

AWS Trainium/Neuron

PyTorch/JAX/LLVM/MLIR/XLA/Triton

Tools

LLVM

MLIR

XLA

Triton

PyTorch

JAX

Job description

United States Digital Space LLC seeks a Software Engineer, Trainium, to advance inference workloads on AWS Trainium and build the software stack for frontier models. You will work across kernels, compilers, and model execution to optimize performance on Trainium.

The role involves cross-stack work, identifying bottlenecks and delivering scalable solutions, with a focus on high-performance kernel development and compiler enhancements for accelerator hardware.