Principal AI Inference Runtime Architect

Arm Limited; Seattle (WA); Hybrid; USD 263,000 - 355,000; Full time

Benefits offered by this job

Hybrid working

Accommodations during recruitment

Job summary

Arm Limited is seeking a Principal Software Engineer for the AI Inference Runtime team in Seattle. You will set technical direction for distributed inference workloads, optimizing scheduling, batching, KV-cache, memory management, and kernel development to improve model throughput.

You will partner with AI Infrastructure, compute, and product teams to drive performance, reliability, and efficiency across platforms, shaping Arm’s AI platform strategy and production workloads.

Qualifications

• 8+ years in ML or high-performance systems or production AI inference.
• Deep understanding of modern AI inference, including model execution and KV-cache behavior.
• Proficient in C++, Rust, Python and concurrent programming.

Responsibilities

• Define architecture and roadmap for AI inference runtime capabilities.
• Enable new model architectures end-to-end through optimization of scheduling and memory management.
• Profile bottlenecks and develop optimized kernels and data paths across compute, memory, and networking.
• Evaluate new inference techniques and build benchmarking and safe-rollout systems.
• Partner with cloud, framework, compiler, hardware and research teams; mentor engineers.

Skills

ML systems

High-performance systems

Kernel development

Profiling & debugging

Concurrency & parallel programming

C++/Rust/Python

Tools

Kernels

Accelerator programming

intrinsics/assembly

Memory management

Job description

Arm Limited is seeking a Principal Software Engineer for the AI Inference Runtime team in Seattle. You will set technical direction for distributed inference workloads, optimizing scheduling, batching, KV-cache, memory management, and kernel development to improve model throughput.

You will partner with AI Infrastructure, compute, and product teams to drive performance, reliability, and efficiency across platforms, shaping Arm’s AI platform strategy and production workloads.