Senior AI Inference Runtime Architect

Arm; Seattle (WA); Hybrid; USD 263,000 - 355,000; Full time

Benefits offered by this job

Hybrid working

Accommodations during recruitment

Job summary

Arm is seeking a Principal Software Engineer to lead the AI Inference Runtime team in Seattle, shaping technical direction for distributed inference workloads and memory systems. You will collaborate with AI Infra, compute, and product groups to optimize performance across platforms and models.

You will architect scheduling, batching, KV‑cache management, and kernel development, driving end‑to‑end model support and production validation with a focus on latency, throughput, and resource

Qualifications

• 8+ years in ML or high‑performance systems, or equivalent impact.
• Deep AI inference knowledge including model execution and KV-cache behavior.
• Strong coding skills in C++, Rust, or Python with concurrency and memory handling.

Responsibilities

• Define architecture and roadmap for AI inference runtime across platforms.
• Enable end‑to‑end model support with scheduling, batching, memory management.
• Profile bottlenecks and develop optimized kernels and data paths.
• Lead reviews, mentor engineers, and drive performance‑engineering practices.

Skills

ML systems

C++/Rust/Python

Profiling/optimization

Distributed systems

Job description

Arm is seeking a Principal Software Engineer to lead the AI Inference Runtime team in Seattle, shaping technical direction for distributed inference workloads and memory systems. You will collaborate with AI Infra, compute, and product groups to optimize performance across platforms and models.

You will architect scheduling, batching, KV‑cache management, and kernel development, driving end‑to‑end model support and production validation with a focus on latency, throughput, and resource