Senior Staff Engineer, Large-Scale AI Inference

RadixArk

Palo Alto (CA)

On-site

USD 200,000 - 400,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

RadixArk is seeking a Member of Technical Staff — Inference to push the limits of large-scale AI inference. You will work on the core systems that serve frontier models at scale, optimizing performance, latency, throughput, and cost across thousands of GPUs.

This role sits at the intersection of systems engineering, ML infrastructure, and performance optimization. Your work will directly shape how state-of-the-art models are deployed and experienced by users worldwide.

Qualifications

  • 5+ years in systems engineering, ML infrastructure, or backend systems
  • Expertise in large-scale inference for LLMs or generative models
  • Deep GPU architecture and performance understanding
  • Experience reducing latency and increasing throughput in production
  • Strong knowledge of distributed systems and networking fundamentals
  • Proficiency in Python, Rust, C++, or Go for production
  • Experience profiling compute-intensive workloads
  • Strong debugging across system layers (model, runtime, kernel, network)

Responsibilities

  • Design and build large-scale inference systems for frontier AI models
  • Optimize latency, throughput, and GPU utilization in production
  • Develop and improve model serving architectures and runtimes
  • Work on batching, scheduling, and memory management strategies
  • Collaborate with kernel, compiler, and systems teams on performance optimization
  • Debug performance bottlenecks across the stack
  • Drive reliability and scalability of inference infrastructure
  • Build tooling for observability, profiling, and performance analysis
  • Contribute to long-term inference architecture and strategy

Skills

Systems engineering
ML infrastructure
Performance optimization
Python
Rust
C++
Go
Distributed systems
Networking
Profiling

Tools

SGLang
vLLM
TensorRT-LLM
CUDA
Triton

Job description

RadixArk is seeking a Member of Technical Staff — Inference to push the limits of large-scale AI inference. You will work on the core systems that serve frontier models at scale, optimizing performance, latency, throughput, and cost across thousands of GPUs.

This role sits at the intersection of systems engineering, ML infrastructure, and performance optimization. Your work will directly shape how state-of-the-art models are deployed and experienced by users worldwide.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Member of Technical Staff — Inference-Core Engine
Member of Technical Staff — Inference-Core Engine

RadixArk • Palo Alto (CA)

On-site
USD 200,000 - 400,000
Staff Engineer, DevTech: Fast AI Inference on GPUs
Staff Engineer, DevTech: Fast AI Inference on GPUs

RadixArk • Palo Alto (CA)

On-site
USD 180,000 - 240,000
Equity
Flexible work arrangements
Member of Technical Staff, Inference Systems
Member of Technical Staff, Inference Systems

Wintermeyer Ventures • Palo Alto (CA)

On-site
USD 230,000 - 350,000
Member of Technical Staff — Developer Technology
Member of Technical Staff — Developer Technology

RadixArk • Iowa

On-site
USD 180,000 - 240,000
Equity
Flexible work arrangements
Senior/Staff Software Engineer — Scalable AI Inference Systems
Senior/Staff Software Engineer — Scalable AI Inference Systems

EngineersOfAI • New York (NY)

On-site
USD 180,000 - 280,000
Engineering Manager - AI Inference & Systems, Equity
Engineering Manager - AI Inference & Systems, Equity

Inferact • San Francisco (CA)

On-site
USD 200,000 - 400,000
Health benefits
Dental benefits
Vision benefits
+1
Senior Distributed Systems Engineer — AI Inference Platform
Senior Distributed Systems Engineer — AI Inference Platform

Baseten • San Francisco (CA)

On-site
USD 180,000 - 240,000
Equity
Comprehensive medical for US
Flexible PTO
+4
Senior Staff ML Engineer — Real-Time Inference at Scale
Senior Staff ML Engineer — Real-Time Inference at Scale

Inworld AI • United States

Remote
USD 188,000 - 269,000
Equity
Benefits
Staff Engineer, AI Inference & Performance
Staff Engineer, AI Inference & Performance

Reflection • United States

Remote
USD 150,000 - 230,000
Top-tier compensation
Stock options
Health & wellness
+5
Inference Systems Engineer — High-Performance AI Serving
Inference Systems Engineer — High-Performance AI Serving

River AI • Palo Alto (CA)

On-site
USD 200,000 - 420,000
Health insurance
Dental insurance
Vision insurance
+3