Senior Inference Systems Engineer (Large-Scale GPU)

RadixArk; Palo Alto (CA); On-site; USD 200,000 - 400,000; Full time

Job summary

RadixArk is seeking a Member of Technical Staff — Inference to push the limits of large-scale AI inference. You will work on the core systems that serve frontier models at scale, optimizing performance, latency, throughput, and cost across thousands of GPUs.

This is a deeply technical role at the hardware-software boundary. The compensation for this role ranges from $200,000 to $400,000 USD per year plus equity.

Qualifications

• 5+ years in systems engineering, ML infrastructure, or performance-critical backends.
• Proven expertise in large-scale inference for LLMs or generative models.
• Deep understanding of GPU architecture and performance.
• Proficient in Python, Rust, C++, or Go for production systems.
• Experience profiling and optimizing compute-intensive workloads.
• Strong debugging across model, runtime, kernel, and network layers.

Responsibilities

• Design and build large-scale inference systems for frontier AI models.
• Optimize latency, throughput, and GPU utilization in production inference.
• Develop and improve model serving architectures and runtimes.
• Work on batching, scheduling, and memory management strategies.
• Collaborate with kernel, compiler, and systems teams on performance optimization.
• Debug performance bottlenecks across the stack.
• Drive reliability and scalability of inference infrastructure.
• Build tooling for observability, profiling, and performance analysis.
• Contribute to long-term inference architecture and strategy.

Skills

Large-scale inference systems

GPU architecture

Distributed systems

Latency optimization

Profiling

Debugging

Performance optimization

Python

Rust

C++

Go

Tools

Python

Rust

C++

Go

SGLang

vLLM

TensorRT-LLM

CUDA

Triton

Job description

RadixArk is seeking a Member of Technical Staff — Inference to push the limits of large-scale AI inference. You will work on the core systems that serve frontier models at scale, optimizing performance, latency, throughput, and cost across thousands of GPUs.

This is a deeply technical role at the hardware-software boundary. The compensation for this role ranges from $200,000 to $400,000 USD per year plus equity.