Inference Systems Engineer — High-Performance AI Serving

River AI; Palo Alto (CA); Sur place; USD 200 000 - 420 000; Plein temps

Avantages offerts par ce poste

Health insurance

Dental insurance

Vision insurance

Unlimited PTO

Relocation assistance

Visa sponsorship

Résumé du poste

River AI in Palo Alto is seeking exceptional inference systems engineers to build the engines that serve large models through the River API. Your goal is to deliver fast, reliable inference while making efficient use of GPU compute and memory.

You will own the serving runtime from request scheduling and continuous batching to KV-cache management, distributed model execution, and checkpoint loading. Working closely with GPU kernel engineers, researchers, and infrastructure engineers, you will

Qualifications

• Bachelor’s degree in CS, CE, or equivalent practical experience.
• Experience building inference engines or performance-sensitive distributed services.
• Strong knowledge of transformer inference, GPU memory, concurrency, and networking.
• Proficiency in Python and C++ or Rust.
• Strong debugging and profiling skills across models, runtimes, and services.
• Collaborative mindset with ownership of outcomes.

Responsabilités

• Optimize inference for dense and mixture-of-experts models.
• Improve batching, caching, and admission control for throughput and latency.
• Accelerate multi-GPU execution while preserving model-version consistency.
• Improve RL sampling throughput with correct model versioning.
• Build reliable streaming, cancellation, and recovery under load.
• Profile bottlenecks and validate improvements with reproducible tests.

Connaissances

Python

C++

Rust

Transformer inference

GPU memory

Concurrency

Networking

Debugging

Profiling

Distributed services

Formation

Bachelor's degree in CS/CE

Outils

SGLang

vLLM

TensorRT-LLM

CUDA

NVIDIA profiling tools

Description du poste

River AI in Palo Alto is seeking exceptional inference systems engineers to build the engines that serve large models through the River API. Your goal is to deliver fast, reliable inference while making efficient use of GPU compute and memory.

You will own the serving runtime from request scheduling and continuous batching to KV-cache management, distributed model execution, and checkpoint loading. Working closely with GPU kernel engineers, researchers, and infrastructure engineers, you will