Senior Inference Systems Engineer for High-Performance AI

Causal Labs; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Causal Labs in San Francisco is seeking an experienced Infrastructure Engineer to build high-throughput inference systems for large-scale evaluation and backtesting against historical physical observations.

You will design techniques to improve latency and throughput, optimize the inference stack to exhaust hardware, and extend Kubernetes, Ray, and Slurm for distributed inference, ensuring reliability and reproducibility across the stack in collaboration with researchers.

Qualifications

• Experience building inference/serving systems for throughput and latency.
• Understanding of distributed compute, GPU parallelism, and hardware-aware optimization.
• Deep familiarity with deep learning frameworks (e.g. PyTorch, JAX) and their underlying system architectures.
• Strong engineering skills: performant, maintainable code and the ability to debug complex codebases.
• Bonus: contributions to open-source inference or systems infrastructure (e.g. vLLM, SGLang, Triton).

Responsibilities

• Build high-throughput inference systems for large-scale evaluation, backtesting, and scoring against historical physical observations
• Design and implement techniques that improve latency, throughput, and efficiency for real-time inference
• Optimize the inference stack to fully utilize hardware FLOPs, bandwidth, and memory
• Extend orchestration frameworks (e.g. Kubernetes, Ray, Slurm) for distributed inference and large-batch evaluation sweeps
• Establish standards for reliability, observability, and reproducibility across the inference stack, so every evaluation is trustworthy and repeatable
• Collaborate with researchers to enable high-performance inference for novel architectures as they emerge

Skills

High-throughput inference systems

Distributed compute

GPU acceleration

PyTorch/JAX

Kubernetes/Ray/Slurm

Performance debugging

Tools

TensorRT

CUDA

Job description

Causal Labs in San Francisco is seeking an experienced Infrastructure Engineer to build high-throughput inference systems for large-scale evaluation and backtesting against historical physical observations.

You will design techniques to improve latency and throughput, optimize the inference stack to exhaust hardware, and extend Kubernetes, Ray, and Slurm for distributed inference, ensuring reliability and reproducibility across the stack in collaboration with researchers.