Senior C++ Training/Inference Engineer - High-Performance AI

xAI; Palo Alto, Seattle (CA, WA); On-site; USD 200,000 - 375,000; Full time

Job summary

SpaceXSI is hiring a Software Engineer, Training/Inference (C++) to design and optimize large-scale model serving systems used by Grok across millions of users. You will own distributed infrastructure from global KV cache to auto-scaling and batching, focusing on high performance and reliability.

The role demands strong low-level programming in C/C++ or Rust, with experience in GPU kernels, quantization, and performance tuning to deliver fast, scalable inference engines.

Qualifications

• Bachelor’s degree in computer science, computer engineering, or science discipline.
• 5+ years of low-level system programming in C/C++ or Rust.

Responsibilities

• Architect and implement scalable distributed model serving infrastructure with load balancing and auto-scaling.
• Optimize latency and throughput for model inference under production workloads.
• Build and operate high-concurrency serving systems with low tail latency at scale.
• Develop CI/CD tooling for seamless deployment of inference engines.

Skills

Low-level programming (C/C++)

Rust

High-concurrency systems

GPU/ML inference optimization

Education

Bachelor’s degree in CS/CE/Science

Tools

GPU inference engines (vLLM, Triton, TensorRT-LLM)

Job description

SpaceXSI is hiring a Software Engineer, Training/Inference (C++) to design and optimize large-scale model serving systems used by Grok across millions of users. You will own distributed infrastructure from global KV cache to auto-scaling and batching, focusing on high performance and reliability.

The role demands strong low-level programming in C/C++ or Rust, with experience in GPU kernels, quantization, and performance tuning to deliver fast, scalable inference engines.