Senior C++ ML Inference Engineer — High-Scale, Low Latency

X AI; Palo Alto (CA); On-site; USD 220,000 - 357,000; Full time

Benefits offered by this job

Equity

Medical coverage

401(k) retirement plan

Job summary

SpaceXSI is building a high-performance inference platform to serve Grok to millions of users. The Software Engineer, Training/Inference (C++) will own distributed infrastructure, including global KV cache, batching, and auto-scaling, and will optimize GPU kernels and low-level components.

The role emphasizes end-to-end design, benchmarking, CI/CD tooling, and research on scaling compute for next-generation systems, with impact on throughput, latency, and reliability.

Qualifications

• Bachelor’s degree in computer science, computer engineering, or science discipline.
• 5+ years of experience with low-level system programming in C/C++ or Rust.

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build and operate high-concurrency serving systems with excellent tail latency and high reliability at scale.

Skills

C/C++

Rust

Low-level optimization

GPU inference

Concurrency

CI/CD

Education

Bachelor’s degree in computer science, computer engineering, or related field

Job description

SpaceXSI is building a high-performance inference platform to serve Grok to millions of users. The Software Engineer, Training/Inference (C++) will own distributed infrastructure, including global KV cache, batching, and auto-scaling, and will optimize GPU kernels and low-level components.

The role emphasizes end-to-end design, benchmarking, CI/CD tooling, and research on scaling compute for next-generation systems, with impact on throughput, latency, and reliability.