Senior Software Engineer, High-Performance C++ Inference

SpaceX Super Intelligence; Palo Alto (CA); In loco; USD 220.000 - 357.000; Tempo pieno

Vantaggi offerti da questo lavoro

Equity

Medical, vision & dental

401(k) retirement plan

Disability & life insurance

Descrizione del lavoro

SpaceXSI is hiring a Sr. Software Engineer, Training/Inference (C++) to design and optimize the high-performance model serving platform used by Grok products.

You will own distributed infrastructure from global KV caches to auto-scaling, with focus on low-latency, reliable inference for millions of users. The role demands deep experience in C++/Rust, GPU kernel work, and advanced optimizations such as batching, quantization, and tail latency reduction.

Competenze

• Bachelor’s degree in computer science, computer engineering, or related field.
• 5+ years of low-level system programming experience in C/C++ or Rust.
• Experience with large-scale production serving of models.
• Strong background in system optimizations: batching, caching, load balancing.

Mansioni

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build and operate high-concurrency serving systems with excellent tail latency and high reliability at scale.
• Benchmark, fine-tune, and accelerate inference engines (including low-level GPU kernel work and code generation).
• Develop custom tools to trace, replay, and fix issues across the full stack — from orchestration down to GPU kernels.
• Create robust CI/CD infrastructure for seamless endpoint deployment, image publishing, and inference engine updates.

Conoscenze

Low-level system programming

Large-scale distributed serving

GPU inference engines

Performance optimization

CI/CD for inference services

Formazione

Bachelor's degree in CS/CE or related field

Strumenti

Triton

TensorRT-LLM

vLLM

SGLang

Descrizione del lavoro

SpaceXSI is hiring a Sr. Software Engineer, Training/Inference (C++) to design and optimize the high-performance model serving platform used by Grok products.

You will own distributed infrastructure from global KV caches to auto-scaling, with focus on low-latency, reliable inference for millions of users. The role demands deep experience in C++/Rust, GPU kernel work, and advanced optimizations such as batching, quantization, and tail latency reduction.