Senior Inference Systems Engineer

SpaceX Super Intelligence; Palo Alto (CA); On-site; USD 180,000 - 440,000; Full time

Benefits offered by this job

Equity

Medical, Vision, Dental

401(k)

Disability Insurance

Life Insurance

Employee discounts

Job summary

SpaceXSI in Palo Alto is building the high-performance inference platform that serves Grok to millions of users daily with speed and reliability. As a Member of Technical Staff - Inference, you will own end-to-end design of distributed infrastructure, from global KV cache and auto-scaling to low-level GPU kernel work and code generation.

This role demands a strong track record in systems programming and optimizing large-scale model serving, with exposure to vLLM, Triton, or TensorRT-LLM.

Qualifications

• Deep low-level systems programming in C/C++ or Rust.
• Experience with large-scale, high-concurrent production serving.
• Experience with GPU inference engines (vLLM, SGLang, Triton, TensorRT-LLM).
• Strong background in system optimizations: batching, caching, load balancing.
• Low-level inference optimizations: GPU kernels, code generation.
• Algorithmic inference optimizations: quantization, speculative decoding.
• Experience with testing, benchmarking, and reliability of inference services.
• Experience designing and implementing CI/CD infrastructure for inference.

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache).
• Optimize latency and throughput of model inference under real production workloads.
• Build reliable, high-concurrency serving systems that serve billions of users with 100% uptime, 0% error rate, and excellent tail latency.
• Benchmark, fine-tune, and accelerate inference engines (including low-level GPU kernel work and code generation).
• Develop custom tools to trace, replay, and fix issues across the full stack — from orchestration down to GPU kernels.
• Create robust CI/CD infrastructure for seamless endpoint deployment, image publishing, and inference engine updates.
• Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems.

Skills

Low-level systems

High-concurrency serving

GPU inference

Batching & caching

CI/CD for inference

Quantization

Code generation

Reliability testing

Tools

Triton

TensorRT-LLM

vLLM

SGLang

Job description

SpaceXSI in Palo Alto is building the high-performance inference platform that serves Grok to millions of users daily with speed and reliability. As a Member of Technical Staff - Inference, you will own end-to-end design of distributed infrastructure, from global KV cache and auto-scaling to low-level GPU kernel work and code generation.

This role demands a strong track record in systems programming and optimizing large-scale model serving, with exposure to vLLM, Triton, or TensorRT-LLM.