AI Inference Systems Engineer (High-Throughput, Low-Latency)

SpaceX; Palo Alto (CA); On-site; USD 135,000 - 210,000; Full time

Benefits offered by this job

Stock options

Excellent medical coverage

401(k) plan

Job summary

SpaceX in Palo Alto is seeking a Software Engineer, Inference (AI Data Engineering) to design and optimize large-scale model serving systems. You will own distributed infrastructure, drive high-throughput, low-latency inference for SpaceX's critical workloads.

You will work on GPU kernels, quantization, and acceleration techniques, collaborating with diverse teams to deliver reliable, scalable AI infrastructure that powers our ambitious missions. On-site work in Palo Alto is required.

Qualifications

• Bachelor's degree in computer science or related field, or 2+ years of professional software experience.
• Experience designing, implementing, and maintaining reliable, horizontally scalable distributed systems.
• 1+ years of full stack or backend production systems experience.
• 1+ years of experience with Rust or C++.

Responsibilities

• Develop highly reliable, high-throughput inference systems serving internal AI models.
• Architect scalable distributed infrastructure for model serving with autoscaling and batching.
• Optimize latency and throughput under production workloads using GPU techniques.
• Build reliable, high-concurrency serving systems with observability and uptime.
• Own components like request routing, SDK development, rate limiting, and scaling.
• Benchmark and accelerate inference engines (e.g., SGLang, vLLM, TensorRT-LLM).
• Develop tools for tracing, replaying, and debugging across the stack.

Skills

Rust

C++

Python

Go

Docker

Kubernetes

gRPC

GPU kernels

Education

Bachelor's degree in CS or related

2+ years of professional software experience

Tools

PostgreSQL

ClickHouse

MongoDB

Job description

SpaceX in Palo Alto is seeking a Software Engineer, Inference (AI Data Engineering) to design and optimize large-scale model serving systems. You will own distributed infrastructure, drive high-throughput, low-latency inference for SpaceX's critical workloads.

You will work on GPU kernels, quantization, and acceleration techniques, collaborating with diverse teams to deliver reliable, scalable AI infrastructure that powers our ambitious missions. On-site work in Palo Alto is required.