AI Inference Systems Engineer - High-Throughput Serving

SpaceX; Palo Alto (CA); On-site; USD 135,000 - 210,000; Full time

Benefits offered by this job

Stock options

Medical, vision and dental coverage

401(k) retirement plan

Paid vacation and holidays

Job summary

SpaceX in Palo Alto is seeking a Software Engineer, Inference (AI Data Engineering) to design and optimize large-scale model serving systems. You will own distributed infrastructure and deep low-level optimizations to deliver high-throughput AI inference for SpaceX programs.

We value engineers who are fair, collaborative, and motivated to impact a mission-driven team. This role reports through SpaceX Internal AI Infrastructure and requires onsite work in Palo Alto.

Qualifications

• Bachelor's degree or 2+ years of equivalent software experience.
• Experience designing and maintaining scalable distributed systems.
• 1+ years of backend or full-stack development with production systems.
• 1+ years of Rust or C++.

Responsibilities

• Develop highly reliable, high-throughput inference systems that serve internal AI models across SpaceX.
• Architect and implement scalable distributed infrastructure for model serving, including load balancing and auto-scaling.
• Optimize latency and throughput under real production workloads, including GPU-related optimizations.
• Build reliable, high-concurrency serving systems with 100% uptime and observability.
• Own end-to-end components such as routing, SDK development, rate limiting, and scaling for internal AI inference platforms.
• Benchmark, fine-tune, and accelerate inference engines (e.g., SGLang, vLLM, TensorRT-LLM).
• Develop tools for tracing, replaying, and issue resolution across full stack.
• Create CI/CD for endpoint deployment and inference engine updates.
• Collaborate with SpaceXAI teams to integrate inference across systems.

Skills

Distributed systems design

Backend development

Rust or C++

Python or Go

Full stack experience

Education

Bachelor's degree in CS/Engineering/Math or related field

Tools

Docker

Kubernetes

gRPC

PostgreSQL

ClickHouse

MongoDB

Job description

SpaceX in Palo Alto is seeking a Software Engineer, Inference (AI Data Engineering) to design and optimize large-scale model serving systems. You will own distributed infrastructure and deep low-level optimizations to deliver high-throughput AI inference for SpaceX programs.

We value engineers who are fair, collaborative, and motivated to impact a mission-driven team. This role reports through SpaceX Internal AI Infrastructure and requires onsite work in Palo Alto.