AI Inference Engineer - Scalable, Low-Latency Systems

SPACE EXPLORATION TECHNOLOGIES CORP; Palo Alto, Northern (CA, KY); Hybrid; USD 135,000 - 210,000; Full time

Benefits offered by this job

401(k)

Medical, vision and dental coverage

Paid parental leave

Paid vacation and holidays

Employee stock purchase plan

Disability insurance

Life insurance

Job summary

SpaceX in Palo Alto seeks a Software Engineer, Inference (AI Data Engineering) to design and optimize high-throughput AI model serving systems. You will own distributed infrastructure from routing to batching and work with SpaceX AI teams to deliver reliable, scalable inference that powers mission-critical applications.

You will collaborate across teams while enhancing performance with GPU kernels, quantization, and low-level optimizations, reporting through SpaceX Internal AI Infrastructure.

Qualifications

• Bachelor's degree or 2+ years of software experience in lieu of degree.
• Experience designing and maintaining reliable, horizontally scalable distributed systems.
• 1+ years full stack or backend development with production systems.
• 1+ years of Rust or C++ programming.

Responsibilities

• Develop high-throughput inference systems across SpaceX internals.
• Architect scalable distributed infrastructure for model serving (load balancing, auto-scaling, batching).
• Optimize latency and throughput under production workloads with GPU work and quantization.
• Build reliable, high-concurrency serving systems with observability.
• Own end-to-end components for internal SpaceX AI inference platforms.
• Benchmark and accelerate inference engines (SGLang, vLLM, TensorRT-LLM).
• Create tools for tracing, replaying issues across the stack.
• Develop CI/CD for endpoint deployment and updates.
• Collaborate with SpaceX AI teams to integrate inference into workflows.

Skills

Distributed systems design

Full stack/backend development

Rust or C++

LLM inference engines

Docker/Kubernetes

gRPC

Python/Go

Performance profiling

High-concurrency systems

Education

Bachelor's degree in CS/Engineering/Math

Tools

Docker

Kubernetes

PostgreSQL

ClickHouse

MongoDB

SGLang

vLLM

TensorRT-LLM

GPU kernels

Job description

SpaceX in Palo Alto seeks a Software Engineer, Inference (AI Data Engineering) to design and optimize high-throughput AI model serving systems. You will own distributed infrastructure from routing to batching and work with SpaceX AI teams to deliver reliable, scalable inference that powers mission-critical applications.

You will collaborate across teams while enhancing performance with GPU kernels, quantization, and low-level optimizations, reporting through SpaceX Internal AI Infrastructure.