AI Inference Engineer - High-Throughput Distributed Systems

United States Digital Space LLC; Palo Alto (CA); On-site; USD 135,000 - 210,000; Full time

Benefits offered by this job

Stock options

Medical Insurance

Vision Insurance

Dental Insurance

401(k) plan

Disability Insurance

Life Insurance

Paid parental leave

Paid vacation

Holidays

Job summary

United States Digital Space LLC in Palo Alto is seeking a Software Engineer, Inference (AI Data Engineering) to design and optimize large-scale model serving systems. You will own end-to-end components from distributed infrastructure to low-level optimizations and ensure high throughput for mission-critical applications.

This onsite role requires collaboration across SpaceXAI teams, strong background in Rust/C++, and experience with Docker/Kubernetes, gRPC, and performance profiling.

Qualifications

• Bachelor's degree in computer science, engineering, math, or related field, or 2+ years of experience.
• Experience designing, implementing, and maintaining reliable distributed systems.
• 1+ years of experience in full stack development or backend development with production systems.
• 1+ years of experience with Rust or C++.

Responsibilities

• Develop high-throughput inference systems serving internal AI models.
• Architect scalable distributed model-serving infrastructure.
• Optimize latency under production workloads including GPU work.
• Build reliable, highly concurrent serving systems with observability.
• Own end-to-end components like request routing and SDKs.
• Benchmark and accelerate inference engines (SGLang, vLLM, TensorRT-LLM).
• Create CI/CD for deployment and updates.
• Collaborate with SpaceXAI teams to integrate into broader workflows.

Skills

Rust

C++

Python

Go

Distributed systems

gRPC

Docker

Kubernetes

High-concurrency

Education

Bachelor's degree in CS/Engineering

Tools

Docker

Kubernetes

PostgreSQL

ClickHouse

MongoDB

TensorRT-LLM

SGLang

vLLM

Triton

Job description

United States Digital Space LLC in Palo Alto is seeking a Software Engineer, Inference (AI Data Engineering) to design and optimize large-scale model serving systems. You will own end-to-end components from distributed infrastructure to low-level optimizations and ensure high throughput for mission-critical applications.

This onsite role requires collaboration across SpaceXAI teams, strong background in Rust/C++, and experience with Docker/Kubernetes, gRPC, and performance profiling.