Staff ML Engineer — Production Inference & Serving

Orbifold AI, Inc.; Palo Alto (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Orbifold AI is building the infrastructure layer for Physical AI in a scale-oriented environment. You will own the serving architecture for production models, designing and running the end-to-end inference stack on Ray Serve and PyTorch across a heterogeneous GPU fleet with variable workloads.

We expect 3+ years building production ML systems, deep Python and PyTorch expertise, and a strong grasp of GPU execution, batching, quantization, and kernel-level optimization.

Qualifications

• 3+ years building production ML systems with Python and PyTorch.
• Proven ownership of inference performance for real workloads with measurable metrics.
• Mental model of GPU execution: memory bandwidth, occupancy and kernel launch costs.
• Experience operating distributed serving or compute frameworks under real load.
• Comfort being measured on throughput, latency and cost over time.
• Judgment on when to optimize and when to leave things as is.

Responsibilities

• Own model serving end to end; design and run the serving layer for production models on Ray Serve and PyTorch.
• Optimize throughput and utilization: batching, scheduling, concurrency, queueing.
• Make models fit: quantization, memory layout, activation and cache management with verification.
• Delve into compiler paths, custom kernels where beneficial and know when not to.
• Handle high-volume video and multimodal inference workloads.
• Build benchmarking harnesses to ensure reproducible performance claims.
• Operate in production with autoscaling, fault tolerance, observability and cost per workload.

Skills

Python

PyTorch

Inference performance

GPU execution model

Distributed serving (Ray/Kubernetes)

Throughput & latency focus

Profiling & optimization

Tools

Ray Serve

TensorRT

Triton

XLA

Torch.compile

CUDA

TensorRT-LLM

Open-source serving stacks

Job description

Orbifold AI is building the infrastructure layer for Physical AI in a scale-oriented environment. You will own the serving architecture for production models, designing and running the end-to-end inference stack on Ray Serve and PyTorch across a heterogeneous GPU fleet with variable workloads.

We expect 3+ years building production ML systems, deep Python and PyTorch expertise, and a strong grasp of GPU execution, batching, quantization, and kernel-level optimization.