ML Inference & Performance Engineer

Bonfirevc; Palo Alto (CA); On-site; USD 180,000 - 250,000; Full time

Job summary

Orbifold AI in Palo Alto is seeking a Member of Technical Staff, ML Engineer (Inference & Performance) to own the end-to-end serving layer for production models on Ray Serve and PyTorch, across a heterogeneous GPU fleet. This role focuses on throughput, latency, memory fit, and cost efficiency.

You will push quantization, memory layout, and kernel-level optimizations, with opportunities to develop custom CUDA/Triton code, benchmarks, and production-grade observability for scalable, reliable

Qualifications

• 3+ years building production ML systems with Python and PyTorch.
• Experience owning inference performance for a real workload with measurable improvements.
• Understanding of GPU execution: memory hierarchy, bandwidth, and kernel launch overhead.
• Experience operating a distributed serving framework such as Ray or Kubernetes in production.
• Comfortable being measured on throughput, latency, and cost.
• Judgment about when to optimize and when to leave something alone.

Responsibilities

• Own the serving end-to-end for production models on Ray Serve and PyTorch across a heterogeneous GPU fleet.
• Design batching strategies, scheduling, concurrency, queueing, and related performance optimizations.
• Implement quantization, memory layout, and cache management while validating output quality.
• Develop compiler paths or custom CUDA/Triton kernels where beneficial.
• Serve high-volume video and multimodal inference workloads with fast rollout cycles.
• Build a benchmarking harness for reproducible performance claims and to catch regressions.
• Operate in production with autoscaling, fault tolerance, and observability.
• Onboard partner models onto our infrastructure and ensure parity with local results.

Skills

Python

PyTorch

Production ML systems

Ray

Kubernetes

GPU execution

Throughput optimization

Tools

Ray Serve

TensorRT-LLM

Torch.compile

Job description

Orbifold AI in Palo Alto is seeking a Member of Technical Staff, ML Engineer (Inference & Performance) to own the end-to-end serving layer for production models on Ray Serve and PyTorch, across a heterogeneous GPU fleet. This role focuses on throughput, latency, memory fit, and cost efficiency.

You will push quantization, memory layout, and kernel-level optimizations, with opportunities to develop custom CUDA/Triton code, benchmarks, and production-grade observability for scalable, reliable