Inference Optimization ML Engineer

Rhoda AI; Mountain View (WY); On-site; USD 180,000 - 260,000; Full time

Job summary

Rhoda AI is hiring an Inference Optimization MLE to build and operate systems that make foundation models run fast in production. Youll squeeze performance across cloud and on-robot targets, collaborating with research to close the training-deployment gap.

Youll own end-to-end optimization, implement quantization, pruning, distillation, and compilation, and improve attention, KV caching, and memory layouts for multimodal models. A fast, impact-driven environment awaits.

Qualifications

• 3+ years of experience in inference optimization, ML systems, or a closely related field.
• Hands-on experience with modern ML stacks (PyTorch required; JAX a plus).
• Strong understanding of compute, memory bandwidth, and I/O bottlenecks in large model inference.
• Experience with optimization techniques: quantization, distillation, pruning, and compilation.
• Familiarity with inference serving frameworks like Triton, TensorRT, vLLM, TorchServe.
• Excellent debugging and measurement skills, with a track record of improvements.

Responsibilities

• Own inference performance end-to-end: diagnose and improve latency, throughput, and efficiency of large foundation models in production.
• Build systematic performance attribution: latency decomposition and bottleneck prioritization across model families.
• Apply and develop optimization techniques including quantization, pruning, distillation, and model compilation.
• Optimize attention mechanisms, KV caching, and memory layouts for multimodal models.
• Collaborate with research engineers to translate innovations into deployment-ready implementations.
• Develop benchmarking and regression detection infrastructure: latency baselines and throughput curves.
• Work with kernel-level tooling (CUDA, Triton) to identify hotspots and tune custom kernels.

Skills

Inference optimization

ML systems

PyTorch

JAX

Quantization

Pruning

Distillation

Model compilation

CUDA

Triton

TensorRT

vLLM

TorchServe

Profiling

Latency optimization

Tools

TensorRT

CUDA

TorchServe

JAX

torch.compile

Job description

At Rhoda AI, were building the next generation of generalist intelligent robots. We own the full robotics stack from high-performance hardware and robot systems to the infrastructure and state-of-the-art foundation world models that control our robots. Our robots are designed to be generalists capable of operating in complex, real-world environments and handling long-tail edge cases, made possible by our cutting edge research and end-to-end system design. Weve raised over $450M and are investing aggressively in model research, infrastructure, hardware development, and manufacturing scale-up to make generalist robotics a reality.

Were looking for an Inference Optimization MLE to help build and operate the systems that make our foundation models run fast and efficiently in production. Youll be responsible for squeezing maximum performance out of large multimodal models, across cloud and on-robot deployment targets. You will working closely with research and robotics teams to close the gap between training and real-world deployment.

What Youll Do

• Own inference performance end-to-end diagnose and improve latency, throughput, and efficiency of large foundation models in production

• Build systematic performance attribution: latency decomposition (compute vs. memory bandwidth vs. I/O), bottleneck identification, and prioritization across model families

• Apply and develop optimization techniques including quantization, pruning, distillation, operator fusion, and model compilation (e.g., TensorRT, torch.compile, XLA)

• Optimize attention mechanisms, KV caching, and memory layouts for large multimodal models (vision, video, language, proprioception)

• Work with kernel-level tooling (e.g., CUDA, Triton) to identify hotspots and implement or tune custom kernels where needed

• Build benchmarking and regression detection infrastructure: latency baselines, throughput curves, and automated detection of performance regressions across model versions

• Collaborate closely with research engineers to translate model innovations into optimized, deployment-ready implementations

What Were Looking For

• 3+ years of experience in inference optimization, ML systems, or a closely related field

• Deep hands-on experience with modern ML stacks (PyTorch required; JAX a plus)

• Strong understanding of compute, memory bandwidth, and I/O bottlenecks in large model inference

• Experience with model optimization techniques: quantization (INT8/FP8/AWQ), distillation, pruning, and compilation

• Familiarity with inference serving frameworks (e.g., Triton, TensorRT, vLLM, TorchServe)

• Exceptional debugging and measurement ability: turn inference is slow into clear bottlenecks, experiments, and validated improvements

• High ownership mindset and comfort in a fast-moving environment

Nice to Have (But Not Required)

• GPU kernel or compiler-level experience (CUDA, Triton, graph capture, operator fusion)

• Experience with multimodal or video model inference (variable-length sequences, packing/bucketing)

• Familiarity with edge/cloud hybrid deployment patterns and on-robot inference constraints

• Experience with speculative decoding, continuous batching, or other LLM serving optimizations

• Background in streaming or low-latency systems relevant to real-time robot control

Why This Role

• Direct leverage on research velocity and real-world robot performance every efficiency gain you make accelerates model iteration and tightens the loop between model and robot behavior

• Own the optimization layer that determines how quickly and efficiently our foundation models run in the real world high ownership, high impact, small elite team