Senior Inference Optimization Engineer for Multivendor AI

Roboticsconnected; Mountain View (WY); On-site; USD 150,000 - 210,000; Full time

Job summary

Rhoda AI is seeking an Inference Optimization engineer to accelerate large multimodal foundation models across diverse hardware, from AMD ROCm to TPU and consumer GPUs. You will balance latency, throughput, and cost while ensuring correctness and portability across platforms.

You will work closely with research, robotics, and infrastructure teams to deploy on Jetson/Thor-class embedded compute and decide how to split workloads between robot and cloud to meet real-time control requirements.

Qualifications

• 2+ years of inference optimization or ML systems work.
• Deep hands-on experience with PyTorch.
• Strong understanding of compute, memory bandwidth and interconnect bottlenecks across architectures.
• Experience with model optimization: quantization, distillation, pruning, and compilation tools.
• Familiarity with inference serving frameworks across hardware.
• High-ownership, pragmatic mindset focusing on total cost and performance.

Responsibilities

• Own end-to-end inference optimization for Rhoda's foundation models.
• Port and optimize models for non-NVIDIA and lower-cost hardware.
• Deploy and optimize on Jetson/Thor-class embedded compute with real-time constraints.
• Build cost/performance models and write actionable hardware recommendations.
• Develop hardware-portable kernel tuning across vendors when possible.
• Apply model compression techniques (quantization, pruning, distillation).
• Ensure numerical parity and robust comparisons across backends.
• Develop benchmarking and regression tests for multiple targets.

Skills

PyTorch

Performance optimization

ML systems

MLIR/Compiler tech

XLA/TensorRT/TVM

Debugging/measurement

Ownership mindset

Heterogeneous hardware knowhow

Hardware-software co-design

Tools

torch.compile

XLA

TensorRT

IREE/MLIR

TVM

vendor compilers

Job description

Rhoda AI is seeking an Inference Optimization engineer to accelerate large multimodal foundation models across diverse hardware, from AMD ROCm to TPU and consumer GPUs. You will balance latency, throughput, and cost while ensuring correctness and portability across platforms.

You will work closely with research, robotics, and infrastructure teams to deploy on Jetson/Thor-class embedded compute and decide how to split workloads between robot and cloud to meet real-time control requirements.