ML Inference Infra Engineer for Robotic AI

Rhoda AI; Mountain View (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Rhoda AI is building the next generation of generalist intelligent robots and operates a full robotics stack from hardware to world-models. We seek a Software Engineer to run large foundation models efficiently across cloud and on‑prem environments, focusing on resource management and infrastructure scalability.

You will design Kubernetes-based deployment pipelines, own GPU scheduling across clusters, and integrate ML frameworks such as Triton, Ray Serve, and TorchServe to support fast iteration

Qualifications

• 3+ years of experience in ML infrastructure, MLOps, or distributed systems.
• Strong proficiency with Kubernetes and containerized deployment pipelines.
• Experience with GPU orchestration and resource scheduling across large distributed jobs.
• Experience with cloud providers (e.g., AWS, GCP) and hybrid cloud/on-prem infrastructure.
• Familiarity with ML frameworks (e.g., PyTorch, JAX) and model serving tools (e.g., Triton, Ray Serve, TorchServe).
• Strong debugging instincts and ownership mentality — comfortable driving issues to resolution across the stack.

Responsibilities

• Design and operate large-scale infrastructure to run model workloads across cloud and on-prem environments.
• Build and maintain Kubernetes-based deployment pipelines for managing distributed ML workloads.
• Own resource scheduling and orchestration across GPU clusters — optimizing utilization, workload balancing, and cost-performance tradeoffs.
• Integrate and manage ML frameworks and model serving systems (e.g., Triton, Ray Serve, TorchServe) across research and production use cases.
• Build tooling for model deployment, versioning, and observability to support fast iteration cycles.
• Contribute to the reliability and scalability of the infrastructure stack as model complexity and deployment footprint grow.

Skills

Kubernetes

ML Infrastructure

Distributed systems

Cloud (AWS/GCP)

GPU scheduling

Model serving

Tools

Triton

Ray Serve

TorchServe

Job description

Rhoda AI is building the next generation of generalist intelligent robots and operates a full robotics stack from hardware to world-models. We seek a Software Engineer to run large foundation models efficiently across cloud and on‑prem environments, focusing on resource management and infrastructure scalability.

You will design Kubernetes-based deployment pipelines, own GPU scheduling across clusters, and integrate ML frameworks such as Triton, Ray Serve, and TorchServe to support fast iteration