Embodied AI Evaluation Engineer

Bonfirevc; Palo Alto (CA); On-site; USD 150,000 - 190,000; Full time

Job summary

Orbifold AI is seeking a Research Engineer to design evaluation methodologies for embodied AI policies and to run robust, real-world tests. You will work with partner teams to translate failures into data specs and drive the next data collection cycle.

You will build scalable evaluation infrastructure, create fine-grained failure taxonomies, and validate policies on real hardware to demonstrate concrete improvements rather than isolated benchmarks.

Qualifications

• PhD or equivalent research experience in robot learning or closely related field.
• Real policies trained and debugged on real robots; ability to distinguish failing policy from miscalibrated robot.
• Strong PyTorch proficiency; comfortable at scale on Ray; able to work in a codebase not owned by you.

Responsibilities

• Design evaluation methodology end to end: task suites, held-out slices, scoring, and statistics.
• Build fine-grained failure taxonomies: occlusion, deformables, long-horizon, distribution shift, edge cases.
• Train and fine-tune reference policies on curated data to show data moves metrics.

Skills

PhD or equivalent

Real robot policies

PyTorch

Ray

Self-driven

Education

PhD in robotics or ML

Tools

Ray

OpenVLA

Job description

Orbifold AI is seeking a Research Engineer to design evaluation methodologies for embodied AI policies and to run robust, real-world tests. You will work with partner teams to translate failures into data specs and drive the next data collection cycle.

You will build scalable evaluation infrastructure, create fine-grained failure taxonomies, and validate policies on real hardware to demonstrate concrete improvements rather than isolated benchmarks.