Robotics Evaluation Scientist: Data-Driven Teleoperation ML

Amazon Inc.; New York (NY); On-site; USD 172,000 - 223,000; Full time

Job summary

Amazon.com Services LLC is seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will design how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks.

You will build infrastructure that connects teleoperation, evaluation, and learning, define task structures and success criteria, and develop operator-friendly interfaces to enable scalable data collection.

Qualifications

• 3+ years of building models for business application experience.
• Experience in patents or publications at top-tier peer-reviewed conferences or journals.
• Experience programming in Java, C++, Python or related language.
• Algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, HPC.

Responsibilities

• Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios.
• Develop task definitions, success criteria, and benchmarking methodologies for reproducible evaluation of policies.
• Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs.
• Build and iterate on teleoperation workflows and operator interfaces to support scalable data collection.
• Analyze evaluation results to identify performance gaps and opportunities for targeted data collection.
• Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the robotics stack.
• Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to improve internal approaches.
• Lead technical projects from conception through production deployment.
• Mentor junior scientists and engineers.

Skills

Java

C++

Python

Tools

Unix/Linux

Software development

Job description

Amazon.com Services LLC is seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will design how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks.

You will build infrastructure that connects teleoperation, evaluation, and learning, define task structures and success criteria, and develop operator-friendly interfaces to enable scalable data collection.