Robotics Evaluation Scientist – Data, Benchmarks & Systems

Amazon; Carlisle (Cumberland County); On-site; USD 157,000 - 213,000; Full time

Benefits offered by this job

Health insurance

RSUs

401(k) matching

Paid time off

Job summary

Amazon in the United States is seeking a Research Scientist to lead evaluation frameworks and data collection protocols for robotic capabilities. You will design how we measure, stress-test, and improve robot behavior across real-world tasks, contributing to datasets and policies that accelerate system performance.

The role is cross-disciplinary, spanning robotics, machine learning, and human-in-the-loop systems, with emphasis on turning ambitious capability goals into measurable evaluation

Qualifications

• PhD or Master's with 4+ years of quantitative field research experience
• Experience applying scientific principles to business problems and products
• Experience analyzing both experimental and observational data sets

Responsibilities

• Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios
• Develop task definitions, success criteria, and benchmarking methodologies for consistent evaluation of policies
• Create and refine data collection protocols generating high-quality, task-relevant datasets aligned with model development
• Build teleoperation workflows and operator interfaces to support scalable data collection
• Analyze evaluation results to identify performance gaps and opportunities for targeted data collection
• Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines
• Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning
• Lead technical projects from conception through production deployment
• Mentor junior scientists and engineers

Tools

R

MATLAB

Python

Job description

Amazon in the United States is seeking a Research Scientist to lead evaluation frameworks and data collection protocols for robotic capabilities. You will design how we measure, stress-test, and improve robot behavior across real-world tasks, contributing to datasets and policies that accelerate system performance.

The role is cross-disciplinary, spanning robotics, machine learning, and human-in-the-loop systems, with emphasis on turning ambitious capability goals into measurable evaluation