Applied Scientist II LLM-Powered Shopping Quality

Amazon; Seattle (WA); On-site; USD 143,000 - 193,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

RSUs

Job summary

Amazon is seeking a researcher to design and scale LLM-based labeling for perception-driven defects across the shopping experience. You will build reusable tooling, validate labeling against ground truth, and extend measurement methodologies to homepages and detail pages, ensuring reliable, scalable quality metrics at Amazon scale.

The role emphasizes collaboration with science, engineering and product teams, publishing standards, and presenting results to stakeholders while advancing labeling

Qualifications

• PhD, or Master’s degree and 4+ years of CS, CE, ML or related field experience.
• Experience programming in Java, C++, Python or related language.
• Experience in algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing.

Responsibilities

• Design and improve LLM-based labeling for perception-driven defects: prompt design, sampling strategy, and the split between model judgment and human annotation.
• Validate labeling quality against human ground truth, and build and maintain the golden datasets that make that validation possible.
• Extend perceived-duplicate measurement to other parts of the shopping experience, designing the methodology where none exists and evaluating approaches already in use where one does.
• Build reusable, production-grade labeling and evaluation tooling, batch inference, quality sampling, prompt and model version control, that operates on Amazon-scale data.
• Define and publish the standards other teams adopt for using LLMs to measure customer experience.
• Partner with science, engineering, and product teams across Stores to make quality measurement usable in their decisions, and present metric results and methodology changes to stakeholders.

Skills

Java

C++

Python

Education

PhD or Master’s degree

Tools

Unix/Linux

Job description

Amazon is seeking a researcher to design and scale LLM-based labeling for perception-driven defects across the shopping experience. You will build reusable tooling, validate labeling against ground truth, and extend measurement methodologies to homepages and detail pages, ensuring reliable, scalable quality metrics at Amazon scale.

The role emphasizes collaboration with science, engineering and product teams, publishing standards, and presenting results to stakeholders while advancing labeling