LLM-Powered Shopping Quality Scientist

Amazon Science; Seattle (WA); On-site; USD 143,000 - 193,000; Full time

Benefits offered by this job

Health insurance

401(k) matching

Paid time off

Parental leave

RSUs

Job summary

Amazon Science is seeking a candidate to own an LLM-based labeling and measurement pipeline, improving perception-driven defects across the shopping experience. You will design prompts, build golden datasets, and extend measurement to the homepage and detail pages.

You will create production-grade tooling for labeling, batch inference, and quality sampling, publishing standards for cross-team adoption and presenting results to stakeholders.

Qualifications

• PhD or Master's with 4+ years in CS/CE/ML or related field.
• Experience with Java, C++, Python or related language.
• Strong knowledge of algorithms and data structures, and HPC concepts.

Responsibilities

• Design and implement LLM-based labeling for perception-driven defects.
• Validate labeling quality against ground truth and maintain golden datasets.
• Extend measurement to other shopping experiences and evaluate methods.
• Build reusable labeling and evaluation tooling for large-scale data.
• Define standards for using LLMs to measure customer experience.
• Collaborate with science, engineering, and product teams to present metrics.

Skills

Java

C++

Python

Algorithms

Data structures

Parallel computing

Distributed computing

High-performance computing

Education

PhD in CS/CE/ML or related field

Master's degree with 4+ years experience

Tools

Unix/Linux

Software development

Job description

Amazon Science is seeking a candidate to own an LLM-based labeling and measurement pipeline, improving perception-driven defects across the shopping experience. You will design prompts, build golden datasets, and extend measurement to the homepage and detail pages.

You will create production-grade tooling for labeling, batch inference, and quality sampling, publishing standards for cross-team adoption and presenting results to stakeholders.