AGI Quality & Data Science Manager - Automation & Auditing

Amazon; Bellevue (WA); On-site; USD 184,000 - 249,000; Full time

Benefits offered by this job

Health insurance

RSUs

Paid time off

Parental leave

Job summary

Amazon seeks an Applied Science Manager for the AGI team to lead a group of scientists and engineers focused on data quality, auditing, and automated evaluation of LLMs and multimodal systems. You will design quality frameworks, recruit and coach staff, and partner with senior leadership to shape product strategy.

The role combines people leadership with hands-on ML work, including building auditing rubrics, judge architectures, and data-collection workflows, to deliver high-quality training

Qualifications

• 3+ years of scientists or machine learning engineers management experience.
• Knowledge of ML, NLP, Information Retrieval and Analytics.

Responsibilities

• Lead and mentor a team of Applied Scientists to develop quality strategies and auditing frameworks.
• Design auditing strategies with SOPs, quality metrics, and sampling methodologies.
• Oversee expert-level manual audits, meta-audits, and coaching to uplift quality.
• Manage development of LLM-as-a-Judge systems and evaluation models.
• Recruit, hire, and develop team members; communicate with senior leadership.

Skills

ML leadership

Quality assurance leadership

Job description

Amazon seeks an Applied Science Manager for the AGI team to lead a group of scientists and engineers focused on data quality, auditing, and automated evaluation of LLMs and multimodal systems. You will design quality frameworks, recruit and coach staff, and partner with senior leadership to shape product strategy.

The role combines people leadership with hands-on ML work, including building auditing rubrics, judge architectures, and data-collection workflows, to deliver high-quality training