Applied Science Manager, AGI Quality & Evaluation

Amazon; Boston (MA); On-site; USD 184,000 - 249,000; Full time

Benefits offered by this job

RSUs

Health insurance

401(k) matching

Paid time off

Parental leave

Job summary

Amazon is seeking an Applied Science Manager for the AGI Frontier AI Assets: Assessments team to lead a group of scientists in developing quality strategies, auditing frameworks, and automated evaluation systems for LLMs and multimodal models.

You will recruit, coach, and set growth plans, partner with senior leadership, and drive data-quality improvements that power state-of-the-art AI products at Amazon.

Qualifications

• 3+ years of scientists or machine learning engineers management experience.
• Knowledge of ML, NLP, Information Retrieval and Analytics.

Responsibilities

• Lead and mentor a team of Applied Scientists on quality strategies and auditing.
• Design auditing strategies with SOPs, quality metrics, and sampling methodologies.
• Oversee development of LLM-as-a-Judge systems and evaluation rubrics.
• Recruit, hire, and develop team members; conduct performance reviews.

Skills

ML management

ML/NLP/IR/Analytics

Job description

Amazon is seeking an Applied Science Manager for the AGI Frontier AI Assets: Assessments team to lead a group of scientists in developing quality strategies, auditing frameworks, and automated evaluation systems for LLMs and multimodal models.

You will recruit, coach, and set growth plans, partner with senior leadership, and drive data-quality improvements that power state-of-the-art AI products at Amazon.