Applied Science Manager, LLM Quality & Evaluation

Amazon Science; Bellevue (WA); On-site; USD 184,000 - 249,000; Full time

Job summary

Amazon Science is seeking an Applied Science Manager for the AGI team to lead a group of Applied Scientists focused on data quality, auditing, and evaluation systems for LLMs and multimodal models.

You will build auditing strategies, oversee expert-level audits, and coach the team to raise quality capabilities while collaborating with senior leaders and cross-functional partners to ship high-impact AI systems.

Qualifications

• 3+ years of scientists or machine learning engineers management experience.
• Knowledge of ML, NLP, Information Retrieval and Analytics.

Responsibilities

• Lead and mentor Applied Scientists to develop quality strategies and auditing frameworks.
• Oversee data collection workflows, auditing, and judge architectures for automated QA.
• Recruit, hire, and develop team members; conduct performance reviews and growth plans.
• Communicate requirements and product strategy with leadership and cross-functional teams.

Skills

ML

NLP

Information Retrieval

Analytics

Management

Job description

Amazon Science is seeking an Applied Science Manager for the AGI team to lead a group of Applied Scientists focused on data quality, auditing, and evaluation systems for LLMs and multimodal models.

You will build auditing strategies, oversee expert-level audits, and coach the team to raise quality capabilities while collaborating with senior leaders and cross-functional partners to ship high-impact AI systems.