Lead Scientist - Human-AI Collaboration & Evaluation

Amazon Web Services (AWS); Seattle (WA); On-site; USD 199,000 - 269,000; Full time

Job summary

Amazon Development Center U.S., Inc. seeks a scientist to lead the scientific agenda for human-AI collaboration in agentic and multimodal systems.

You will convert behavioral goals into end-to-end plans covering data, architecture, evaluation, and product experiments, and you’ll code weekly with the team. The role starts hands-on with product and engineering, then evolves into a product-team setting to drive adoption of proven methods.

Qualifications

• PhD in computer science, machine learning, AI, or related field; or Master's with equivalent applied science experience.
• Experience developing agentic AI systems or LLM applications end-to-end.
• Experience designing evaluation for systems without standard benchmarks.
• Experience writing substantial research or production code in Python or equivalent.
• Experience leading complex scientific work with engineering and product partners.
• Evidence of independent decisions in ambiguous technical environments.

Responsibilities

• Own the scientific strategy for human-AI collaboration across agentic and multimodal systems.
• Translate desired interactions into falsifiable hypotheses, evaluation tasks, and system requirements.
• Build an evaluation system that measures teammate behavior, trust, adaptation, and recovery.
• Design agent systems with memory, tools, planning, and recovery; test with end users.
• Design data collection and curation for language, speech, and interaction traces.
• Develop substantial parts of the experimental stack and related components.
• Diagnose failures across data, models, orchestration, evaluation, and product interaction.
• Define requirements to turn proven methods into product capabilities.
• Collaborate with design, product, engineering, and behavioral research through validation.
• Run experiments with partner teams and report results and changes.

Skills

Agentic AI concepts

End-to-end product cycle

Evaluation design

Independent decision making

Leadership in science

Mentoring

Education

PhD in CS/ML or related field

Master's degree with applicable experience

Tools

Python

Experimentation stack development

Job description

Amazon Development Center U.S., Inc. seeks a scientist to lead the scientific agenda for human-AI collaboration in agentic and multimodal systems.

You will convert behavioral goals into end-to-end plans covering data, architecture, evaluation, and product experiments, and you’ll code weekly with the team. The role starts hands-on with product and engineering, then evolves into a product-team setting to drive adoption of proven methods.