Applied Scientist: Agent-Centric CX & AI Metrics

Amazon Inc.; Seattle (WA); On-site; USD 143,000 - 193,000; Full time

Job summary

Amazon’s CXBT team is seeking an Applied Scientist to advance the science of evaluating agent-driven builder experiences on AWS. You will design principled measurement methodologies for autonomous agents, implement rigorous experiments, and reason about architectural trade-offs in cloud services.

The role requires expertise in GenAI and NLP, with strong publication or patent credentials, and the ability to work across scientists, engineers, and product managers in a fast-paced environment.

Qualifications

• 3+ years building models for business applications.
• PhD, or Master’s in CS, CE, ML or related field.
• Publications in top-tier peer-reviewed venues or patents are a plus.
• Experience programming in Java, C++, Python or related language.
• Hands-on ML models for NLP tasks using GenAI.

Responsibilities

• Develop novel scientific methodologies for evaluating autonomous, tool-using agents, advancing agent behavioral measurement.
• Design rigorous, reproducible experiments yielding causal insights into agent decision-making.
• Apply cloud-architecture knowledge to reason about service trade-offs and deployment considerations.
• Develop verification and synthesis methods ensuring evidence-grounded conclusions.

Skills

ML/NLP models

GenAI experience

Cloud architecture

Education

PhD or MS in CS/ML

Tools

Java

C++

Python

AWS tooling

Job description

Amazon’s CXBT team is seeking an Applied Scientist to advance the science of evaluating agent-driven builder experiences on AWS. You will design principled measurement methodologies for autonomous agents, implement rigorous experiments, and reason about architectural trade-offs in cloud services.

The role requires expertise in GenAI and NLP, with strong publication or patent credentials, and the ability to work across scientists, engineers, and product managers in a fast-paced environment.