Applied Scientist, Support Agent Intelligence and Evaluation (Advertising)

Amazon; Seattle (WA); On-site; USD 143,000 - 193,000; Full time

Benefits offered by this job

Health insurance

RSUs

401(k) matching

Paid time off

Parental leave

Job summary

Amazon Advertising is seeking an Applied Scientist to join the Support Agent Intelligence and Evaluation team in Seattle. You will enhance agent capabilities across Sponsored Products, Brands, DSP and more, designing evaluation frameworks and building retrieval and generation models with strict latency SLAs.

You will drive multilingual evaluation, partner with product and engineering teams, and translate research into product decisions while shaping metrics that truly measure advertiser success.

Qualifications

• 3+ years of building models for business application experience.
• PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience
• Experience in patents or publications at top-tier peer-reviewed conferences or journals
• Experience programming in Java, C++, Python or related language
• Experience in algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing

Responsibilities

• Enhance support agent capabilities across the broad suite of Amazon Advertising products.
• Design and own the evaluation framework for agent quality, including automated LLM-based scoring of answer correctness, confidence calibration, and conversation-level resolution signals
• Develop novel metrics that capture whether advertisers actually got the help they needed (beyond surface-level deflection rates)
• Build and improve retrieval and generation models that power real-time advertiser interactions under strict latency SLAs
• Drive multilingual science — improve non-English resolution rates through cross-lingual retrieval, translation quality modeling, and locale-aware evaluation
• Partner with product, engineering, and business teams to productize research and inform roadmap decisions with data

Skills

Model development

Java

C++

Python

Algorithms

Education

PhD or MS + CS/CE/ML

Tools

Unix/Linux

Job description

Applied Scientist, Support Agent Intelligence and Evaluation

Job ID: 10481484 | Amazon.com Services LLC

Amazon Advertising is one of Amazon's fastest growing and most profitable businesses. Amazon's advertising portfolio helps merchants, retail vendors, and brand owners succeed via native advertising, which grows incremental sales of their products sold through Amazon. The primary goals are to help shoppers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and build a sustainable business that continuously innovates on behalf of customers.

Millions of advertisers rely on Amazon's self-service support experience to resolve issues, unblock campaigns, and grow their business. Our Support Agents team is building the science behind intelligent, conversational support — systems that understand advertiser intent, retrieve the right knowledge, generate accurate answers, and know when to elevate. We serve ~2M monthly active advertisers across dozens of locales and languages, and every percentage point of improvement in resolution quality translates directly into advertiser success and retention.

A day in the life

1. Enhance support agent capabilities across the broad suite of Amazon Advertising products — expanding coverage, depth of resolution, and advertiser task completion across Sponsored Products, Sponsored Brands, DSP, AMC, and more
2. Design and own the evaluation framework for agent quality — including automated LLM-based scoring of answer correctness, confidence calibration, and conversation-level resolution signals
3. Develop novel metrics that capture whether advertisers actually got the help they needed (beyond surface-level deflection rates)
4. Build and improve retrieval and generation models that power real-time advertiser interactions under strict latency SLAs
5. Drive multilingual science — improve non-English resolution rates through cross-lingual retrieval, translation quality modeling, and locale-aware evaluation
6. Partner with product, engineering, and business teams to productize research and inform roadmap decisions with data

About the team

This role sits within Amazon Advertising's broader Agentic Intelligence organization — a community of multiple Applied Science and Engineering teams building the next generation of AI-powered experiences for advertisers. You'll have access to Principal Engineers and Principal Applied Scientists to pressure-test ideas and elevate your work. The team is cross-functional — scientists and engineers work shoulder-to-shoulder, from problem framing through production deployment.

Basic Qualifications

• 3+ years of building models for business application experience
• PhD, or Master's degree and 4+ years of CS, CE, ML or related field experience
• Experience in patents or publications at top-tier peer-reviewed conferences or journals
• Experience programming in Java, C++, Python or related language
• Experience in any of the following areas: algorithms and data structures, parsing, numerical optimization, data mining, parallel and distributed computing, high-performance computing

Preferred Qualifications

• Experience using Unix/Linux
• Experience in professional software development
• Experience in advertising

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits .

USA, WA, SEATTLE - 142,800.00 - 193,200.00 USD annually

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.