Senior AI Evaluation & GenAI Benchmarking Lead

ServiceNow

Santa Clara (CA)

On-site

USD 201,300 - 352,300

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

ServiceNow is seeking a Senior Engineering Manager to establish and lead AI evaluation practices for Now Assist agentic workflows. You will build infrastructure, rigorous validation frameworks, and benchmarks that quantify performance across multi-step orchestration, tool-calling, and enterprise-grounded reasoning.

You will bridge frontier AI research and production metrics, directly impacting trust and adoption of autonomous workflows for millions of users.

Qualifications

  • 8+ years of professional software engineering or machine learning experience.
  • 3+ years managing or technically leading AI/ML teams.
  • Experience integrating AI into enterprise work processes.
  • Strong knowledge of frontier AI SDKs and agentic architectures.
  • Experience implementing AI metrics at enterprise scale.
  • Bachelor’s or higher degree in Computer Science, Data Science, ML, or related quantitative field.

Responsibilities

  • Build and scale AI evaluation infrastructure, unit/integration tests, and production drift monitors.
  • Define enterprise AI benchmarks for complex business workflows and multi-agent orchestration.
  • Validate RAG pipelines, hybrid search, and semantic re-ranking systems.
  • Benchmark frontier LLMs for latency, context-window efficiency, and costs.
  • Lead a high-performing AI-native engineering team and promote best practices.
  • Collaborate with Core Product, ML Platforms, and Engineering leads to drive product improvements.

Skills

Senior AI/ML leadership
Python proficiency
AI evaluation metrics
Multi-agent orchestration
Data analytics (SQL, Pandas, NumPy)
MLOps tracking platforms

Education

Bachelor's or higher in CS/DS/ML

Tools

Python
SQL
Pandas
NumPy
MLOps platforms

Job description

ServiceNow is seeking a Senior Engineering Manager to establish and lead AI evaluation practices for Now Assist agentic workflows. You will build infrastructure, rigorous validation frameworks, and benchmarks that quantify performance across multi-step orchestration, tool-calling, and enterprise-grounded reasoning.

You will bridge frontier AI research and production metrics, directly impacting trust and adoption of autonomous workflows for millions of users.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI-Enabled ServiceNow Consultant
Senior AI-Enabled ServiceNow Consultant

Ahead Now Aps • United States

On-site
USD 160,000 - 190,000
Medical, Dental, and Vision Insurance
Senior AI Architect for ServiceNow Platforms
Senior AI Architect for ServiceNow Platforms

Medium • Chicago (IL)

On-site
USD 160,000 - 190,000
Medical Insurance
Dental Insurance
Vision Insurance
Senior AI-Native Engineering Manager
Senior AI-Native Engineering Manager

ServiceNow • California

On-site
USD 201,000 - 352,000
Health plans
401(k) match
ESPP
+2
Senior AI-Enabled ServiceNow Consultant
Senior AI-Enabled ServiceNow Consultant

Remote • San Francisco (CA)

On-site
USD 180,000 - 260,000
Health insurance
Paid time off
Flexible work arrangements
+3
Senior AI Platform Engineering Manager
Senior AI Platform Engineering Manager

ServiceNow • San Diego (CA)

On-site
USD 171,900 - 300,800
Equity (when applicable)
Health plans
401(k) Plan with company match
+4
Senior GenAI Engineer for Enterprise AI & Production
Senior GenAI Engineer for Enterprise AI & Production

ServiceNow • California

Hybrid
USD 161,000 - 274,000
Health plans
401(k) plan with company match
ESPP
+3
Senior AI Adoption Architect & Escalation Lead
Senior AI Adoption Architect & Escalation Lead

ServiceNow • San Diego (CA)

On-site
USD 199,000 - 348,000
Health plans
401(k) Plan with company match
ESPP
+1
Senior AI Enablement Architect
Senior AI Enablement Architect

ServiceNow • Santa Clara (CA)

On-site
USD 114,000 - 199,000
Health plans
401(k) Plan with company match
ESPP
+3
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations

ServiceNow • Santa Clara (CA)

On-site
USD 201,300 - 352,300
Senior Staff AI Platform Architect
Senior Staff AI Platform Architect

ServiceNow • Santa Clara (CA)

Hybrid
USD 191,000 - 334,000
Health plans
401k Plan
ESPP
+3