Senior AI Evaluation & GenAI Benchmarking Lead

ServiceNow; Santa Clara (CA); On-site; USD 201,300 - 352,300; Full time

Job summary

ServiceNow is seeking a Senior Engineering Manager to establish and lead AI evaluation practices for Now Assist agentic workflows. You will build infrastructure, rigorous validation frameworks, and benchmarks that quantify performance across multi-step orchestration, tool-calling, and enterprise-grounded reasoning.

You will bridge frontier AI research and production metrics, directly impacting trust and adoption of autonomous workflows for millions of users.

Qualifications

• 8+ years of professional software engineering or machine learning experience.
• 3+ years managing or technically leading AI/ML teams.
• Experience integrating AI into enterprise work processes.
• Strong knowledge of frontier AI SDKs and agentic architectures.
• Experience implementing AI metrics at enterprise scale.
• Bachelor’s or higher degree in Computer Science, Data Science, ML, or related quantitative field.

Responsibilities

• Build and scale AI evaluation infrastructure, unit/integration tests, and production drift monitors.
• Define enterprise AI benchmarks for complex business workflows and multi-agent orchestration.
• Validate RAG pipelines, hybrid search, and semantic re-ranking systems.
• Benchmark frontier LLMs for latency, context-window efficiency, and costs.
• Lead a high-performing AI-native engineering team and promote best practices.
• Collaborate with Core Product, ML Platforms, and Engineering leads to drive product improvements.

Skills

Senior AI/ML leadership

Python proficiency

AI evaluation metrics

Multi-agent orchestration

Data analytics (SQL, Pandas, NumPy)

MLOps tracking platforms

Education

Bachelor's or higher in CS/DS/ML

Tools

Python

SQL

Pandas

NumPy

MLOps platforms

Job description

ServiceNow is seeking a Senior Engineering Manager to establish and lead AI evaluation practices for Now Assist agentic workflows. You will build infrastructure, rigorous validation frameworks, and benchmarks that quantify performance across multi-step orchestration, tool-calling, and enterprise-grounded reasoning.

You will bridge frontier AI research and production metrics, directly impacting trust and adoption of autonomous workflows for millions of users.