Staff AI Engineer, Enterprise-Scale Orchestration

ServiceNow; Santa Clara (CA); On-site; USD 176,000 - 308,000; Full time

Benefits offered by this job

Health plans

401(k) Plan with company match

ESPP

Matching donations

Flexible time away plan

Family leave programs

Job summary

ServiceNow in Santa Clara, CA is seeking a Staff AI Engineer to own the agent harness, enabling AI agents to reason over enterprise data and act across workflows at Fortune 500 scale.

You will design the orchestration layer, ensure reliability, observability, and cost awareness, and drive production AI discipline through architecture decisions, code reviews, and mentoring. This role sits at the intersection of AI, software engineering, and enterprise-scale platforms.

Qualifications

• 7+ years building production software systems with a strong track record on reliability, performance, and scalability.
• Hands-on experience shipping generative AI products — not just integrating LLM APIs or building prototypes, but owning AI-powered features that production users depend on.
• Solid depth in how large language models work: failure modes, context constraints, and how prompt design shapes model behavior at scale.
• Python, Kubeflow,Experience with SageMaker, Vertex AI or AzureML, SQL
• Practical prompt engineering experience: systematically designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles
• A real track record in eval engineering — not just familiarity, but a portfolio of evaluation suites designed, shipped, and used to drive quality decisions in production AI systems
• Cost and efficiency awareness at the system level: experience reasoning about model routing, inference cost, and latency tradeoffs in production
• Strong software engineering fundamentals: distributed systems, API design, and testing discipline
• Comfort operating in fast-moving, ambiguous, startup-like AI product environments

Responsibilities

• Harness engineering: Design and build the agent execution harness — the orchestration layer that routes inputs, manages context, invokes tools, handles retries, and surfaces execution state across multi-step agentic workflows
• Reliability at scale: Own the runtime's fault tolerance, latency, and throughput; design for enterprise workflows that cannot fail silently or unpredictably
• Observability: Instrument the harness with tracing, cost attribution, and latency visibility so the team can reason about agent behavior in production and catch failures before customers do
• Prompt infrastructure: Build prompt management systems — versioning, templating, and systematic evaluation — that keep agent behavior stable across model updates and configuration changes
• Eval engineering: Design and own evaluation frameworks (unit evals, integration evals, production monitors) that measure agent quality, catch regressions, and drive data-informed decisions
• LLM integration: Integrate with and abstract over frontier LLMs, managing model routing, fallback strategies, cost, and latency trade - offs in production
• Technical leadership: Set technical standards through architecture decisions, code reviews, and coaching — particularly on agentic design patterns and production AI discipline.
• System boundary design: Define where agent logic lives — what's a tool call, a sub-agent, a hardcoded path, or a human escalation — and establish those design standards across the team

Skills

Production software

Generative AI

Large language models

Python

Kubeflow

SageMaker

Vertex AI

AzureML

SQL

Prompt engineering

Eval engineering

Cost and efficiency

Distributed systems

API design

Testing discipline

Startup-like environments

Tools

LangChain

LlamaIndex

Job description

ServiceNow in Santa Clara, CA is seeking a Staff AI Engineer to own the agent harness, enabling AI agents to reason over enterprise data and act across workflows at Fortune 500 scale.

You will design the orchestration layer, ensure reliability, observability, and cost awareness, and drive production AI discipline through architecture decisions, code reviews, and mentoring. This role sits at the intersection of AI, software engineering, and enterprise-scale platforms.