Senior Staff AI/ML Inference Engineer

ServiceNow; Washington; Hybrid; USD 201,000 - 352,000; Full time

Benefits offered by this job

Equity

Health plans

401(k) plan with company match

ESPP

Matching donations

Flexible time away plan and family le​

Job summary

ServiceNow is seeking a Senior Staff Software Engineer - AI/ML Inference Engineering to lead architecture, optimization, and operations for our AI inference platform. You will own the end-to-end lifecycle—from onboarding cutting-edge models to deploying optimized engines in production across Kubernetes backends.

You’ll design the gateway layer with Envoy/ext_proc for access control, observability, and routing, combining GPU compute, LLM APIs, and scalable software.

Qualifications

• 8+ years of Backend software engineering experience.
• 4+ years of designing and operating distributed systems.
• Bachelor’s degree in computer science, software engineering, or a closely related technical field required.

Responsibilities

• Lead architecture, optimization, and operational excellence of AI inference platform.
• Own end-to-end inference lifecycle from onboarding models to deploying optimized engines.
• Architect intelligent gateway layer using Envoy for access control, observability, and routing.

Skills

Go or Python

OOP

Design Patterns

algorithms

LLM APIs

Distributed systems

Unit testing

Profiling

Code tuning

Education

Bachelor's degree in CS/Software Engineering or closely related field

Master's degree may offset experience

Tools

Kubernetes

Helm

CI/CD pipelines

Envoy / ext_proc

TensorRT-LLM / Triton / vLLM

Job description

ServiceNow is seeking a Senior Staff Software Engineer - AI/ML Inference Engineering to lead architecture, optimization, and operations for our AI inference platform. You will own the end-to-end lifecycle—from onboarding cutting-edge models to deploying optimized engines in production across Kubernetes backends.

You’ll design the gateway layer with Envoy/ext_proc for access control, observability, and routing, combining GPU compute, LLM APIs, and scalable software.