Senior Staff AI/ML Inference Platform Engineer

ServiceNow; California; Hybrid; USD 181,000 - 317,000; Full time

Benefits offered by this job

Health plans

Flexible spending accounts

401(k) with company match

ESPP

Matching donations

Flexible time away plan

Family leave programs

Job summary

ServiceNow is seeking a Senior Staff Software Engineer - AI/ML Inference Engineering in California to lead architecture, optimization, and production readiness of the AI inference platform. You will own the end-to-end inference lifecycle, from onboarding models to deploying optimized engines across backends on Kubernetes, and design the gateway layer with Envoy for access control and observability.

You will work with Go or Python, ML-serving systems, and modern tooling to deliver scalable,

Qualifications

• 8+ years of Backend software engineering experience.
• 4+ years designing and operating distributed systems.
• Bachelor’s degree in computer science, software engineering, or a closely related technical field required.
• A master’s degree may offset up to one year of the experience minimum.

Responsibilities

• Lead architecture, optimization, and operational excellence of AI inference platform.
• Own end-to-end inference lifecycle from onboarding models to deploying optimized inference engines.
• Architect intelligent gateway layer using Envoy and ext_proc for access control, observability, and routing.
• Collaborate across teams to build products and end-to-end AI-powered work experiences.
• Demonstrate pragmatic engineering with measurable performance impact.

Skills

Go

Python

OOP

Design Patterns

Algorithms

Model-serving

LLM APIs

Production software

Unit testing

Profiling

Code tuning

Education

Bachelor's degree in CS/Software Engineering

Master's degree offset up to 1 year experience

Tools

vLLM

TensorRT-LLM

Triton Inference Server

Envoy

ext_proc

Kubernetes

Helm

CI/CD

Job description

ServiceNow is seeking a Senior Staff Software Engineer - AI/ML Inference Engineering in California to lead architecture, optimization, and production readiness of the AI inference platform. You will own the end-to-end inference lifecycle, from onboarding models to deploying optimized engines across backends on Kubernetes, and design the gateway layer with Envoy for access control and observability.

You will work with Go or Python, ML-serving systems, and modern tooling to deliver scalable,