Senior ML Engineer (DevOps) - AI Inference on Kubernetes

ServiceNow; California; On-site; USD 201,000 - 352,000; Full time

Benefits offered by this job

Health plans

401(k) Plan with company match

ESPP

Matching donations

Flexible time away plan

Family leave programs

Job summary

ServiceNow is seeking a Senior Staff Machine Learning Engineer (DevOps) to lead the architecture, optimization, and operational excellence of our AI inference platform. You will own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes infrastructure.

You will architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable

Qualifications

• 8+ years of backend software engineering experience.
• 4+ years designing and operating distributed systems.
• 3+ years Kubernetes in production, including Helm and CI/CD.

Responsibilities

• Own end-to-end inference lifecycle, from onboarding models to deploying optimized inference engines.
• Architect the gateway layer using Envoy for access control, observability and routing.
• Improve performance with GPU compute, LLM APIs, inference frameworks, containers and orchestration.
• Collaborate with teams to deliver AI-powered work experiences.

Skills

Go

Python

OOP

Design Patterns

Algorithms

AI/ML knowledge

Education

Bachelor's degree in CS/SE

Master's degree offset

Tools

Envoy

Kubernetes

Helm

CI/CD

Job description

ServiceNow is seeking a Senior Staff Machine Learning Engineer (DevOps) to lead the architecture, optimization, and operational excellence of our AI inference platform. You will own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes infrastructure.

You will architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable