Senior Staff ML Engineer - AI Inference & Kubernetes

ServiceNow; Santa Clara (CA); On-site; USD 201,000 - 352,000; Full time

Benefits offered by this job

Health plans

401(k) Plan with company match

ESPP

Matching donations

Flexible time away plan

Family leave programs

Job summary

ServiceNow is seeking a Senior Staff Machine Learning Engineer (DevOps) to lead architecture, optimization, and operations of our AI inference platform in a production Kubernetes environment. You will own the end-to-end lifecycle from onboarding models to deploying optimized inference engines across backends and backends on Kubernetes.

You will architect the gateway layer with Envoy/ext_proc for access control, observability, and routing, balancing pragmatic engineering with measurable

Qualifications

• 8+ years of Backend software engineering experience.
• 4+ years designing and operating distributed systems.
• 3+ years Kubernetes in production, including Helm and CI/CD.
• Bachelor’s degree in computer science, software engineering, or a closely related technical field.

Responsibilities

• Lead architecture, optimization, and operational excellence of the AI inference platform.
• Own the end-to-end inference lifecycle from onboarding models to production deployment.
• Architect the intelligent gateway layer using Envoy and ext_proc for access control and observability.
• Ensure high performance with GPU compute, LLM APIs, and inference backends.
• Collaborate with researchers, engineers, and product managers to deliver AI-powered work experiences.

Skills

Go

Python

OOP

Design Patterns

Algorithms

Distributed systems

Kubernetes

Helm

CI/CD

Envoy

ext_proc

GitHub Copilot

Observability

Incident response

Education

Bachelor's degree in computer science

Master's degree preferred

Tools

Kubernetes

Helm

CI/CD

Envoy

ext_proc

GitHub Copilot

Cursor

Job description

ServiceNow is seeking a Senior Staff Machine Learning Engineer (DevOps) to lead architecture, optimization, and operations of our AI inference platform in a production Kubernetes environment. You will own the end-to-end lifecycle from onboarding models to deploying optimized inference engines across backends and backends on Kubernetes.

You will architect the gateway layer with Envoy/ext_proc for access control, observability, and routing, balancing pragmatic engineering with measurable