Senior ML Engineer, DevOps - AI Inference Platform

ServiceNow; Santa Clara (CA); Hybrid; USD 201,000 - 352,000; Full time

Benefits offered by this job

Health plans

401(k) Plan

ESPP

Matching donations

Flexible time away

Family leave programs

Job summary

ServiceNow is seeking a Senior Staff Machine Learning Engineer (DevOps) to lead the AI inference platform on Kubernetes. You will own the end-to-end lifecycle, from onboarding models to deployment of optimized engines across backends, and you will architect an intelligent gateway using Envoy for access control and observability.

You will work with Go/Python, maintain production-grade systems, and drive performance improvements while collaborating with a broad AI Services Platform group.

Qualifications

• 8+ years of Backend software engineering experience.
• 4+ years designing and operating distributed systems.
• 3+ years Kubernetes in production, including Helm and CI/CD.
• Bachelor’s degree in computer science, software engineering, or a closely related technical field.

Responsibilities

• Own end-to-end inference lifecycle from onboarding models to deploying inference engines.
• Architect the gateway layer using Envoy for access control, observability and routing.
• Develop and optimize AI inference across multiple backends on Kubernetes.
• Ensure instrumented observability with metrics, logs, tracing and alerts.
• Collaborate with researchers and engineers to push performance impact.

Skills

Go or Python

OOP

Design Patterns

Time/Space Efficient Algorithms

AI/ML fundamentals

Unit testing

Profiling

Code tuning

Distributed systems

Education

Bachelor's degree in Computer Science/Software Engineering or related field

Tools

Kubernetes

Helm

CI/CD

Envoy

API gateway

Go

Python

Job description

ServiceNow is seeking a Senior Staff Machine Learning Engineer (DevOps) to lead the AI inference platform on Kubernetes. You will own the end-to-end lifecycle, from onboarding models to deployment of optimized engines across backends, and you will architect an intelligent gateway using Envoy for access control and observability.

You will work with Go/Python, maintain production-grade systems, and drive performance improvements while collaborating with a broad AI Services Platform group.