Senior Distributed Systems Engineer, AI Inference Factory

NVIDIA Corporation; Santa Clara (CA); On-site; USD 184,000 - 288,000; Full time

Job summary

NVIDIA is seeking a senior engineer to design and build factory infrastructure and automation for NVIDIA Inference Microservices (NIMs). You will architect scalable pipelines, interfaces, data models, and observability to validate AI models across heterogeneous hardware and software environments.

You'll work with leaders to define strategies, mentor teams, and drive advancements in Docker, Kubernetes, and cloud tooling to accelerate NIM deployment from development to production.

Qualifications

• Experience building distributed compute systems and microservices.
• Proven ability to mentor and lead teams across functions.
• Deep expertise with Docker, Kubernetes, and cloud tooling.
• BS or MS in CS/CE or related field (or equivalent experience).
• 8+ years of experience in performant microservice and cloud software roles.
• Strong collaboration with architects and multi-functional teams.
• Experience with CI/CD pipelines and automated testing.

Responsibilities

• Design and implement a scalable factory pipeline for model deployment across Cloud, On-Prem, and Kubernetes.
• Collaborate with AI model teams to understand requirements and improve productivity.
• Define metrics, observability, and drive performance improvements across the factory pipeline.
• Mentor team members and foster cross-team collaboration.

Skills

Distributed systems

Microservices

Cloud technologies

Backend services

Mentorship

Education

BS/MS in CS/CE or related field

Tools

Docker

Kubernetes (K8s)

Cloud Endpoints

Helm

Prometheus

CI/CD pipelines

Temporal

Kafka

Job description

NVIDIA is seeking a senior engineer to design and build factory infrastructure and automation for NVIDIA Inference Microservices (NIMs). You will architect scalable pipelines, interfaces, data models, and observability to validate AI models across heterogeneous hardware and software environments.

You'll work with leaders to define strategies, mentor teams, and drive advancements in Docker, Kubernetes, and cloud tooling to accelerate NIM deployment from development to production.