Senior MLOps Engineer

Nace AI; Palo Alto (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Equity

Premium benefits

Job summary

Nace AI in Palo Alto is seeking a Senior MLOps Engineer to own the production-grade ML infrastructure, enabling training, deployment, and evaluation of specialized models in real time. You will design pipelines, scale multi-GPU clusters, and implement observability to support enterprise-grade AI for financial and compliance workflows.

Join a fast-paced, high-impact team building an OS-like platform for professional knowledge with strong cloud and on‑prem capabilities.

Qualifications

• 5+ years of experience in MLOps, ML infrastructure, or platform engineering, with substantial production ownership.
• Experience deploying and scaling LLM/inference infrastructure in production, including model serving frameworks such as TRT, vLLM, SGLang or TGI.
• Strong proficiency with Kubernetes, containerization (Docker), and infrastructure‑as‑code (Terraform or similar).
• Hands‑on experience with GPU cluster management and distributed training/serving environments.
• Proficient in Python with a strong track record of building substantial, maintainable systems.
• Experience with ML pipeline and orchestration tooling (e.g., Airflow, Kubeflow, Ray, MLflow, Weights & Biases).
• Solid foundation in computer science fundamentals and cloud architecture (AWS, GCP, or Azure).
• Self‑starter comfortable working in a fast‑paced, dynamic environment.

Responsibilities

• Design, build, and operate end‑to‑end ML infrastructure: training orchestration, experiment tracking, model registries, CI/CD for models, and automated evaluation pipelines.
• Own LLM/SLM serving infrastructure — scale low‑latency, high‑throughput inference using frameworks like vLLM, including batching, caching, and autoscaling strategies.
• Build and manage multi‑GPU training and inference clusters (scheduling, utilization, cost optimization) across cloud and on‑prem environments.
• Implement observability for models in production: latency, throughput, drift, regression, and quality monitoring with actionable alerting.
• Apply inference‑time optimizations — quantization (AWQ, GPTQ, FP8/GGUF), distillation support, KV‑cache management, and deployment tuning — in partnership with our ML and Research Engineers.
• Harden our stack for enterprise deployment: reproducibility, versioning, access controls, and audit‑ready traceability of model behavior.
• Set MLOps best practices and tooling standards as an early, senior member of the infrastructure team.

Skills

MLOps

Kubernetes

Docker

Terraform

Python

Airflow

Kubeflow

GPU Clusters

CI/CD

Cloud (AWS/GCP/Azure)

Education

BS in CS

MS in CS

Tools

vLLM

TGI

TRT

Job description

Palo Alto, CA | Full-Time | On-site

About Nace AI

Nace AI is an enterprise AI product and research company in Palo Alto (backed by General Catalyst, Walden Catalyst, and Intel). We build long-running AI agents powered by our own specialized SLMs — we started with financial audit and accounting workflows and are expanding from there. Real enterprise deployments, not demos.

Role Overview

As a Senior MLOps Engineer, you will own the infrastructure that takes Nace.AI's models from research to reliable, production-grade systems. Our infrastructure generates task‑specific Small Language Models (SLMs) in real time — which means our training, serving, and evaluation infrastructure isn't an afterthought; it is the product. You will design and operate the pipelines, orchestration, and serving layers that allow us to train, deploy, monitor, and continuously improve many specialized models at once, with the reliability that high‑stakes audit, compliance, and finance workflows demand. This role sits at the intersection of ML engineering, LLM inference infrastructure, and platform reliability, and requires both strong systems instincts and hands‑on execution.

Key Responsibilities

• Design, build, and operate end‑to‑end ML infrastructure: training orchestration, experiment tracking, model registries, CI/CD for models, and automated evaluation pipelines.
• Own LLM/SLM serving infrastructure — scale low‑latency, high‑throughput inference using frameworks like vLLM, including batching, caching, and autoscaling strategies.
• Build and manage multi‑GPU training and inference clusters (scheduling, utilization, cost optimization) across cloud and on‑prem environments.
• Implement observability for models in production: latency, throughput, drift, regression, and quality monitoring with actionable alerting.
• Apply inference‑time optimizations — quantization (AWQ, GPTQ, FP8/GGUF), distillation support, KV‑cache management, and deployment tuning — in partnership with our ML and Research Engineers.
• Harden our stack for enterprise deployment: reproducibility, versioning, access controls, and audit‑ready traceability of model behavior.
• Set MLOps best practices and tooling standards as an early, senior member of the infrastructure team.

Qualifications

• 5+ years of experience in MLOps, ML infrastructure, or platform engineering, with substantial production ownership.
• Proven experience deploying and scaling LLM, inference infrastructure in production, including model serving frameworks such as TRT, vLLM, SGLang or TGI.
• Strong proficiency with Kubernetes, containerization (Docker), and infrastructure‑as‑code (Terraform or similar).
• Hands‑on experience with GPU cluster management and distributed training/serving environments.
• Proficient in Python with a strong track record of building substantial, maintainable systems.
• Experience with ML pipeline and orchestration tooling (e.g., Airflow, Kubeflow, Ray, MLflow, Weights & Biases).
• Solid foundation in computer science fundamentals and cloud architecture (AWS, GCP, or Azure).
• BS degree in CS or related technical field.
• Self‑starter comfortable working in a fast‑paced, dynamic environment.

Preferred Qualifications

• MS in CS or related technical field.
• Experience operating multi‑node GPU training infrastructure.
• Hands‑on experience with quantization techniques (AWQ, GPTQ, FP8/GGUF) and other inference‑time optimizations.
• Familiarity with data processing stacks such as Spark and Airflow.
• Experience supporting fine‑tuning workflows for LLMs/VLMs (instruction tuning, RLHF/DPO pipelines).
• Experience in regulated or enterprise environments where reliability, security, and auditability are first‑class requirements.
• Contributor to open‑source ML infrastructure projects.

Why Nace AI

• Pedigree: Work with a team from top‑tier institutions and companies, backed by the best VCs in the world.
• Impact: You are joining early enough to shape the infrastructure foundations of a company aiming to be the "OS" for professional knowledge.
• Competitive Package: Silicon Valley‑standard salary, significant equity, and premium benefits.