Senior AI Inference & Runtime Platform Engineer

Provn

Bellevue (WA)

Remote

USD 140,000 - 230,000

Full time

2 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health insurance
Equity
Flexible PTO
Bonus eligibility
Fully remote culture

Job summary

AZX seeks a senior systems engineer to own the AI execution layer, including the inference control plane, agent-sandboxing, and related Kubernetes orchestration. You will implement scalable, secure, low-latency runtimes for open-weight models across GPU clouds and customer-managed clusters.

You'll write Rust and Kubernetes controllers, collaborate on FastAPI endpoints, and design the fork engine and multi-substrate model lifecycle. Remote-friendly, Seattle cluster noted.

Qualifications

  • 5+ years shipping production systems in a systems language (Rust preferred).
  • Experience operating Kubernetes workloads with controllers, autoscaling, and node lifecycle.
  • Threat-model isolation boundaries (namespaces, cgroups, seccomp) with secure sandbox practices.
  • Hands-on deploying open-weight LLM serving infrastructure (vLLM/SGLang or similar).
  • Performance-focused distributed systems experience; measure before optimizing.
  • Proficiency in Rust (tokio), Python/FastAPI, Kubernetes operators, KEDA, Karpenter, GPU plugins, with willingness to learn others.
  • Familiarity with isolation tech (Firecracker, Kata, gVisor), secrets management and egress control, and stateful hosting (vector stores, graph stores).
  • Cloud/GPU substrates (AWS/Azure/GCP) with Terraform/OpenTofu and OpenTelemetry.

Responsibilities

  • Manage the serving tier for open-weight models: deployment, config, cold-start, per-model SLOs, upgrades.
  • Administer Kubernetes layer for inference and sandbox workloads: operators/CRDs, autoscaling, GPU scheduling, node lifecycle.
  • Own the stateful data plane end-to-end with tested restore procedures.
  • Oversee sandbox runtime and host-side control plane: lifecycle, exec, snapshot/fork, teardown, metering, audit.
  • Direct the FastAPI control-plane services, Terraform/OpenTofu, Bicep, dashboards.
  • Run the layer the backend services team builds on and debug their services while reviewing dashboards.
  • Manage open-source posture: OSS-compliant builds, docs, reproducible builds.

Skills

Rust
Go
C/C++
Zig

Education

Bachelor's Degree
Master's degree

Tools

Kubernetes
Terraform/OpenTofu
Bicep
OpenTelemetry
KEDA
Karpenter
Vault

Job description

AZX seeks a senior systems engineer to own the AI execution layer, including the inference control plane, agent-sandboxing, and related Kubernetes orchestration. You will implement scalable, secure, low-latency runtimes for open-weight models across GPU clouds and customer-managed clusters.

You'll write Rust and Kubernetes controllers, collaborate on FastAPI endpoints, and design the fork engine and multi-substrate model lifecycle. Remote-friendly, Seattle cluster noted.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Inference & Runtime Platform Engineer (Remote)
Senior AI Inference & Runtime Platform Engineer (Remote)

AZX • United States

Remote
USD 180,000 - 275,000
Equity
Health insurance
Flexible PTO
+1
Senior Software Engineer (AI Inference & Runtime Platform)
Senior Software Engineer (AI Inference & Runtime Platform)

AZX • Seattle (WA)

Hybrid
USD 150,000 - 190,000
Health insurance
Equity
Fully remote culture with Seattle team
+2
Senior Software Engineer (AI Inference & Runtime Platform)
Senior Software Engineer (AI Inference & Runtime Platform)

AZX • United States

Remote
USD 180,000 - 275,000
Equity
Health insurance
Flexible PTO
+1
Staff Backend Engineer - AI Inference & Platform (Remote)
Staff Backend Engineer - AI Inference & Platform (Remote)

AZX • United States

Remote
USD 140,000 - 210,000
Health insurance
Equity
Flexible PTO
+2
Senior Software Engineer (AI Inference & Runtime Platform)
Senior Software Engineer (AI Inference & Runtime Platform)

Provn • Bellevue (WA)

Remote
USD 140,000 - 230,000
Health insurance
Equity
Flexible PTO
+2
Senior AI Inference & Runtime Platform Engineer
Senior AI Inference & Runtime Platform Engineer

AZX • Seattle (WA)

Hybrid
USD 150,000 - 190,000
Health insurance
Equity
Fully remote culture with Seattle team
+2
Senior AI Infrastructure Engineer - Rust & High-Scale Systems
Senior AI Infrastructure Engineer - Rust & High-Scale Systems

Kraken International • United States

Remote
USD 180,000 - 280,000
Senior AI Inference Systems Engineer
Senior AI Inference Systems Engineer

Cloudflare, Inc. • San Francisco (CA)

Hybrid
USD 180,000 - 240,000
Senior AI Inference Reliability Engineer - GPU & Kubernetes
Senior AI Inference Reliability Engineer - GPU & Kubernetes

FDE Times • San Francisco (CA)

On-site
USD 160,000 - 280,000
Equity
Health insurance
Kubernetes Platform Engineer for AI Inference
Kubernetes Platform Engineer for AI Inference

present of AI • San Francisco (CA)

On-site
USD 180,000 - 240,000