Senior AI Inference & Runtime Platform Engineer

Matcha; Northern (KY); Hybrid; USD 150,000 - 190,000; Full time

Benefits offered by this job

Health insurance with dependents

Bonus eligibility

Equity

Flexible paid time off

Fully remote culture

Job summary

AZX in the United States seeks an experienced platform engineer to own the layer where AI work happens: the machines, isolation boundary, and the models running on them. You’ll write Rust in the morning, a Kubernetes controller after lunch, and a FastAPI control-plane endpoint before you go home — building the fork engine, the guest agent, and the multi-substrate model lifecycle.

We’re looking for individuals who’ve built this stack, operated it at scale, or both.

Qualifications

• 5+ years of shipping production systems in a systems language.
• Experience with Rust, Go, C/C++, or Zig and async runtimes.
• Experience with Kubernetes operators, autoscaling, and node lifecycle.
• Experience deploying open-weight LLM serving infrastructure (vLLM/SGLang).
• Ability to threat-model isolation boundaries and secure sandbox environments.

Responsibilities

• Manage the serving tier for open-weight models, including deployment, config, and SLOs.
• Administer the Kubernetes layer for inference and sandbox workloads, including GPUs.
• Own the data plane: hosted vector stores and graph stores with backup and restore.
• Oversee sandbox runtime and host control plane: lifecycle, exec, teardown, metering, isolation threat model.
• Direct the FastAPI control-plane services, IaC tooling, and dashboards.
• Run the layer the backend services team builds and read their dashboards.
• Maintain OSS posture: releases, docs, and reproducible builds.

Skills

Rust

Kubernetes

Security threat modeling

LLM serving

Distributed systems

Open-source tooling

Tools

vLLM/SGLang

Kubebuilder/CRDs

Terraform/OpenTofu

Bicep

KEDA

Karpenter

OpenTelemetry

Job description

AZX in the United States seeks an experienced platform engineer to own the layer where AI work happens: the machines, isolation boundary, and the models running on them. You’ll write Rust in the morning, a Kubernetes controller after lunch, and a FastAPI control-plane endpoint before you go home — building the fork engine, the guest agent, and the multi-substrate model lifecycle.

We’re looking for individuals who’ve built this stack, operated it at scale, or both.