Senior ML Infra Engineer - Remote, Scalable Inference

AZX; Seattle (WA); Remote; USD 180,000 - 240,000; Full time

Benefits offered by this job

Health insurance

Equity

Flexible paid time off

Bonus eligibility

Fully remote culture with Seattle team

Job summary

AZX is seeking an ML Engineer to own the technical backbone of model serving at scale, shaping inference platforms, GPU scheduling, and autoscaling across cloud and customer-managed clusters for vLLM/SGLang models.

You’ll set standards, guide evaluation systems, and mentor engineers in a high-impact, architecture-focused role that balances cost, performance, and reliability for critical AI workloads.

Qualifications

• 3+ years of experience with ML infrastructure and inference serving at production scale.
• Strong background in evaluation and reliability engineering for ML/LLM systems.
• Solid Kubernetes experience, GPU scheduling constraints (node pools, autoscaling).
• Track record of technical leadership at staff/senior level.
• Research fluency (PhD/publications) is a plus but infra-first role.

Responsibilities

• Own architecture for inference serving and GPU scheduling — Kubernetes operators, autoscaling, and dynamic capacity across vLLM/SGLang.
• Design and calibrate eval systems for model, prompt, and agent changes, including golden datasets and CI integration.
• Advise on cost-aware model routing and cascading decisions, balancing latency, cost, and quality.
• Apply physics-informed ML and enterprise AI expertise to client and platform problems.
• Set technical standards for ML infrastructure and evaluation practice; mentor engineers.
• Partner with related teams to keep architecture coherent as the platform grows.

Skills

ML infrastructure

Kubernetes

GPU scheduling

Staff/senior leadership

Evaluation systems

Reliability engineering

Tools

vLLM

SGLang

TensorRT-LLM

PyTorch

CUDA

Job description

AZX is seeking an ML Engineer to own the technical backbone of model serving at scale, shaping inference platforms, GPU scheduling, and autoscaling across cloud and customer-managed clusters for vLLM/SGLang models.

You’ll set standards, guide evaluation systems, and mentor engineers in a high-impact, architecture-focused role that balances cost, performance, and reliability for critical AI workloads.