Staff Backend Engineer - AI Inference & Platform (Remote)

AZX; United States; À distance; USD 140 000 - 210 000; Plein temps

Avantages offerts par ce poste

Health insurance

Equity

Flexible PTO

Bonus eligibility

Training opportunities

Résumé du poste

AZX is seeking a Staff or Senior ML Engineer to own the technical backbone of model serving at scale. You’ll lead backend development for our LLM gateway, including routing, rate limiting, key management, and observability, while shaping sandboxing and security strategies.

You will collaborate across gateway, sandbox, and inference platform teams, build high-performance services in Go/Rust/Python, and scale the platform across cloud and customer-managed clusters.

Qualifications

• 4+ years of backend engineering experience with distributed systems, API design, and Go/Rust/async Python.
• Familiarity with LLM backend concerns (rate limiting, caching, token accounting) is a plus.
• Exposure to Kubernetes and containerization; interest in sandboxing or security is a plus.
• Comfort working across a range of platform concerns rather than a single specialty.
• Eagerness to grow into deeper specialization in gateway, sandbox, or inference infra over time.

Responsabilités

• Build and maintain backend services for the LLM gateway (routing, rate limiting, key management, observability).
• Contribute to sandboxing and isolation infra with security-focused engineers.
• Support Kubernetes-based platform services, including operators and autoscaling logic.
• Write high-performance backend code in Go, Rust, or async Python (FastAPI/Starlette).
• Instrument services with OpenTelemetry for observability of behavior, latency, and cost.
• Collaborate across gateway, sandbox, and inference platform teams.

Connaissances

Backend engineering

Distributed systems

API design

Go

Rust

Async Python

Kubernetes

Security / sandboxing

Outils

Envoy

gRPC

FastAPI/Starlette

OpenTelemetry

Docker

Description du poste

AZX is seeking a Staff or Senior ML Engineer to own the technical backbone of model serving at scale. You’ll lead backend development for our LLM gateway, including routing, rate limiting, key management, and observability, while shaping sandboxing and security strategies.

You will collaborate across gateway, sandbox, and inference platform teams, build high-performance services in Go/Rust/Python, and scale the platform across cloud and customer-managed clusters.