Senior Backend Engineer, AI Platform & LLM Infra (Remote)

Provn

Bellevue (WA)

Remote

USD 140,000 - 230,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity
Health insurance
Flexible PTO
Fully remote
Seattle cluster

Job summary

AZX is seeking a Backend Engineer to build the LLM infrastructure that powers its products and client solutions. The role focuses on a governed gateway over hosted and self-hosted models, with concrete feedback from clients and engineers.

You’ll own routing, metering, budgeting, and guardrails across the platform. Responsibilities include retrieval services, eval harness development, and evolving the async framework for durable execution of agent runs.

Qualifications

  • 5+ years of production async Python experience.
  • Deep PostgreSQL MVCC knowledge and distributed systems experience.
  • Experience shipping API surfaces with versioning, idempotency, and documentation.
  • Hands-on with OpenTelemetry and OpenAI/Anthropic provider APIs.

Responsibilities

  • Own the LLM gateway: routing, metering, budgets, guardrails, and adapters.
  • Manage the retrieval and knowledge service: hybrid search and reranking.
  • Build the eval harness with evidence, confidence intervals, and per-segment breakdowns.
  • Own the async framework and its public API, rate-limit algorithms, and admin tooling.
  • Design authorization: token hierarchies, capability grammars, per-tenant isolation.
  • Define done on all shipped work: evals, traces, cost accounting, and dashboards.

Skills

async Python
PostgreSQL
API design
LLM gateway
Cost accounting
Distributed systems
OpenTelemetry
Redis/Dragonfly

Tools

FastAPI/Starlette
Pydantic v2
asyncpg/SQLAlchemy
Redis/Dragonfly
Lua
pgvector

Job description

AZX is seeking a Backend Engineer to build the LLM infrastructure that powers its products and client solutions. The role focuses on a governed gateway over hosted and self-hosted models, with concrete feedback from clients and engineers.

You’ll own routing, metering, budgeting, and guardrails across the platform. Responsibilities include retrieval services, eval harness development, and evolving the async framework for durable execution of agent runs.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff Backend Engineer - AI Inference & Platform (Remote)
Staff Backend Engineer - AI Inference & Platform (Remote)

AZX • United States

Remote
USD 140,000 - 210,000
Health insurance
Equity
Flexible PTO
+2
Senior Backend Engineer — Remote, Equity & Impact in AI
Senior Backend Engineer — Remote, Equity & Impact in AI

AZX • United States

Remote
USD 140,000 - 230,000
Health insurance
Equity
Flexible paid time off
+1
Backend Engineer – AI Platform & LLM Orchestration
Backend Engineer – AI Platform & LLM Orchestration

Axiamatic • Menlo Park (CA)

Hybrid
USD 140,000 - 200,000
Backend Engineer, LLM Agent Platform
Backend Engineer, LLM Agent Platform

Anduril Industries • Washington

On-site
USD 220,000 - 292,000
Equity grants
Competitive benefits package
Performance incentives
Senior Backend Engineer - AI/LLM & Microservices
Senior Backend Engineer - AI/LLM & Microservices

Coinbase, Inc. • New York (NY)

On-site
USD 150,000 - 210,000
Medical Insurance
Dental Insurance
Vision Insurance
+3
Remote AI/ML Engineer: LLM Agents & Backend
Remote AI/ML Engineer: LLM Agents & Backend

24-MAG • United States

Remote
USD 220,000 - 300,000
Fully remote
Senior Backend Engineer, LLM Agent Platform
Senior Backend Engineer, LLM Agent Platform

Anduril Industries, Inc. • Washington

On-site
USD 220,000 - 292,000
Equity grants
Health benefits
Senior Backend Engineer - AI Platform (Remote)
Senior Backend Engineer - AI Platform (Remote)

TMTG group • Sarasota (FL)

Hybrid
USD 180,000 - 240,000
Senior Backend Engineer, AI Platform (Remote)
Senior Backend Engineer, AI Platform (Remote)

Affirm, Inc. • Chicago (IL)

Remote
USD 173,000 - 233,000
Health coverage for dependents
Monthly tech stipend
Flexible time off
+1
Senior AI Engineer - LLM Platform in Production (Remote)
Senior AI Engineer - LLM Platform in Production (Remote)

Stable10 • Boston (MA)

Remote
USD 150,000 - 190,000
Remote-friendly
Startup environment