AI Backend Engineer: Inference & Orchestration

A1; Palo Alto (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

A1 is hiring a Backend Engineer, AI, in Palo Alto to own the inference and orchestration layer powering AI interactions in our product. You will build and operate production systems that turn model capabilities into fast, stable APIs accessed by mobile and desktop clients.

You will design multi-step inference pipelines, manage tool calls and retries, and optimize routing, caching, batching, streaming, and state management to meet latency and throughput goals.

Qualifications

• Experience building backend systems for AI-powered features.
• Design and operate inference pipelines and orchestration layers.
• Strong emphasis on latency, reliability, and observability.

Responsibilities

• Build and operate backend systems that serve AI-powered features in production.
• Design inference pipelines and orchestration layers that handle multi-step workflows, tool calls, and retries.
• Manage the full lifecycle of AI requests: routing, caching, batching, streaming, and state management.
• Optimize latency, throughput, and cost across model inference and downstream systems.
• Design systems that remain reliable despite non-deterministic model behavior and external dependencies.
• Implement observability for AI systems, including logging, tracing, and debugging of model outputs and failures.
• Collaborate with ML and product teams to translate model capabilities into stable, production-grade APIs.

Skills

Python

NodeJs

Pytorch

OpenAI / Anthropic / open-source LLMs

SQL & NoSQL

Docker

Job description

A1 is hiring a Backend Engineer, AI, in Palo Alto to own the inference and orchestration layer powering AI interactions in our product. You will build and operate production systems that turn model capabilities into fast, stable APIs accessed by mobile and desktop clients.

You will design multi-step inference pipelines, manage tool calls and retries, and optimize routing, caching, batching, streaming, and state management to meet latency and throughput goals.