AI Inference Backend Engineer: Low-Latency & Scalable

Axe Recruiting; United States; Remote; USD 140,000 - 180,000; Full time

Job summary

Axe Recruiting is seeking an experienced backend engineer to own the inference and orchestration layer that sits between the models and every user interaction. You will build and run production systems that turn model capability into fast, observable APIs for mobile and desktop clients.

You’ll design inference pipelines, orchestration layers and service boundaries around models, and own production concerns such as monitoring, logging and incident response.

Qualifications

• Experience running high-throughput, low-latency services.
• Familiarity with AI inference patterns (LLMs, embeddings, multimodal).
• Comfort debugging distributed systems under load.
• Bias toward shipping and learning from production behavior.

Responsibilities

• Build and operate the backend systems that serve AI-powered features in production.
• Design the inference pipelines, orchestration layers and service boundaries around models.
• Own production concerns: monitoring, logging, alerting and incident response.
• Optimize latency and throughput across inference, caching, batching and streaming.
• Deliver stable, clear APIs that integrate cleanly with frontend and ML systems.
• Detect, diagnose and resolve production incidents quickly, keeping user impact low.
• Drive continuous gains in performance and reliability from real usage.

Skills

Backend engineering

Low latency

Distributed systems

Production monitoring

AI inference patterns

Tools

Python

Node.js

PyTorch

SQL

NoSQL

Kubernetes

Docker

Job description

Axe Recruiting is seeking an experienced backend engineer to own the inference and orchestration layer that sits between the models and every user interaction. You will build and run production systems that turn model capability into fast, observable APIs for mobile and desktop clients.

You’ll design inference pipelines, orchestration layers and service boundaries around models, and own production concerns such as monitoring, logging and incident response.