Backend Engineer AI (Agent Systems)

USA Tech Recruit; San Francisco (CA); Remote; USD 180,000 - 240,000; Full time

Job summary

USA Tech Recruit is seeking a Backend Engineer AI for a San Francisco-based startup operating remotely in the US. You will build backend systems powering AI-powered features, design inference pipelines, and own production concerns with a focus on latency and throughput.

The role requires strong backend fundamentals, experience with high-throughput services, and familiarity with AI inference patterns. Proficiency in Python, NodeJs, Pytorch, and various LLMs is essential.

Qualifications

• Strong backend engineering fundamentals in production environments.
• Experience running high-throughput, low-latency services.
• Familiarity with AI inference patterns (LLMs, embeddings, multimodal).
• Comfortable debugging distributed systems under load.
• Bias toward shipping and learning from production behavior.

Responsibilities

• Build and operate backend systems that serve AI-powered features in production.
• Design inference pipelines, orchestration layers, and service boundaries around models.
• Own production concerns: monitoring, logging, alerting, and incident response.
• Optimize latency and throughput across inference, caching, batching, and streaming.

Skills

Backend fundamentals

Low latency

High throughput

AI inference awareness

Distributed systems debugging

Tools

Python

NodeJs

Pytorch

OpenAI / Anthropic / open-source LLMs

SQL

NoSQL

Kubernetes

Docker

Job description

Backend Engineer AI (Agent Systems) - San Francisco (Remote) - Start-up Equity

A long-standing client is expanding into the US and Europe. Starting with several hires in the US. This will involve working on a proactive AI assistant rather than a traditional chatbot, with highly reliable AI capable of completing multi-step tasks with minimal user prompting.

What You'll be Doing

• Build and operate backend systems that serve AI-powered features in production.
• Design inference pipelines, orchestration layers, and service boundaries around models.
• Own production concerns: monitoring, logging, alerting, and incident response.
• Optimize latency and throughput across inference, caching, batching, and streaming.

What You'll Bring

• Strong backend engineering fundamentals in production environments.
• Experience running high-throughput, low-latency services.
• Familiarity with AI inference patterns (LLMs, embeddings, multimodal).
• Comfortable debugging distributed systems under load.
• Bias toward shipping and learning from production behavior.
• Python, NodeJs, Pytorch, OpenAI / Anthropic / open-source LLMs, SQl & noSQL, Kubernetes, Docker

By applying to this role, you understand that we may collect your personal data and store and process it on your systems. For more information, please see our Privacy Notice (https://eu-recruit.com/about-us/privacy-notice/)