Senior Backend Engineer, AI Platform

Jobgether; United States; Remote; USD 140,000 - 210,000; Full time

Benefits offered by this job

Remote work US

Technical ownership

High-scale engineering

Modern stack

Professional growth

Collaborative environment

Job summary

Jobgether is seeking an experienced Backend Engineer to architect and scale our AI platform in production for millions of users. You will own backend services, design APIs, and lead integration with multiple LLM providers, including retrieval-augmented generation and token streaming.

Ideal candidates have 5+ years of backend experience with Go or similar statically typed languages, plus hands-on work with vector databases, model serving, and observability.

Qualifications

• Backend engineering experience with a statically typed language (Go/Java/Kotlin/C#).
• Production LLM features in production (retrieval-augmented generation, streaming).
• LLM serving with multi-provider routing, failover, token streaming, and usage metering.
• Experience building vector search and embedding pipelines with source-backed answers.
• Understanding AI failure modes; implement instrumentation and safeguards.
• Ability to own backend services lifecycle from schema to on-call support.
• Location in United States and work authorization.

Responsibilities

• Design, build, and scale backend infrastructure for AI features across a high-traffic platform.
• Develop and maintain LLM integration and serving services with model routing and streaming.
• Build retrieval-augmented generation systems with vector search and citations.
• Own API design and backend service architecture across deployment and operations.
• Monitor live services, troubleshoot incidents, improve resilience.
• Collaborate with platform architects to align engineering practices and AI capabilities.

Skills

Backend engineering

Go

Java

Python

LLM integration

API design

Observability

Cloud/DevOps

Tools

LiteLLM

vLLM

TGI

Qdrant

pgvector

LoRA

Ruby on Rails

React/TypeScript

Job description

Accountabilities

- AI Platform Development: Design, build, and scale the backend infrastructure required to deliver production-grade AI features across a platform serving millions of users.

- LLM Integration and Serving: Develop and maintain services that integrate multiple LLM providers, supporting model routing, failover, token streaming, and cost and usage tracking.

- Retrieval-Augmented Generation: Build and optimize retrieval systems, including vector search, embedding pipelines, context assembly, and citation-supported responses.

- Backend Architecture and APIs: Design robust APIs and backend services, taking end-to-end ownership of service architecture, schemas, implementation, deployment, and operational support.

- Reliability and Observability: Anticipate and address production risks such as hallucinations, retrieval failures, provider outages, latency issues, and unexpected costs through appropriate monitoring and instrumentation.

- AI Evaluation and Quality: Develop practical, repeatable evaluation methods to assess response relevance, safety, source quality, and overall AI performance.

- Production Operations: Monitor live services, troubleshoot incidents, improve system resilience, and support ongoing performance and reliability improvements.

- Cross-Functional Collaboration: Work closely with platform architects and fellow developers to shape technical decisions, establish engineering best practices, and integrate AI capabilities with existing application services.

Requirements

- Backend Engineering Experience: At least five years of backend engineering experience using a statically typed language such as Go, Java, Kotlin, or C#.

- Production LLM Expertise: Demonstrated experience shipping and maintaining LLM-powered features in production, including retrieval-augmented generation, streaming responses, and tool use.

- LLM Serving Infrastructure: Hands-on experience with model-serving systems, including multi-provider routing, failover strategies, token streaming, and cost and usage metering.

- Retrieval Systems: Experience developing vector search solutions, embedding pipelines, context assembly workflows, and systems that provide source-backed answers.

- Reliability and Observability: Strong understanding of AI failure modes and the ability to implement instrumentation, monitoring, and safeguards to detect and address production issues.

- AI Evaluation: Ability to assess AI output quality using straightforward, repeatable tests covering relevance, safety, and source quality rather than relying exclusively on academic benchmarks.

- API Design and Ownership: Strong API design skills and the ability to own backend services throughout their lifecycle, from data schema and implementation to deployment and on-call support.

- Location and Work Authorization: Must be based in the United States and authorized to work in the country.

- Preferred Programming Skills: Go experience is strongly preferred. Familiarity with Python is also beneficial.

- Preferred AI Infrastructure Experience: Experience with LLM gateways or serving frameworks such as LiteLLM, vLLM, TGI, or comparable technologies.

- Preferred Data and Model Experience: Familiarity with vector databases such as Qdrant or pgvector, embedding pipelines, and fine-tuning open-weight models using LoRA or full fine-tuning methods.

- Additional Advantages: Experience with content moderation or AI safety tooling, as well as familiarity with Ruby on Rails or React/TypeScript for application integration.

Benefits

- Remote Work: Full-time remote work for candidates based in the United States, subject to the role's location requirements.

- Technical Ownership: The opportunity to build greenfield AI infrastructure and influence architectural decisions, development practices, and platform capabilities.

- High-Scale Engineering: Work on AI-powered features designed for a platform serving millions of users.

- Modern Technology Stack: Exposure to Go, Ruby on Rails, Python, React/TypeScript, PostgreSQL, Redis, and RabbitMQ, with services deployed across multiple data centers.

- Professional Growth: Opportunity to deepen expertise in LLM infrastructure, retrieval systems, AI evaluation, distributed services, and production engineering.

- Collaborative Environment: Direct collaboration with platform architects and developers in a fast-paced engineering organization.

- Salary and Additional Benefits: Compensation, healthcare coverage, retirement benefits, paid time off, and other employee perks are not specified in the available job description and should be confirmed during the hiring process.

#LI-CL1