Senior Backend Engineer - AI Infra & LLM Focus

Underdog: Verified engineers. Interview-ready. Searching confidentially. And companies apply to you; United States; On-site; USD 150,000 - 230,000; Full time

Benefits offered by this job

Foundational Impact

Equity Ownership

Competitive Compensation

Cutting-Edge Tech Stack

A-Player Team

Job summary

Underdog is seeking a Software Engineer (LLM Focus) to design and scale core backend systems for an AI-driven platform. You will work with Python or Golang to build high-performance services, APIs, and data pipelines that support LLM inference at scale.

You will collaborate with ML researchers to productionize models, optimize for latency and cost, and own features from concept through deployment in a fast-paced startup setting.

Qualifications

• 3+ years of professional software development experience focused on backend systems.
• Production-level services built with Python or Golang.

Responsibilities

• Design, develop, and maintain high-performance distributed backend services in Golang or Python.
• Architect APIs and data pipelines for LLM-based inference systems.
• Integrate with LLM providers (OpenAI, Anthropic) and open-source models; optimize latency and cost.
• Contribute across the SDLC from concept to deployment and operations.
• Collaborate with ML researchers to productionize models and algorithms.

Skills

Python

Golang

Backend development

Cloud platforms

Docker

Kubernetes

Tools

Docker

Kubernetes

LLM providers

Job description

Underdog is seeking a Software Engineer (LLM Focus) to design and scale core backend systems for an AI-driven platform. You will work with Python or Golang to build high-performance services, APIs, and data pipelines that support LLM inference at scale.

You will collaborate with ML researchers to productionize models, optimize for latency and cost, and own features from concept through deployment in a fast-paced startup setting.