Backend Engineer, ML/LLM Evals Infra

OpenAI, Inc.; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

OpenAI in San Francisco is seeking a Backend Software Engineer focused on ML/LLM evals infrastructure. You will design and build scalable backend services, APIs, and data pipelines that support evaluation at scale, collaborating with Data Science and Research partners.

The role emphasizes reliability, repeatability, and cross-functional impact as you own the full lifecycle of new backend capabilities, with Python, FastAPI, and Postgres as core tech.

Qualifications

• 4+ years of backend engineering experience at product-driven companies.
• Proficiency in Python, FastAPI, and Postgres.
• Experience designing and scaling distributed systems, APIs, or data processing pipelines.
• Experience building AI agents or applications, including evals and prompting for performance improvements.
• Familiarity with evaluation methods for LLMs and multi-agent workflows.

Responsibilities

• Design eval pipelines that are reliable, reproducible, and extendable.
• Build infrastructure for continuous eval monitoring (regression/drift) and robust golden datasets.
• Design, build, and maintain backend services and APIs to support intelligent automation.
• Integrate and structure data across internal platforms for AI workflows.
• Collaborate with data, research, and engineering teams to integrate OpenAI models into workflows.
• Own the full development lifecycle of new backend systems and platform capabilities.

Skills

Backend engineering

Python

Distributed systems

API design

ML evals

Tools

PostgreSQL

FastAPI

Job description

OpenAI in San Francisco is seeking a Backend Software Engineer focused on ML/LLM evals infrastructure. You will design and build scalable backend services, APIs, and data pipelines that support evaluation at scale, collaborating with Data Science and Research partners.

The role emphasizes reliability, repeatability, and cross-functional impact as you own the full lifecycle of new backend capabilities, with Python, FastAPI, and Postgres as core tech.