Backend Engineer, Evals — Build AI Eval Pipelines

OpenAI; Los Angeles (CA); On-site; USD 230,000 - 385,000; Full time

Job summary

A technology company in Los Angeles is seeking a Backend Software Engineer to design and build an evals infrastructure for automation. Candidates should have over 4 years of backend experience, with proficiency in Python, FastAPI, and Postgres. This role focuses on creating reliable eval pipelines and integrating AI models into workflows. The position offers competitive compensation ranging from $230K to $385K plus equity options.

Qualifications

• 4+ years of backend engineering experience at product-driven companies.
• Proficiency in backend technologies including Python, FastAPI, and Postgres.
• Experience designing and scaling distributed systems and APIs.
• Experience in building AI applications and evaluating ML models at scale.

Responsibilities

• Design eval pipelines that are reliable, reproducible, and extendable.
• Build infrastructure for continuous evaluation monitoring frameworks.
• Design and maintain backend services and APIs for automation.
• Integrate and structure data across internal platforms.

Skills

Backend engineering experience

Proficiency in Python

Experience with FastAPI

Experience with Postgres

Experience with distributed systems

Experience with AI applications

Familiarity with evaluation methods for LLMs

Pragmatic mindset

Job description

A technology company in Los Angeles is seeking a Backend Software Engineer to design and build an evals infrastructure for automation. Candidates should have over 4 years of backend experience, with proficiency in Python, FastAPI, and Postgres. This role focuses on creating reliable eval pipelines and integrating AI models into workflows. The position offers competitive compensation ranging from $230K to $385K plus equity options.