Senior Backend Engineer, AI Evaluation Platforms

Ellipsis Health; San Francisco (CA); Hybrid; USD 160,000 - 210,000; Full time

Benefits offered by this job

401(k) matching

Health insurance

Flexible PTO

Job summary

Ellipsis Health is seeking a Senior Backend Engineer to build the infrastructure and internal tooling for the AI Evaluation team. You will develop APIs, data integrations, and reporting tools to run, extend, and operate evaluations at scale.

You will productionalize frameworks for ASR, LLMs, TTS, knowledge bases, and guardrails, and create debugging tools for end-to-end inspections. A pragmatic, infrastructure-minded engineer with emphasis on developer experience will thrive in this role.

Qualifications

• 5+ years of professional software engineering experience focused on backend systems.
• Experience designing production-grade infrastructure with APIs, services, and data pipelines.
• Proficient in Python, TypeScript/JavaScript, or Java.
• Experience with test automation, evaluation pipelines, or CI/CD testing.

Responsibilities

• Build and maintain infrastructure and tooling for the AI evaluations platform used by internal teams.
• Develop and productionalize evaluation frameworks for ASR, LLMs, TTS, knowledge bases, and guardrails.
• Partner with ML, engineering and QA teams to translate evaluation requirements into robust infrastructure.
• Improve developer experience by making evaluation systems easy to extend and reliable.

Skills

Backend engineering

APIs

CI/CD

Observability

Debugging

Tools

Python

TypeScript

Java

CI/CD tooling

Job description

Ellipsis Health is seeking a Senior Backend Engineer to build the infrastructure and internal tooling for the AI Evaluation team. You will develop APIs, data integrations, and reporting tools to run, extend, and operate evaluations at scale.

You will productionalize frameworks for ASR, LLMs, TTS, knowledge bases, and guardrails, and create debugging tools for end-to-end inspections. A pragmatic, infrastructure-minded engineer with emphasis on developer experience will thrive in this role.