Senior Backend Engineer, AI Evaluation Platform

Zillow Group Inc.; Northern (KY); Hybrid; USD 161,000 - 257,000; Full time

Job summary

Zillow Group Inc. in the United States seeks a collaborative backend engineer to design, build, and scale evaluation frameworks for agentic AI experiences that power Zillow’s AI assistants.

You will work with applied scientists, ML engineers, and product leaders to ensure reliability, performance, and measurable impact while staying current with LLM tooling and evolving evaluation techniques. This remote role offers a base salary competitive with the market and equity opportunities; US-based

Qualifications

• 4+ years of backend engineering with ML services
• Experience building platform infrastructure consumed by multiple teams
• Familiarity with ML evals, observability, and LLM tooling
• Ability to partner across disciplines to deliver reliable AI platforms

Responsibilities

• Design, build, and scale evaluation frameworks for Zillow’s agentic AI experiences.
• Build tracing, observability, and quality measurement systems for production AI agents.
• Create platform infrastructure enabling domain teams to build atop the Evals platform.
• Collaborate with scientists and engineers to integrate new evaluation capabilities.
• Evolve how Zillow measures AI quality, reliability, and impact.
• Stay current with agentic AI paradigms and practical tooling.

Skills

Backend engineering

APIs & reliability

ML evals & observability

LangChain / LangGraph

Developer tools for AI

Cross-team collaboration

LangSmith / Prompt tooling

Education

Advanced degree in CS/ML

Tools

Databricks MLflow

LangSmith

Braintrust

Promptfoo

Job description

Zillow Group Inc. in the United States seeks a collaborative backend engineer to design, build, and scale evaluation frameworks for agentic AI experiences that power Zillow’s AI assistants.

You will work with applied scientists, ML engineers, and product leaders to ensure reliability, performance, and measurable impact while staying current with LLM tooling and evolving evaluation techniques. This remote role offers a base salary competitive with the market and equity opportunities; US-based