Remote Senior Backend Engineer - AI Evaluation

Zillow Group Inc.; United States; Remote; USD 153,000 - 257,000; Full time

Benefits offered by this job

Equity awards

Job summary

Zillow Group is seeking a collaborative, product-minded backend engineer to design, build, and scale evaluation frameworks for agentic AI experiences. You will work with applied scientists, ML engineers, and product leaders to deliver reliable, scalable AI evaluation capabilities across Zillow’s production systems.

You will partner with teams to integrate new evaluation capabilities, optimize performance and cost, and stay ahead of evolving LLM tooling and agentic paradigms.

Qualifications

• 4+ years backend engineering experience shipping production ML services.
• Experience designing platform infrastructure and developer-facing services.
• Familiarity with ML evals, observability frameworks, and LLM tooling.

Responsibilities

• Design, build, and scale evaluation frameworks for Zillow’s agentic AI experiences.
• Build tracing, observability, and quality measurement systems for production AI agents.
• Create platform infrastructure enabling domain teams to build on the Evals platform.
• Collaborate with scientists/engineers to integrate new evaluation capabilities into production systems.
• Advance how Zillow measures AI quality, reliability, and impact.
• Stay current with agentic AI paradigms and translate them into platform innovations.
• Support scaling, performance, and cost management for the evaluation layer.

Skills

Backend engineering

APIs & reliability

ML evals & observability

Build vs. buy decisions

跨-disciplinary collaboration

LLM tooling

Education

Advanced degree or equivalent

Tools

Databricks MLflow

LangChain

LangGraph

MCP

LangSmith

Job description

Zillow Group is seeking a collaborative, product-minded backend engineer to design, build, and scale evaluation frameworks for agentic AI experiences. You will work with applied scientists, ML engineers, and product leaders to deliver reliable, scalable AI evaluation capabilities across Zillow’s production systems.

You will partner with teams to integrate new evaluation capabilities, optimize performance and cost, and stay ahead of evolving LLM tooling and agentic paradigms.