Senior Software Development Engineer, Agentic Memory & Evals Team

Zillow Group; United States; Remote; USD 180,000 - 260,000; Full time

Benefits offered by this job

Equity awards

Job summary

Zillow Group's Agentic AI team is seeking a collaborative backend engineer with strong agentic AI fundamentals to design, build, and scale production ML services that power intelligent real estate experiences.

You will own evaluation frameworks, observability, and platform infrastructure, partnering with applied scientists and engineers to ship reliable systems, stay current with LLM tooling, and mentor teammates.

Qualifications

• 8+ years of backend engineering experience designing, shipping, and operating scalable production ML services.
• Bachelor's degree in Computer Science, Computer Engineering, or a related field, or equivalent experience.
• Experience building platform infrastructure/backend services consumed by multiple teams with attention to APIs, reliability and internal customers.
• Experience with ML evaluation and observability frameworks (Databricks MLflow, or evolving LLM frameworks like LangSmith/Braintrust/Promptfoo).
• Experience evaluating third-party AI platforms and contributing to build-vs-buy decisions for platform infra.
• Fluency crossing applied science, ML, product, design, and engineering disciplines with precision.

Responsibilities

• Design, build, and scale evaluation frameworks for Zillow's agentic AI experiences.
• Build tracing, observability, and quality measurement systems for production AI agents.
• Create platform infrastructure enabling domain teams to build on top of the Evals platform without bottlenecks.
• Partner with applied scientists and ML engineers to integrate new AI evaluation capabilities into production systems.
• Help evolve how Zillow measures AI quality, reliability, trustworthiness, and product impact.
• Stay current with agentic AI paradigms, evaluation techniques, and large language model tooling; translate them into practical platform innovations.
• Support scaling, reliability, performance optimization, incident response, and cost management for the evaluation layer.
• Apply first-principles thinking to ambiguous problems and iterate quickly on novel solutions.
• Mentor and support the growth of fellow engineers on the team.

Skills

Backend engineering

APIs & reliability

Cross-functional collaboration

Owner mindset

Education

Bachelor's degree in CS/CE or related

Tools

Databricks MLflow

LangSmith

Braintrust

Promptfoo

LangChain

LangGraph

MCP

Job description

About the team

The Agentic AI team at Zillow is transforming the real estate industry by helping millions of people use AI assistants to find their next home. We are building always-on AI experiences that combine personalized user insights with Zillow's deep real estate knowledge. The Memory & Evals team builds foundational capabilities that make these experiences more useful, trustworthy, and scalable. This includes user memory and context systems, evaluation frameworks, tracing, observability tools, and feedback loops that help teams understand agent behavior, identify failure modes, and continuously improve. As part of this lean, customer-focused team, you will partner with applied scientists, software engineers, machine learning engineers, and product leaders to evolve Zillow's next-generation AI platform.

About the role

We are seeking a collaborative, product-minded backend engineer with strong Agentic AI fundamentals and a passion for building reliable, scalable systems that power AI applications. In this role, you will:

• Design, build, and scale evaluation frameworks for Zillow's agentic AI experiences.
• Build tracing, observability, and quality measurement systems for production AI agents.
• Create platform infrastructure that enables domain teams across Zillow to build on top of the Evals platform without bottlenecks.
• Partner with applied scientists and machine learning engineers to integrate new AI evaluation capabilities into production systems.
• Help evolve how Zillow measures AI quality, reliability, trustworthiness, and product impact.
• Stay current with emerging agentic AI paradigms, evaluation techniques, and large language model (LLM) tooling, and translate them into practical platform innovation.
• Support scaling, reliability, performance optimization, incident response, and cost management for the evaluation layer.
• Apply first-principles thinking to ambiguous problems and iterate quickly on novel solutions.
• Mentor and support the growth of fellow engineers on the team.

This role has been categorized as a teleworker position. Teleworkers do not have a permanent corporate office workplace and instead work from a physical location of their choice, which must be identified to the Company. Employees may live anywhere in Mexico, with availability to travel to Mexico City, as we recommend attendance at occasional office events.

In addition to a competitive base salary and benefits, this position is also eligible for equity awards based on factors such as experience, performance and location.

Who you are

• 8+ years of backend engineering experience, with a track record of designing, shipping, and operating scalable production machine learning (ML) services.
• Bachelor's degree in Computer Science, Computer Engineering, or a related field, or equivalent experience.
• Experience building platform infrastructure/backend services or developer-facing services consumed by multiple teams; you've thought carefully about APIs, user experience, reliability, and what it means to have internal customers.
• Experience with ML evaluation and observability frameworks (Databricks MLflow, or evolving LLM frameworks preferred: LangSmith, Braintrust, Promptfoo, or similar).
• Experience evaluating third-party AI platforms and contributing to principled build vs. buy decisions for platform infrastructure.
• Fluency working across applied science, ML, product, design, and engineering; you translate between disciplines without losing precision.
• Comfort with LLMs, agentic systems, and evaluation frameworks; familiarity with orchestration tools (LangChain, LangGraph, Model Context Protocol (MCP), or similar).
• A habit of reaching for AI-assisted development tools in your own workflow; you don't just build for AI, you build with it.
• Strong ownership, high agency and a growth mindset; you're energized by ambiguity, not slowed by it.

Get to know us

At Zillow, we’re reimagining how people move—through the real estate market and through their careers. As the most-visited real estate platform in the U.S., we help people navigate buying, selling, financing and renting with greater ease and confidence. Whether you're working in tech, sales, operations, or design, you’ll be part of a company that's reshaping an industry and helping more people make home a reality.

How we work is a key part of that transformation. Through Cloud HQ, our strategic approach to distributed work, most employees have the flexibility to work from wherever they’re most productive—enabling us to move fast, stay connected and deliver on our people promise.

Zillow is honored to be recognized among the best workplaces in the U.S.. Zillow was named one of FORTUNE 100 Best Companies to Work For® in 2026, and included on TIME’s America’s Best Companies 2026 list, reflecting our commitment to creating an innovative, inclusive, and engaging culture where employees are empowered to grow.

No matter where you sit in the organization, your work will help drive innovation, support our customers, and move the industry—and your career—forward, together.

Zillow Group is committed to fostering an inclusive, innovative environment and to providing equal employment opportunities. We comply with applicable laws that prohibit discrimination in employment and recruitment.

If you have a disability or special need that requires reasonable accommodation during the recruitment process, please contact your recruiter directly.

Qualified applicants will be considered for employment based on their qualifications and ability to perform the functions of the position, without discrimination and in accordance with applicable law.

Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees, supervisors, and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service; and follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.