Senior Staff AI Engineer — Autonomous Agents & Evaluation

Wwshemi; Palo Alto (CA); Hybrid; USD 200,000 - 300,000; Full time

Job summary

Wwshemi in Palo Alto is hiring to build autonomous AI assistants for executive workflows. You will develop agent capabilities and evaluation systems to ensure real-world tasks are completed reliably, with measurable quality.

The role emphasizes end-to-end agent development, designing evaluation frameworks, and shipping production-grade ML systems. It offers a high annual salary and an on-site four days a week in Palo Alto with a flexible remote component.

Qualifications

• 3+ years of data science or ML experience building evaluation systems, metrics, or quality measurement for products.
• Production-quality Python and SQL skills, including experience analyzing large datasets and building automated pipelines.
• Experience designing evaluation frameworks, datasets, measurable criteria, and appropriate metrics.
• A track record delivering production AI, machine learning, agent, or agent-evaluation systems for real users.
• Experience improving agent reliability, owning systems end to end, and measuring results in production.

Responsibilities

• Build agents that complete delegated work end to end for real users.
• Define reliable measures of success for agent tasks and maintain representative evaluations.
• Develop new agent capabilities and validate them before they reach users.
• Investigate production failures and deliver fixes that address underlying causes.
• Build systems to detect known failure patterns and emerging issues at scale.
• Improve agent recovery when tools or websites fail.
• Create repeatable methods for engineers to measure the impact of agent changes.

Skills

Data science

Machine learning

Evaluation metrics

Python

SQL

Experiment design

Statistics

Production AI

End-to-end systems

Shipping in production

Tools

Django

React

TypeScript

Braintrust

Raindrop

Job description

Wwshemi in Palo Alto is hiring to build autonomous AI assistants for executive workflows. You will develop agent capabilities and evaluation systems to ensure real-world tasks are completed reliably, with measurable quality.

The role emphasizes end-to-end agent development, designing evaluation frameworks, and shipping production-grade ML systems. It offers a high annual salary and an on-site four days a week in Palo Alto with a flexible remote component.