Member of Technical Staff, Agents

Wwshemi; Palo Alto (CA); Hybrid; USD 200,000 - 300,000; Full time

Job summary

Wwshemi in Palo Alto is seeking a senior data science/ML engineer to build autonomous AI agents and robust evaluation systems for executive workflows. The role focuses on end-to-end agent capabilities, reliability, and measurement against real-world tasks.

You will design evaluation frameworks, manage production-grade pipelines in Python/SQL, and ship tools with Django, React, and TypeScript. Four days in-office per week; visa sponsorship not available.

Qualifications

• 3+ years in data science or ML focused on evaluation systems or product quality.
• Proficient in Python and SQL for large datasets and pipelines.
• Experience designing evaluation frameworks with proper metrics and datasets.
• Familiarity with statistics, experiment design, and end-to-end simulations.
• Experience shipping systems to users; RL knowledge is a plus.

Responsibilities

• Build agents that complete delegated work end to end without user oversight.
• Design reliable measures of agent success and evaluation baselines.
• Develop agent capabilities for tools executives use and test readiness.
• Investigate production failures, identify root causes, and implement fixes.
• Create scalable monitoring and failure-detection workflows.
• Improve agent recovery when tools or sites fail.
• Create repeatable workflows to measure performance changes.

Skills

Data science/ML experience

Python

SQL

Evaluation frameworks

Statistics

Experiment design

End-to-end simulations

Reliability improvements

Shipping systems

Tools

Django

React

TypeScript

Job description

This listing's locations Remote - Palo Alto

Employment Full-time

Workplace Remote

Compensation 200000–300000 per year

Join a small, senior team building autonomous AI agents that handle real executive workflows. You will develop agent capabilities and the evaluation systems needed to verify reliability before changes reach users.

What You'll Do

• Build agents that complete delegated work end to end without requiring users to check every step.
• Design reliable measures of agent success and maintain representative evaluations for real-world tasks.
• Develop new agent capabilities for tools executives already use, and verify their readiness before launch.
• Investigate production failures, identify root causes, and deliver fixes that prevent recurrence.
• Build scalable failure detection for known issues and unexpected patterns using evaluation and monitoring tools.
• Improve agent recovery when a tool or website fails.
• Create repeatable workflows so engineers can measure whether agent changes improve performance.

What We're Looking For

• At least 3 years in data science or machine learning roles building or delivering evaluation systems, metrics, or product quality measurement.
• Production-level Python and SQL skills for analysis, large datasets, and automated pipelines.
• Experience designing evaluation frameworks, including success criteria, representative datasets, appropriate metrics, and trustworthy benchmarks.
• Experience using statistics, experiment design, and realistic end-to-end simulations to evaluate agent changes.
• Background building or operating agents for real users and improving their reliability; experience with post-training, fine-tuning, or reinforcement learning is valuable.
• Experience owning and shipping systems to users, with demonstrated impact; familiarity with Django, React, and TypeScript.
• A track record of sustained contribution and growth in a role, founding a startup, or earning a promotion.

Base salary: $200,000 to $300,000 annually. Visa sponsorship is not available.

Location

On-site in Palo Alto, California, with four days per week in the office.