Member of Technical Staff, Agents

Clera; United States; Remote; USD 200,000 - 300,000; Full time

Job summary

Royper Royco LLC is seeking a senior data science/ML engineer to build autonomous AI agents for executive workflows. You will design evaluation systems, develop agent capabilities, and verify readiness before deployment. The role emphasizes end-to-end agent delivery, reliable success metrics, and scalable monitoring in a fast-paced small-team setting.

Base salary ranges from 200,000 to 300,000 USD annually. On-site in Palo Alto, California, with four days per week in the office.

Qualifications

• 3+ years in data science or ML roles building evaluation systems.
• Production-level Python and SQL for large datasets and automated pipelines.
• Experience designing evaluation frameworks, including success criteria, datasets, metrics, and benchmarks.
• Experience using statistics, experiment design, and realistic end-to-end simulations to evaluate agent changes.
• Background building or operating agents for real users and improving reliability; post-training, fine-tuning, or reinforcement learning is valuable.
• Familiarity with Django, React, and TypeScript, shipping systems to users.
• Track record of growth in role, founding a startup, or earning a promotion.

Responsibilities

• Build agents that complete delegated work end to end without requiring users to check every step.
• Design reliable measures of agent success and maintain representative evaluations for real-world tasks.
• Develop new agent capabilities for tools executives already use, and verify readiness before launch.
• Investigate production failures, identify root causes, and deliver fixes to prevent recurrence.
• Build scalable failure detection for known issues and patterns using evaluation and monitoring tools.
• Improve agent recovery when a tool or website fails.
• Create repeatable workflows to measure whether agent changes improve performance.

Skills

Python

SQL

Data analysis

Experiment design

Statistics

Reinforcement learning

Django

React

TypeScript

Evaluation systems

Tools

Pandas

NumPy

Job description

Join a small, senior team building autonomous AI agents that handle real executive workflows. You will develop agent capabilities and the evaluation systems needed to verify reliability before changes reach users.

What You'll Do

• Build agents that complete delegated work end to end without requiring users to check every step.
• Design reliable measures of agent success and maintain representative evaluations for real-world tasks.
• Develop new agent capabilities for tools executives already use, and verify their readiness before launch.
• Investigate production failures, identify root causes, and deliver fixes that prevent recurrence.
• Build scalable failure detection for known issues and unexpected patterns using evaluation and monitoring tools.
• Improve agent recovery when a tool or website fails.
• Create repeatable workflows so engineers can measure whether agent changes improve performance.

What We’re Looking For

• At least 3 years in data science or machine learning roles building or delivering evaluation systems, metrics, or product quality measurement.
• Production-level Python and SQL skills for analysis, large datasets, and automated pipelines.
• Experience designing evaluation frameworks, including success criteria, representative datasets, appropriate metrics, and trustworthy benchmarks.
• Experience using statistics, experiment design, and realistic end-to-end simulations to evaluate agent changes.
• Background building or operating agents for real users and improving their reliability; experience with post-training, fine-tuning, or reinforcement learning is valuable.
• Experience owning and shipping systems to users, with demonstrated impact; familiarity with Django, React, and TypeScript.
• A track record of sustained contribution and growth in a role, founding a startup, or earning a promotion.

Base salary: $200,000 to $300,000 annually. Visa sponsorship is not available.

Location

On-site in Palo Alto, California, with four days per week in the office.