Senior Staff Engineer - Autonomous AI Agents (Production)

Engg; Palo Alto (CA); On-site; USD 200,000 - 300,000; Full time

Job summary

Engg is seeking an engineering leader to build and improve autonomous AI agents that handle executive tasks end to end, from scheduling to relationship management. You will own the full loop: shipping agents, measuring quality, and making improvement a repeatable science for real users in production.

The role requires 5+ years in data science or ML with a focus on evaluation systems, strong Python/SQL, and familiarity with Django/React/TypeScript for cross-domain work.

Qualifications

• 5+ years in data science, machine learning, or analytics with focus on evaluation systems for production AI.
• Experience designing evaluation methodologies: criteria, dataset construction, metrics, signal vs noise.
• Production-quality Python and SQL; familiarity with Django, React and TypeScript for cross-domain work.
• Strong statistical and experimental design skills: sampling, variance, bias, significance testing for non-deterministic systems.
• Experience with LLM-as-a-judge systems, model-based graders, and grader calibration.

Responsibilities

• Build agents that complete delegated executive tasks end to end, without human follow-up.
• Design evaluation frameworks and metrics to measure agent quality.
• Expand tasks agents can handle in tools executives use, with readiness evidence.
• Turn production failures into proven, permanent fixes quickly.
• Own detection at scale using observability and eval tooling to surface failure patterns.
• Build self-recovery so agents stay on task when a tool or external service fails.
• Make agent improvement measurable through end-to-end simulations.

Skills

Data science

Evaluation systems

Experiment design

Statistical analysis

LLM evaluation

Debug prompts

Agentic systems

End-to-end shipping

Education

CS or engineering degree

Tools

Python

SQL

Django

React

TypeScript

Job description

Engg is seeking an engineering leader to build and improve autonomous AI agents that handle executive tasks end to end, from scheduling to relationship management. You will own the full loop: shipping agents, measuring quality, and making improvement a repeatable science for real users in production.

The role requires 5+ years in data science or ML with a focus on evaluation systems, strong Python/SQL, and familiarity with Django/React/TypeScript for cross-domain work.