Applied AI Engineer, Evaluation

SproutsAI, Inc.

San Francisco (CA)

Remote

USD 120,000 - 180,000

Full time

12 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

SproutsAI, Inc. is seeking an ML or software engineer to build an evidence system that determines whether agents are ready for broader responsibility. The role blends evaluation design, experimentation, and production-quality systems to support safer, scalable automation.

You will design offline, shadow, and production evaluations, measure quality, policy, cost, latency, and outcomes, and turn failures into reproducible test cases, collaborating with product and engineering teams.

Qualifications

  • A track record of shipping reliable production software in a relevant domain.
  • Strong systems judgment and comfort reasoning about failure, security, and operability.
  • Ability to explain technical decisions clearly across engineering and product teams.

Responsibilities

  • Create offline, shadow, and production evaluations.
  • Measure quality, policy, cost, latency, and outcomes.
  • Turn failures into reproducible test cases.

Skills

Production software
Evaluation design
Experimentation
Reliability
Security awareness
Clear communication

Tools

Evaluation tooling

Job description

Build the evidence system that determines whether agents are ready for broader responsibility.

About the role

Build the evidence system that determines whether agents are ready for broader responsibility.

An ML or software engineer experienced in evaluation design, experimentation, and production quality systems.

What you’ll do
  • Create offline, shadow, and production evaluations
  • Measure quality, policy, cost, latency, and outcomes
  • Turn failures into reproducible test cases
What we’re looking for
  • A record of shipping reliable production software in a relevant domain
  • Strong systems judgment and comfort reasoning about failure, security, and operability
  • The ability to explain technical decisions clearly across engineering and product teams
About SproutsAI

SproutsAI is an early-stage enterprise software company building a controlled execution layer for agents. The product is designed to preserve identity and source permissions, pause consequential actions for review, and retain evidence from each run.

The early team will work directly with design partners and across traditional product boundaries. We value clear decisions, careful testing, and people who can stay close to both the technical system and the customer problem.

Location and employment

San Francisco or US remote. This role is expected to be full-time. Exact workplace expectations, compensation, benefits, work authorization, and employing entity will be published before applications open.

Candidate information

SproutsAI intends to provide an accessible and equitable hiring process. Formal openings will include applicant privacy, accommodation, and equal employment opportunity information appropriate to the hiring location.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote Evidence System AI Engineer
Remote Evidence System AI Engineer

SproutsAI, Inc. • San Francisco (CA)

Remote
USD 120,000 - 180,000
Applied AI Engineer, Agent Reasoning
Applied AI Engineer, Agent Reasoning

SproutsAI, Inc. • San Francisco (CA)

Hybrid
USD 140,000 - 210,000
Quality Engineer, AI Systems
Quality Engineer, AI Systems

SproutsAI, Inc. • San Francisco (CA)

Remote
USD 110,000 - 160,000
Staff Software Engineer, Agent Runtime
Staff Software Engineer, Agent Runtime

SproutsAI, Inc. • San Francisco (CA)

Remote
USD 140,000 - 210,000
VP, Engineering
VP, Engineering

SproutsAI, Inc. • San Francisco (CA)

Remote
USD 180,000 - 250,000
Enterprise Sales Engineer
Enterprise Sales Engineer

SproutsAI, Inc. • United States

Remote
USD 110,000 - 160,000
Frontend Engineer, Agent Studio
Frontend Engineer, Agent Studio

SproutsAI, Inc. • San Francisco (CA)

Remote
USD 120,000 - 180,000
Head of Product
Head of Product

SproutsAI, Inc. • San Francisco (CA)

Remote
USD 160,000 - 230,000
Engineering Program Manager
Engineering Program Manager

SproutsAI, Inc. • San Francisco (CA)

Remote
USD 120,000 - 180,000
Developer Experience Engineer
Developer Experience Engineer

SproutsAI, Inc. • San Francisco (CA)

Remote
USD 140,000 - 210,000