Staff Engineer - AI Evaluation & Metrics Platform

Kindredventures; San Francisco (CA); On-site; USD 140,000 - 200,000; Full time

Job summary

Kindredventures is seeking research engineers to build a central evaluation framework and scalable evaluation pipelines for models and datasets. You will design benchmarks, implement baselines, and create dashboards that translate results into actionable insights for the team.

The role emphasizes measurable rigor, collaboration with research and domain teams, and end-to-end ownership of evaluation tooling from data collection to automated metrics.

Qualifications

• Strong software engineering skills and experience building data or evaluation pipelines at scale (e.g., large data processing, experiment infrastructure).
• Experience turning research or model outputs into metrics, benchmarks, and visualizations that teams rely on.
• Solid grasp of probability and statistics, with the judgment to design evaluations that measure what they claim to.

Responsibilities

• Design and build a central, reusable evaluation framework used across models and teams.
• Implement evaluation pipelines, benchmark suites, and baselines to make model quality measurable and comparable.
• Build visualization and dashboard tools that turn raw results into actionable insights for the team.
• Establish sound statistical methodology for evaluation to distinguish real improvements from noise.
• Partner with research and domain teams to translate “good” into standardized, automated metrics.

Skills

Software engineering

Data pipelines

Metrics & visualization

Statistics

Full-stack development

Delivery ownership

Job description

Kindredventures is seeking research engineers to build a central evaluation framework and scalable evaluation pipelines for models and datasets. You will design benchmarks, implement baselines, and create dashboards that translate results into actionable insights for the team.

The role emphasizes measurable rigor, collaboration with research and domain teams, and end-to-end ownership of evaluation tooling from data collection to automated metrics.