Staff Engineer - Synthetic Data & Agentic Evaluation

Kindredventures

Palo Alto (CA)

On-site

USD 120,000 - 180,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Kindredventures is seeking researchers with a strong engineering background to lead synthetic data, environment generation and evaluation work on large language models, with a strong focus on agentic, real-world tasks. You will drive the research and implementation of building systems to generate and evaluate data with minimal human intervention.

You will own data and evaluation quality end to end, from production usage and synthetic generation through to the evals that decide what ships.

Qualifications

  • Experience building training data, environments, or evaluations for language models.
  • Strong intuition for reward and grader design and for how data choices affect model behavior, quality, and reliability.
  • Experience with data or evaluation for reasoning or agentic tasks.
  • Strong Python skills, and experience with large-scale data pipelines and distributed systems.
  • Strong experimental design skills and a disciplined approach to evaluation.
  • Able to deeply understand research papers and turn them into working, efficient code.

Responsibilities

  • Create and curate training data, including synthetic data and data drawn from production usage.
  • Build training environments that reflect how agents are actually used in the real world, including long-horizon and multi-agent tasks.
  • Design reward signals, graders, and verifiers for customer and domain-specific tasks.
  • Build robust systems to detect and prevent reward hacking, false positives, and brittle verifiers.
  • Design evaluations that reflect real production use, and use them to decide what works and what ships.
  • Drive development of safe automated data and evaluation pipelines that run continuously and scale across customers and domain-specific tasks with minimal human intervention.
  • Work closely with colleagues to understand how data and environments shape model behavior, and close the loop between eval signal and training decisions.
  • Set research and engineering standards and mentor other engineers as the team grows.

Skills

Training data
Environments
Evaluations
Python
Distributed systems
Experimental design
Research papers

Job description

Kindredventures is seeking researchers with a strong engineering background to lead synthetic data, environment generation and evaluation work on large language models, with a strong focus on agentic, real-world tasks. You will drive the research and implementation of building systems to generate and evaluate data with minimal human intervention.

You will own data and evaluation quality end to end, from production usage and synthetic generation through to the evals that decide what ships.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Member of Technical Staff, Data & Evaluation
Member of Technical Staff, Data & Evaluation

Kindredventures • Palo Alto (CA)

On-site
USD 120,000 - 180,000
Research Engineer - Synthetic Data & Eval Systems
Research Engineer - Synthetic Data & Eval Systems

Fleet AI, Inc. • New York (NY)

On-site
USD 200,000 - 600,000
Equity
Staff Engineer, Autonomous Agents & Self-Improvement
Staff Engineer, Autonomous Agents & Self-Improvement

Kindredventures • Palo Alto (CA)

On-site
USD 180,000 - 240,000
Synthetic Data & Evals
Synthetic Data & Evals

Fleet AI, Inc. • New York (NY)

On-site
USD 200,000 - 600,000
Equity
Research Engineer — Synthetic Data & Task Pipeline
Research Engineer — Synthetic Data & Task Pipeline

Pragmatike • United States

Remote
USD 110,000 - 170,000
Remote Synthetic Data Research Engineer
Remote Synthetic Data Research Engineer

Pragmatike • Oregon

Hybrid
USD 150,000 - 190,000
Synthetic Data Research Engineer (Remote/Hybrid)
Synthetic Data Research Engineer (Remote/Hybrid)

Pragmatike • San Francisco (CA)

Hybrid
AUD 207,000 - 280,000
Research Engineer, Synthetic Data (Fully Remote)
Research Engineer, Synthetic Data (Fully Remote)

Pragmatike • United States

Remote
USD 110,000 - 170,000
Research Engineer, Synthetic Data Pipelines
Research Engineer, Synthetic Data Pipelines

Pragmatike • United States

Hybrid
USD 180,000 - 240,000
Remote-friendly
Research Engineer, Synthetic Data
Research Engineer, Synthetic Data

HUD • San Francisco (CA)

On-site
USD 140,000 - 210,000
Medical + dental + vision
Lunch and dinner in office
Holiday break
+4