Member of Technical Staff, Data & Evaluation

Kindredventures

Palo Alto (CA)

On-site

USD 120,000 - 180,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Kindredventures is seeking researchers with a strong engineering background to lead synthetic data, environment generation and evaluation work on large language models, with a strong focus on agentic, real-world tasks. You will drive the research and implementation of building systems to generate and evaluate data with minimal human intervention.

You will own data and evaluation quality end to end, from production usage and synthetic generation through to the evals that decide what ships.

Qualifications

  • Experience building training data, environments, or evaluations for language models.
  • Strong intuition for reward and grader design and for how data choices affect model behavior, quality, and reliability.
  • Experience with data or evaluation for reasoning or agentic tasks.
  • Strong Python skills, and experience with large-scale data pipelines and distributed systems.
  • Strong experimental design skills and a disciplined approach to evaluation.
  • Able to deeply understand research papers and turn them into working, efficient code.

Responsibilities

  • Create and curate training data, including synthetic data and data drawn from production usage.
  • Build training environments that reflect how agents are actually used in the real world, including long-horizon and multi-agent tasks.
  • Design reward signals, graders, and verifiers for customer and domain-specific tasks.
  • Build robust systems to detect and prevent reward hacking, false positives, and brittle verifiers.
  • Design evaluations that reflect real production use, and use them to decide what works and what ships.
  • Drive development of safe automated data and evaluation pipelines that run continuously and scale across customers and domain-specific tasks with minimal human intervention.
  • Work closely with colleagues to understand how data and environments shape model behavior, and close the loop between eval signal and training decisions.
  • Set research and engineering standards and mentor other engineers as the team grows.

Skills

Training data
Environments
Evaluations
Python
Distributed systems
Experimental design
Research papers

Job description

About Us
We are a small team of researchers and engineers focused on pushing the frontier of what agents are capable of. We believe the next leap in agent performance will come not from bigger general models, but from specialization and continual learning: agents that get better at their specific job the more they do it. We build the systems that make this possible, turning real-world usage into lasting improvement.

Summary
We are looking for researchers with a strong engineering background to lead synthetic data, environment generation and evaluation work on large language models, with a strong focus on agentic, real-world tasks. You will drive the research and implementation of building systems to generate and evaluate data with minimal human intervention. You will own data and evaluation quality end to end, from production usage and synthetic generation through to the evals that decide what ships. Your work will directly shape the agents our customers rely on in production.

Responsibilities

  • Create and curate training data, including synthetic data and data drawn from production usage.

  • Build training environments that reflect how agents are actually used in the real world, including long-horizon and multi-agent tasks.

  • Design reward signals, graders, and verifiers for customer and domain-specific tasks.

  • Build robust systems to detect and prevent reward hacking, false positives, and brittle verifiers.

  • Design evaluations that reflect real production use, and use them to decide what works and what ships.

  • Drive development of safe automated data and evaluation pipelines that run continuously and scale across customers and domain-specific tasks with minimal human intervention.

  • Work closely with colleagues to understand how data and environments shape model behavior, and close the loop between eval signal and training decisions.

  • Set research and engineering standards and mentor other engineers as the team grows.

Qualifications

  • Deep, practical experience building training data, environments, or evaluations for language models, shown through shipped models, releases, or publications.

  • Strong intuition for reward and grader design, and for how data choices affect model behavior, quality, and reliability.

  • Experience with data or evaluation for reasoning or agentic tasks.

  • Strong Python skills, and experience with large-scale data pipelines and distributed systems.

  • Strong experimental design skills and a disciplined approach to evaluation.

  • Able to deeply understand research papers and turn them into working, efficient code.

Nice to Have

  • Synthetic data or evaluation experience at a frontier lab or on widely used open models.

  • Publications at top ML conferences or well-known technical blog posts.

  • Experience building environments or evaluations for long-horizon, multi-step tasks.

  • Experience with continual learning or improving models from real-world feedback.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Member of Technical Staff, Agents & Recursive Self-Improvement (RSI)
Member of Technical Staff, Agents & Recursive Self-Improvement (RSI)

Kindredventures • Palo Alto (CA)

On-site
USD 180,000 - 240,000
Member of Technical Staff, Post-Training
Member of Technical Staff, Post-Training

Kindredventures • Palo Alto (CA)

On-site
USD 150,000 - 190,000
Member of Technical Staff, Evaluations
Member of Technical Staff, Evaluations

Bake AI Inc. • San Mateo (CA)

On-site
USD 180,000 - 240,000
Member of Technical Staff - Research Engineering
Member of Technical Staff - Research Engineering

NextToken AI Inc. • Mountain View (CA)

On-site
USD 180,000 - 240,000
Staff Engineer - Synthetic Data & Agentic Evaluation
Staff Engineer - Synthetic Data & Agentic Evaluation

Kindredventures • Palo Alto (CA)

On-site
USD 120,000 - 180,000
Member of Technical Staff, Applied AI Engineer
Member of Technical Staff, Applied AI Engineer

Kindredventures • Palo Alto (CA)

On-site
USD 190,000 - 250,000
Senior AI Researcher, Evaluation & Agent Performance
Senior AI Researcher, Evaluation & Agent Performance

Cobalt • New York (NY)

On-site
USD 160,000 - 230,000
Applied Research — Evaluations & Data
Applied Research — Evaluations & Data

Human Intuition Inc. • New York (NY)

On-site
USD 120,000 - 180,000
Member of Technical Staff, RL Infrastructure
Member of Technical Staff, RL Infrastructure

Kindredventures • Palo Alto (CA)

On-site
USD 150,000 - 230,000
Member of Technical Staff: Agents, Evaluations, and Environments
Member of Technical Staff: Agents, Evaluations, and Environments

Asymmetric Security • New York (NY)

Hybrid
USD 180,000 - 240,000