Founding AI Researcher, Model Evaluation & Data Strategy

Talanto; San Francisco, Northern (CA, KY); Hybrid; USD 200,000 - 350,000; Full time

Job summary

Talanto is seeking a founding Applied AI Researcher focused on model evaluation and data strategy in a research-first role. You will design evaluations, define datasets and reward signals, and work with engineers to scale programs. Publishing is core and prior publication is essential.

The role requires modeling expertise across modalities, strong Python, and the ability to drive research independently. In-person SF Bay Area preference with open remote options across multiple regions.

Qualifications

• Track record of published research at NeurIPS, ICML, ICLR, ACL or EMNLP is essential.
• You have designed studies and evaluations, not just followed rubrics.
• Comfort with non-deterministic domains and fast-moving research frameworks.
• Agentic evaluation proficiency; RL environment experience with engineers.
• Cross-modal understanding: audio, text, and vision require different techniques.
• Strong Python, model APIs, and structured datasets; benchmark design, human eval, statistics.
• Familiarity with SFT, preference optimization, RLHF/RLAIF, reward modeling, synthetic data.
• Excellent technical writing and ability to drive ambiguous research independently.

Responsibilities

• Design evaluations for generative, reasoning, tool-use, and agentic systems across modalities.
• Own experimental design: hypothesize failures, build evals, quantify defects.
• Build RL environments and reward signals with engineers.
• Create quality systems: calibration, blind review, adjudication for non-deterministic domains.
• Run pilots and publish work to position the company as a research partner.

Skills

Python

Publication record

Experimental design

Statistical analysis

Cross-modal understanding

RL environment experience

Benchmark design

Technical writing

Tools

Model APIs

RL environments

Eval frameworks

Job description

Talanto is seeking a founding Applied AI Researcher focused on model evaluation and data strategy in a research-first role. You will design evaluations, define datasets and reward signals, and work with engineers to scale programs. Publishing is core and prior publication is essential.

The role requires modeling expertise across modalities, strong Python, and the ability to drive research independently. In-person SF Bay Area preference with open remote options across multiple regions.