Research Scientist - Data-Driven AI Training & Evaluation

AfterQuery; San Francisco (CA); On-site; USD 150,000 - 450,000; Full time

Job summary

A leading data evaluation company in San Francisco seeks a candidate to design and run training experiments that prove the efficacy of their datasets on model performance. You will analyze data structure and quality, run SFT and RL experiments, and communicate insights with partner labs. Ideal candidates are undergrad or master's researchers who have a passion for actionable data insights in various domains. Total compensation ranges between $250k and $450k, plus equity.

Qualifications

• Strong familiarity with LLM training and evaluation methodologies.
• Genuine obsession with how data structure, selection, and quality drive model behavior.
• Ability to design lightweight experiments and extract actionable insights.

Responsibilities

• Run controlled SFT and RL experiments to measure the impact of our datasets.
• Quantify lift across various capabilities.
• Communicate findings with partner labs to drive sales.
• Work with internal SPLs to iterate on data quality.

Skills

Familiarity with LLM training methodologies

Data structure and quality analysis

Experiment design and execution

Cross-domain communication

Education

Undergraduate or Master's research

Job description

A leading data evaluation company in San Francisco seeks a candidate to design and run training experiments that prove the efficacy of their datasets on model performance. You will analyze data structure and quality, run SFT and RL experiments, and communicate insights with partner labs. Ideal candidates are undergrad or master's researchers who have a passion for actionable data insights in various domains. Total compensation ranges between $250k and $450k, plus equity.