End-to-End Creative AI Study Lead

CONTRA Service Productions International; San Francisco (CA); On-site; USD 150,000 - 200,000; Full time

Benefits offered by this job

Equity

Company laptop

401k matching

Job summary

Contra Labs is a human-centered research lab exploring how creatives use AI and how tools can better serve creative work. We connect a global network of top talent with frontier AI labs and product companies to ground our research in real workflows across design, video, and imagery.

You will own the methodology, delivery, quality, analysis, and the systems behind it—driving end-to-end projects that unlock rigorous, useful insights for clients.

Qualifications

• 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
• Experience with data labeling, annotation, human-in-the-loop systems, or AI/ML evaluation workflows.
• A track record managing several concurrent projects including annotation, evaluation, or QA work.
• A real point of view on creative quality and what 'good' looks like.

Responsibilities

• Own the methodology: annotation protocols, rubrics, and evaluation designs for creative outputs.
• Own delivery: scope, staffing, quality routing, timelines, and client relationships from kickoff to readout.
• Maintain quality: data quality, inter-rater reliability, real-time course correction.
• Conduct analysis and storytelling: turn findings into clear recommendations and workshops.
• Build scalable systems: partner with engineering on tooling to speed and stabilize processes.
• Develop narratives: case studies and benchmark publications that shape Contra Labs' market view.

Skills

Study design

Data analysis

Evaluation leadership

Project management

Communication

Education

Master's or PhD in HCI / cognitive science / psychology / design research

Tools

Python

R

Annotation tooling

Job description

Contra Labs is a human-centered research lab exploring how creatives use AI and how tools can better serve creative work. We connect a global network of top talent with frontier AI labs and product companies to ground our research in real workflows across design, video, and imagery.

You will own the methodology, delivery, quality, analysis, and the systems behind it—driving end-to-end projects that unlock rigorous, useful insights for clients.