Lead, Creative AI Studies & Evaluation

CONTRA Service Productions International; New York (NY); On-site; USD 150,000 - 200,000; Full time

Benefits offered by this job

Medical benefits

Dental benefits

Vision benefits

401k matching

Laptop provided

Equity

Job summary

Contra Labs, based in New York, is seeking a studies-focused researcher to own end-to-end projects from kickoff to readout. You will design studies and evaluation protocols, supervise data labeling and QA, analyze results, and deliver actionable recommendations to clients.

You will collaborate with engineering to improve tooling, and help craft case studies and benchmark narratives that shape industry understanding of creative AI evaluation.

Qualifications

• 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
• Experience with data labeling, annotation, human-in-the-loop systems, or AI and ML evaluation workflows.
• A track record managing several concurrent projects, including annotation, evaluation, or QA work.
• A real point of view on creative quality and what 'good' looks like.

Responsibilities

• Own end-to-end projects across client engagements, from kickoff to readout.
• Design studies, evaluation protocols, rubrics, and reporting templates.
• Lead data collection, labeling, and quality control processes.
• Analyze results and translate into concrete recommendations for clients.
• Collaborate with engineering on tooling to streamline workflows.
• Craft case studies and benchmark narratives for market positioning.

Skills

Study design

Data labeling

Multi-project management

Creative quality framework

Qualitative analysis

Education

Master's or PhD in HCI, cognitive science, psychology, design research, or related field

Tools

Annotation tools

Human-in-the-loop systems

AI/ML evaluation workflows

Job description

Contra Labs, based in New York, is seeking a studies-focused researcher to own end-to-end projects from kickoff to readout. You will design studies and evaluation protocols, supervise data labeling and QA, analyze results, and deliver actionable recommendations to clients.

You will collaborate with engineering to improve tooling, and help craft case studies and benchmark narratives that shape industry understanding of creative AI evaluation.