Creative AI Projects Lead

Contra; New York (NY); On-site; USD 150,000 - 200,000; Full time

Benefits offered by this job

Equity

Medical, dental, vision

401k matching

Company laptop

Job summary

Contra Labs is seeking a seasoned Research Designer to own end-to-end study design, evaluation, and analysis for creative and multimodal AI workflows. You will set protocols, manage staffing and timelines, and ensure high-quality data while collaborating with engineering on tooling.

The role emphasizes translating qualitative judgments into rigorous findings, shaping benchmarks, and producing actionable client-ready outputs through reports and workshops.

Qualifications

• 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
• Experience with data labeling, annotation, human-in-the-loop systems, or AI/ML evaluation workflows.
• Track record managing several concurrent projects, including annotation, evaluation, or QA work.
• A real point of view on creative quality and what 'good' looks like.

Responsibilities

• Design studies and evaluation protocols for creative and multimodal outputs.
• Own delivery: scope, staffing, quality routing, timelines, and client relationships from kickoff to readout.
• Maintain data quality and inter-rater reliability; implement real-time course corrections.
• Perform qualitative and quantitative analysis and turn findings into actionable recommendations, reports, and workshops.
• Collaborate with engineering on tooling to automate and scale the process.
• Develop case studies and benchmark publications to sharpen Contra Labs' market understanding.

Skills

Study design

Data analysis

Findings communication

Project management

Ownership

Creative judgment

Education

Master's or PhD in HCI / cognitive science / psychology / design research

Tools

Data labeling tools

Experimentation platforms

Job description

Contra Labs is seeking a seasoned Research Designer to own end-to-end study design, evaluation, and analysis for creative and multimodal AI workflows. You will set protocols, manage staffing and timelines, and ensure high-quality data while collaborating with engineering on tooling.

The role emphasizes translating qualitative judgments into rigorous findings, shaping benchmarks, and producing actionable client-ready outputs through reports and workshops.