Lead, Creative AI Evaluation & Study Programs

Contra; New York (NY); On-site; USD 150,000 - 200,000; Full time

Benefits offered by this job

Medical, dental, vision

401k matching

Company laptop

Equity

Job summary

Contra Labs in NYC seeks a studies and evaluation lead to own end-to-end research projects across creative AI labs and product teams. You will design studies, annotations, and evaluation designs, ensuring data quality and actionable results for clients.

You will bridge client needs, creative experts, and engineering, driving timelines and delivering clear findings through reports and workshops.

Qualifications

• 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
• Experience with data labeling, annotation, human-in-the-loop systems, or AI and ML evaluation workflows.
• A track record managing several concurrent projects, including annotation, evaluation, or QA work.
• A real point of view on creative quality and what 'good' looks like.

Responsibilities

• Own end-to-end projects from kickoff to final readout, coordinating across teams.
• Define annotation protocols, rubrics, and evaluation designs for creative outputs.
• Manage evaluator staffing, quality routing, timelines, and client relationships.
• Provide qualitative and quantitative analyses turned into findings, reports, and workshops.

Skills

Study design

Project management

Data analysis

Creative judgment

Job description

Contra Labs in NYC seeks a studies and evaluation lead to own end-to-end research projects across creative AI labs and product teams. You will design studies, annotations, and evaluation designs, ensuring data quality and actionable results for clients.

You will bridge client needs, creative experts, and engineering, driving timelines and delivering clear findings through reports and workshops.