End-to-End Creative AI Study Lead

Contra; New York (NY); Sur place; USD 150 000 - 200 000; Plein temps

Avantages offerts par ce poste

Medical, dental, vision

401k matching

Company laptop

Equity

Résumé du poste

Contra Labs is seeking a researcher to own end-to-end studies of creative AI evaluation, connecting clients, creatives, and engineering to produce rigorous, actionable findings.

You will design studies, run evaluations, analyze data, and translate results into recommendations. The role spans benchmarking, dataset work, and QA processes, with a strong emphasis on real-world creative quality.

Qualifications

• 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
• Experience with data labeling, annotation, human-in-the-loop systems, or AI and ML evaluation workflows.
• A track record managing several concurrent projects, including annotation, evaluation, or QA work.
• A real point of view on creative quality and what "good" looks like.
• Based in NYC and comfortable in an in-office setting.

Responsabilités

• Own the methodology for annotation protocols, rubrics, and evaluation designs for creative and multimodal outputs.
• Manage delivery: scope, staffing, quality routing, timelines, and client relationships from kickoff to final readout.
• Ensure data quality, inter-rater reliability, and real-time course correction.
• Perform analysis and storytelling to produce findings, recommendations, and workshops for clients.
• Build systems to turn repeated work into clean, measurable processes with engineering collaboration.
• Develop case studies and benchmark publications to sharpen Contra Labs' market position.

Connaissances

Study design

Data analysis

Project management

Communication

Creative evaluation

Formation

Master's or PhD in HCI or related field

Outils

Annotation tools

Statistical software

Description du poste

Contra Labs is seeking a researcher to own end-to-end studies of creative AI evaluation, connecting clients, creatives, and engineering to produce rigorous, actionable findings.

You will design studies, run evaluations, analyze data, and translate results into recommendations. The role spans benchmarking, dataset work, and QA processes, with a strong emphasis on real-world creative quality.