Creative AI Research & Evaluation Lead

Contra; San Francisco (CA); On-site; USD 150,000 - 200,000; Full time

Benefits offered by this job

Medical

Dental

Vision

401k Matching

Company Laptop

Job summary

Contra is expanding its research team in San Francisco to lead end-to-end studies on how creatives interact with AI. You will design studies, manage data labeling and evaluation workflows, and translate findings into practical guidance for clients across design, video, and imagery.

You will coordinate with creative experts and engineers to ensure rigorous methodologies, timely delivery, and measurable impact, while building reusable processes and benchmarks for ongoing partnerships.

Qualifications

• Design studies for creative and AI evaluation workflows.
• Experience labeling data and human-in-the-loop systems.
• Ability to translate findings into actionable recommendations.
• Track record managing multiple concurrent projects.

Responsibilities

• The methodology: annotation protocols, rubrics, and evaluation designs for creative and multimodal outputs.
• Delivery: scope, evaluator staffing, quality routing, timelines, and client relationships from kickoff to final readout.
• Quality: data quality, inter-rater reliability, and real-time course correction as needed.
• Analysis and storytelling: qualitative and quantitative findings, reports, and workshops.
• The systems behind it: automate repeated tasks and collaborate with engineering on tooling to speed up work.
• The narrative: case studies and benchmark publications to sharpen Contra Labs' market positioning.

Skills

Study design

Data analysis

Creative evaluation

Project management

Communication

Job description

Contra is expanding its research team in San Francisco to lead end-to-end studies on how creatives interact with AI. You will design studies, manage data labeling and evaluation workflows, and translate findings into practical guidance for clients across design, video, and imagery.

You will coordinate with creative experts and engineers to ensure rigorous methodologies, timely delivery, and measurable impact, while building reusable processes and benchmarks for ongoing partnerships.