Strategic Projects Lead

Morpheus Talent Solutions; San Francisco (CA); Hybrid; USD 120.000 - 180.000; Vollzeit

Zusammenfassung

Morpheus Talent Solutions is seeking a Strategic Project Lead for Creative AI Evaluation, a full-time role based in San Francisco or New York with hybrid work. You will own projects that connect AI labs, creative toolmakers and client teams, ensuring data-driven studies meet ambitious timelines and quality standards.

You will design evaluation methods, manage staffing and delivery, ensure data quality, and translate findings into actionable recommendations and client-ready readouts.

Qualifikationen

• 3+ years designing studies, running evaluations and presenting findings.
• Experience with data labeling, annotation, human-in-the-loop systems or AI/ML evaluation workflows.
• Experience managing several concurrent projects, including annotation, evaluation or QA work.
• A strong sense of creative quality and ability to distinguish personal preference from ground truth.
• Ability to turn messy work into clear, measurable processes.
• High ownership, low ego, and bias to action; shipping iterations and delivering results.

Aufgaben

• Design annotation protocols, rubrics and evaluation designs for creative and multimodal outputs.
• Manage scope, evaluator staffing, quality routing, timelines, and the client relationship from kickoff to final readout.
• Ensure data quality, inter-rater reliability and real-time course correction when something feels off.
• Turn analyses into clear findings, recommendations, reports and workshops that clients act on.
• Turn repeated work into clean, measurable processes and partner with engineering on tooling for speed and reliability.
• Develop case studies and benchmark publications that sharpen how the lab is understood in the market.

Kenntnisse

Study design

Data labeling

Human-in-the-loop

Project management

Creative quality judgment

Process design

Ownership

Jobbeschreibung

Strategic Project Lead, Creative AI Evaluation

Full-time

Location

San Francisco or New York - Hybrid

Morpheus are partnered with a human-centered research lab studying how creatives actually use AI, how tools can serve creative work better, and how to protect the originality that gets flattened as everything starts to look the same. They connect a global network of top creative talent with frontier AI labs and product companies, and their work powers benchmarking, evaluation and post-training for leading models and applications. They turn human judgment into the training and evaluation layer that helps these tools serve the people who use them.

The role

The business is scaling across client engagements, including post-training dataset creation, benchmarking programs and ongoing evaluation partnerships. Each project involves a different mix of AI labs, creative-tool companies and research teams, with different requirements, timelines and quality bars. They need someone to own this work end to end.

Think of yourself as the operator for a project or vertical. You sit between three groups that do not naturally talk to each other: the client who needs the data, the creative experts producing it, and the internal engineering team building the tooling. Your job is to make all three feel like they are working on the same thing, and to ship results that are both rigorous and genuinely useful.

What you'll own

• Methodology: annotation protocols, rubrics and evaluation designs for creative and multimodal outputs
• Delivery: scope, evaluator staffing, quality routing, timelines and the client relationship, from kickoff to final readout
• Quality: data quality, inter-rater reliability and real-time course correction when something feels off. You hold the bar and the levers to fix it
• Analysis and storytelling: turning qualitative and quantitative analysis into clear findings, recommendations, reports and workshops that clients act on
• Systems: turning repeated work into clean, measurable process, and partnering with engineering on the tooling that makes it faster and more reliable
• Narrative: case studies and benchmark publications that sharpen how the lab is understood in the market

What you bring

• 3+ years designing studies, running evaluations, analysing data and presenting findings
• Experience with data labeling, annotation, human-in-the-loop systems or AI and ML evaluation workflows
• A track record managing several concurrent projects, including annotation, evaluation or QA work
• A real point of view on creative quality and what \"good\" looks like, and the ability to tell personal preference from something closer to ground truth
• The ability to turn messy work into clear, predictable, measurable process
• High ownership, low ego and a bias to action: you ship a v1, learn and do what it takes to deliver

Nice to have

• A strong research background, qualitative and quantitative, such as a Master's or PhD in HCI, cognitive science, psychology or design research, or equivalent depth from industry
• Statistical methods for evaluation work: inter-rater reliability, sampling, regression, significance testing
• RLHF, LLM evaluation or benchmark design
• Ethnography, UX research or design-study methods