AI Ground Truth Architect & Domain Expert

Agency; United States; Remote; USD 207,000 - 358,000; Full time

Job summary

Agency is seeking elite Subject Matter Experts to partner on high-impact AI evaluation and training projects from a remote location. This consulting engagement targets the top 1% in their field to shape the "Ground Truth" for next-generation LLMs.

You will create "Golden Tasks" and rubrics, analyze model failures, and advise on prompt complexity within a non-employee, independent contract framework. Engagements are project-based and non-exclusive with flexible deliverables.

Qualifications

• Elite Domain Mastery with advanced degrees or equivalent status.
• Professional autonomy as an independent consultant or freelancer.
• Interest in shaping technology and setting a gold standard for AI.
• Ability to articulate reasoning in writing.

Responsibilities

• Rubric architecture to evaluate AI outputs for professional scrutiny.
• Golden data curation: create or certify benchmark prompts/responses.
• Expert gap analysis explaining model failures with human solutions.
• Advisory on prompt complexity for rigorous testing scenarios.

Skills

Elite Domain Mastery

Professional Autonomy

Interest in Shaping Technology

Articulate Reasoning

Education

Advanced degrees (PhD, MD, JD) or equivalent status

Job description

Agency is seeking elite Subject Matter Experts to partner on high-impact AI evaluation and training projects from a remote location. This consulting engagement targets the top 1% in their field to shape the "Ground Truth" for next-generation LLMs.

You will create "Golden Tasks" and rubrics, analyze model failures, and advise on prompt complexity within a non-employee, independent contract framework. Engagements are project-based and non-exclusive with flexible deliverables.