Finance AI Evaluation Expert for GenAI Models

Mercor; New York (NY); On-site; USD 150,000 - 210,000; Full time

Job summary

Cincinnatus LLC is seeking Finance SMEs to join an AI lab's GenAI team, evaluating model outputs against rubrics and grounding judgments in real-world financial practice. This is a W-2 employment position with placement at a leading AI Lab.

You will design finance tasks, provide written feedback, and help build rigorous evaluation guidelines, while collaborating with other experts to ensure consistency. 35 hours/week available on weekdays; strong finance background required.

Qualifications

• 8+ years of dedicated professional experience in finance at top-tier organizations.
• Prior hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria.
• Demonstrable career progression (Analyst → Associate → VP/Director).
• Ability to engage reliably for at least 35 hours/week during weekdays.
• Verbal and written communication, problem-solving and interpersonal skills.

Responsibilities

• Guide research and engineering teams to close knowledge gaps in financial reasoning, analysis, and decision-making.
• Design challenging, domain-relevant finance tasks and write accurate, well-reasoned solutions grounded in real financial practice.
• Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.
• Develop and refine evaluation guidelines and scoring rubrics specific to finance tasks.
• Collaborate with other subject matter experts to ensure consistency and accuracy in training data.

Skills

Finance expertise

AI evaluation experience

Communication skills

Job description

Cincinnatus LLC is seeking Finance SMEs to join an AI lab's GenAI team, evaluating model outputs against rubrics and grounding judgments in real-world financial practice. This is a W-2 employment position with placement at a leading AI Lab.

You will design finance tasks, provide written feedback, and help build rigorous evaluation guidelines, while collaborating with other experts to ensure consistency. 35 hours/week available on weekdays; strong finance background required.