Finance AI Model Evaluation Specialist

Mercor; United States; On-site; USD 180,000 - 280,000; Full time

Job summary

Cincinnatus LLC invites you to join its GenAI team inside a leading AI lab, focusing on finance-oriented evaluation tasks to improve training data for foundational models.

You're an SME with 8+ years in finance, experienced in evaluating outputs against rubrics, and able to commit 35 hours/week, collaborating with engineers to design finance tasks and provide rigorous feedback. This W-2 role places you with the AI lab as part of their extended workforce.

Qualifications

• 8+ years of dedicated professional experience in finance at a top-tier organization.
• Hands-on experience evaluating AI model outputs against rubrics or structured scoring criteria.
• Clear career progression from Analyst to VP/Director.
• Ability to engage reliably for at least 35 hours/week during weekdays.
• Strong verbal and written communication, problem-solving and interpersonal skills.

Responsibilities

• Guide research and engineering teams to close knowledge gaps in financial reasoning, analysis, and decision-making.
• Design challenging, domain-relevant finance tasks and write accurate, well-reasoned solutions grounded in real financial practice.
• Evaluate AI model outputs against structured rubrics and provide clear feedback on correctness, judgment, and reasoning quality.
• Develop and refine evaluation guidelines and scoring rubrics specific to finance tasks.
• Collaborate with other subject matter experts to ensure consistency and accuracy in training data.

Skills

Finance domain expertise

LLM/AI evaluation experience

Career progression awareness

35+ hours/week availability

Communication and problem-solving

Job description

Cincinnatus LLC invites you to join its GenAI team inside a leading AI lab, focusing on finance-oriented evaluation tasks to improve training data for foundational models.

You're an SME with 8+ years in finance, experienced in evaluating outputs against rubrics, and able to commit 35 hours/week, collaborating with engineers to design finance tasks and provide rigorous feedback. This W-2 role places you with the AI lab as part of their extended workforce.