Finance AI Evaluation Lead for GenAI Models

Obsidian; New York (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

Cincinnatus LLC, the employer of record, invites applications for a Finance SME to join an extended workforce embedded with a leading AI lab's GenAI team. You will help build foundational AI models and craft rigorous evaluation tasks grounded in real financial practice.

We seek 8+ years in finance with hands-on experience evaluating AI outputs against rubrics, and you will collaborate with other experts to ensure high-quality training data.

Qualifications

• 8+ years of professional experience in finance.
• Experience evaluating LLM/AI outputs against rubrics is mandatory.
• Demonstrated career progression (Analyst → Associate → VP/Director).
• Availability to engage for at least 35 hours/week during weekdays.
• Strong verbal and written communication, problem-solving, and interpersonal skills.

Responsibilities

• Guide research and engineering teams to close knowledge gaps in financial reasoning, analysis, and decision-making.
• Design finance tasks and write accurate, well-reasoned solutions grounded in real financial practice.
• Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness and reasoning.
• Develop and refine evaluation guidelines and scoring rubrics for finance tasks.
• Collaborate with subject matter experts to ensure consistency and accuracy in training data.

Skills

Finance expertise

LLM evaluation

Rubrics scoring

Communication skills

Job description

Cincinnatus LLC, the employer of record, invites applications for a Finance SME to join an extended workforce embedded with a leading AI lab's GenAI team. You will help build foundational AI models and craft rigorous evaluation tasks grounded in real financial practice.

We seek 8+ years in finance with hands-on experience evaluating AI outputs against rubrics, and you will collaborate with other experts to ensure high-quality training data.