Finance SME - AI Evaluation Expert

Obsidian; New York (NY); On-site; USD 120,000 - 180,000; Full time

Job summary

Cincinnatus LLC, the employer of record, invites applications for a Finance SME to join an extended workforce embedded with a leading AI lab's GenAI team. You will help build foundational AI models and craft rigorous evaluation tasks grounded in real financial practice.

We seek 8+ years in finance with hands-on experience evaluating AI outputs against rubrics, and you will collaborate with other experts to ensure high-quality training data.

Qualifications

• 8+ years of professional experience in finance.
• Experience evaluating LLM/AI outputs against rubrics is mandatory.
• Demonstrated career progression (Analyst → Associate → VP/Director).
• Availability to engage for at least 35 hours/week during weekdays.
• Strong verbal and written communication, problem-solving, and interpersonal skills.

Responsibilities

• Guide research and engineering teams to close knowledge gaps in financial reasoning, analysis, and decision-making.
• Design finance tasks and write accurate, well-reasoned solutions grounded in real financial practice.
• Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness and reasoning.
• Develop and refine evaluation guidelines and scoring rubrics for finance tasks.
• Collaborate with subject matter experts to ensure consistency and accuracy in training data.

Skills

Finance expertise

LLM evaluation

Rubrics scoring

Communication skills

Job description

Join a leading AI lab's cutting-edge GenAI team to be at the core of the AI revolution, where your expertise fuels the development of the most advanced Large Language Models.

1. Overview

Join a leading AI lab's cutting-edge GenAI team and help build foundational AI models from the ground up. We're seeking talented Finance subject-matter experts (SMEs) with deep domain expertise and hands-on experience evaluating AI model outputs against rubrics to bring rigor and real-world financial judgment to our AI training data.

This is a W-2 employment position with Cincinnatus LLC, with the opportunity to be placed at a leading AI Lab as part of their extended workforce.

2. Key Responsibilities

• Guide research and engineering teams to close knowledge gaps in financial reasoning, analysis, and decision-making.

• Design challenging, domain-relevant finance tasks and write accurate, well-reasoned solutions grounded in real financial practice.

• Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.

• Develop and refine evaluation guidelines and scoring rubrics specific to finance tasks.

• Collaborate with other subject matter experts to ensure consistency and accuracy in training data.

3. Core Qualifications

• 8+ years of dedicated professional experience in finance (e.g., investment banking, asset management, corporate finance, financial advisory) at a recognized, top-tier organization (e.g., Goldman Sachs, JPMorgan, Morgan Stanley, BlackRock, Fidelity, Deloitte, PwC, EY, KPMG, or equivalent).

• Prior hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria — mandatory; please describe this experience in your application.

• Demonstrable career progression (e.g., Analyst → Associate → VP/Director).

• Ability to engage reliably for at least 35 hours/week during weekdays.

• Verbal and written communication skills, problem-solving skills, and interpersonal skills.

About Cincinnatus LLC:Cincinnatus LLC is an enterprise staffing company that partners with leading technology companies to source and employ highly skilled professionals for contingent and contract-based opportunities. Cincinnatus serves as the employer of record for these engagements, providing W-2 employment, payroll, benefits, and compliance, while placing employees directly within client teams to work on high-impact initiatives.

Equal Employment Opportunity:Cincinnatus is proud to be an Equal Employment Opportunity employer. We do not discriminate based upon race, religion, color, national origin, sex (including pregnancy, childbirth, reproductive health decisions, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, political views or activity, or any other legally protected characteristic.