Finance AI Model Evaluator - Contract, 20 hrs/week

Mercor; San Francisco (CA); Remote; AUD 68,880 - 103,320; Part time

Job summary

A leading tech company based in Australia is seeking an AI Model Evaluator on a contract basis. The role involves evaluating AI-generated responses, writing prompts, and providing justifications based on specific criteria. Ideal candidates will hold a Master's degree in Finance or Accounting, possess strong communication and critical reasoning skills, and have relevant professional experience. This position supports approximately 20 hours of work per week with a competitive pay rate of $50–$75 per hour.

Qualifications

• Professional experience applying domain expertise in a practitioner or advisory capacity.
• Familiarity with regulations or clinical guidelines.

Responsibilities

• Write realistic prompts that reflect domain-specific guidance.
• Evaluate AI-generated responses for accuracy and usefulness.
• Identify misleading claims across model outputs.
• Score and rank model responses using rubrics.
• Provide written justifications for evaluations.

Skills

Strong written communication

Critical reasoning

Industry-specific standards knowledge

Education

Master’s degree or higher in Finance, Accounting, or relevant field

Job description

A leading tech company based in Australia is seeking an AI Model Evaluator on a contract basis. The role involves evaluating AI-generated responses, writing prompts, and providing justifications based on specific criteria. Ideal candidates will hold a Master's degree in Finance or Accounting, possess strong communication and critical reasoning skills, and have relevant professional experience. This position supports approximately 20 hours of work per week with a competitive pay rate of $50–$75 per hour.