AI Model Evaluation Scientist - Data & Quant

Gramian Consulting Group; New Zealand; On-site; NZD 90,000 - 150,000; Full time

Job summary

Gramian Consultancy is seeking Data Scientists and Quantitative Professionals to evaluate and improve next-generation AI models. You will assess AI-generated solutions, design prompts and datasets, and identify statistical errors while explaining complex concepts with clarity.

The role targets professionals with experience in technology, finance, or research who can convey technical ideas effectively and provide actionable feedback for model improvement.

Qualifications

• At least one year at a top company (tech, finance, or research) and 3 years total DS/SWE experience.
• Bachelor’s degree in DS/SWE (or related) from US/CA/AU/NZ/UK.
• Foundational knowledge in data science, statistics, ML, or quantitative analysis.
• Strong analytical and critical-thinking skills; able to spot errors in data-driven reasoning.
• Excellent written English and ability to explain technical concepts clearly.
• Residence in an English-speaking country.

Responsibilities

• Evaluate AI-generated outputs across data science, statistics, ML, experimentation, and quantitative problem-solving.
• Create expert-level prompts, datasets, and reference solutions for AI training and evaluation.
• Identify flawed methodologies, statistical inaccuracies, unsupported assumptions, and weaknesses in quantitative reasoning.
• Assess the correctness, reliability, and clarity of analytical approaches and conclusions.
• Provide structured, evidence-based feedback to help improve AI model capabilities.
• Apply consistent evaluation criteria and communicate technical findings clearly.

Skills

Data science fundamentals

Statistical reasoning

Analytical thinking

Education

Bachelor’s degree in DS/SWE or related

Job description

Gramian Consultancy is seeking Data Scientists and Quantitative Professionals to evaluate and improve next-generation AI models. You will assess AI-generated solutions, design prompts and datasets, and identify statistical errors while explaining complex concepts with clarity.

The role targets professionals with experience in technology, finance, or research who can convey technical ideas effectively and provide actionable feedback for model improvement.