AI Research Scientist: LLM Benchmarks & Fairness

MakiPeople

Paris

On-site

EUR 90,000 - 130,000

Full time

6 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Maki People seeks an AI Research Scientist to advance its science and AI framework at our Paris office. You will work at the intersection of AI, psychometrics and research to ensure robust, fair and improving automated scoring systems.

Responsibilities include evaluating LLMs for scoring accuracy, building self-hosted models, and calibrating psychometric methods across populations, with an emphasis on fairness and transparency.

Qualifications

  • A PhD or MSc in ML, NLP, data science, or psychology.
  • Hands-on evaluation of LLMs: benchmarks, agreement with human raters, robustness tests.
  • Experience testing AI-driven decisions for fairness and bias.
  • Proficiency in Python or R and statistical analysis.

Responsibilities

  • Evaluate and test LLMs for scoring accuracy and reliability across languages.
  • Build and fine-tune self-hosted scoring models with expert-rated data.
  • Design studies comparing AI scores with human judgments for validity.
  • Develop hybrid scoring pipelines combining AI and human oversight.
  • Analyze bias across demographic groups and contribute to fairness dashboards.
  • Publish reports and present findings to non-technical audiences.

Skills

Python or R
Statistical analysis
Experimental design
Communication of results

Education

PhD or MSc in ML/Computational Linguistics/Data Science/Psychology

Tools

LoRA
Cloud databases

Job description

Maki People seeks an AI Research Scientist to advance its science and AI framework at our Paris office. You will work at the intersection of AI, psychometrics and research to ensure robust, fair and improving automated scoring systems.

Responsibilities include evaluating LLMs for scoring accuracy, building self-hosted models, and calibrating psychometric methods across populations, with an emphasis on fairness and transparency.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Research Scientist: LLM Benchmarking & Fairness
AI Research Scientist: LLM Benchmarking & Fairness

Maki • Paris

On-site
EUR 70,000 - 110,000
Research Scientist: Core Learning & Reasoning for LLMs
Research Scientist: Core Learning & Reasoning for LLMs

Meta • Paris

On-site
EUR 90,000 - 150,000
AI Applied Scientist: Elevate Hiring, Scoring & Fairness
AI Applied Scientist: Elevate Hiring, Scoring & Fairness

MakiPeople • Paris

On-site
EUR 65,000 - 110,000
AI Applied Scientist
AI Applied Scientist

MakiPeople • France

Hybrid
EUR 90,000 - 130,000
AI Solutions Scientist — Client Impact & Fair Scoring
AI Solutions Scientist — Client Impact & Fair Scoring

MakiPeople • France

Hybrid
EUR 90,000 - 130,000
AI Applied Scientist: Client Impact & Fairness Champion
AI Applied Scientist: Client Impact & Fairness Champion

MakiPeople • Paris

On-site
EUR 70,000 - 110,000
Applied AI Evaluation Engineer: LLM Benchmarks & Pipelines
Applied AI Evaluation Engineer: LLM Benchmarks & Pipelines

Mistral • Paris

On-site
EUR 90,000 - 130,000
Healthcare coverage
Parental leave
Retirement plans
+3
AI Research Scientist
AI Research Scientist

MakiPeople • Paris

On-site
EUR 90,000 - 130,000
AI Research Scientist (Paris)
AI Research Scientist (Paris)

Lexsi Labs • Paris

On-site
EUR 60,000 - 90,000
Research Engineer – AI Agents & LLM Systems F/M
Research Engineer – AI Agents & LLM Systems F/M

Adoc Talent Management • Paris

On-site
EUR 85,000 - 120,000