AI Research Scientist

MakiPeople

Paris (TX)

On-site

USD 120,000 - 180,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Maki People is seeking an AI Research Scientist to advance scientifically robust, fair automated scoring systems. You will contribute to evaluating LLMs, building self-hosted scoring models, and calibrating psychometric methods while collaborating with engineers and product teams.

You will design studies comparing AI outputs with expert judgments, publish findings, and help shape governance and transparency around our AI solutions.

Qualifications

  • PhD or MSc in ML, linguistics, data science or psychology.
  • Publication track record (peer‑reviewed papers or conference papers).
  • Hands-on experience evaluating LLMs: benchmarking, measuring agreement with human raters, testing robustness.
  • Experience testing AI-driven decisions for fairness and bias.
  • Proficiency in Python or R, with statistical analysis.
  • Clear communication of results to scientific and business audiences.

Responsibilities

  • Design studies to compare AI scores with expert ratings and human benchmarks.
  • Develop hybrid scoring pipelines combining AI with human oversight.
  • Calibrate psychometric models for validity across populations and forms.
  • Analyze bias across demographic groups and apply mitigation techniques.
  • Publish reports and present findings to both technical and non-technical stakeholders.

Skills

Python
R
Statistical analysis
LLM evaluation
Communication
LLM benchmarking

Education

PhD or MSc in machine learning, computational linguistics, data science or psychology

Tools

LoRA fine-tuning
Cloud databases

Job description

About the Science and AI Team

The Science and AI team is at the heart of Maki People. It builds the science and AI behind our platform, from job analysis and assessments to AI interviews and predicting performance. Led by our Head of Science, Aiden Loe, and working closely with our COO, Paul-Louis Caylar, the team brings together business psychologists, psychometricians and AI scientists. Together, they make sure every hiring decision on our platform is accurate, consistent and fair.

As Maki grows, the Science and AI team’s job is to show, with evidence, that our approach leads to better hiring decisions.

About the Role

The AI Research Scientist works at the intersection of AI, psychometrics and research, ensuring that Maki’s automated scoring systems are scientifically robust, fair, and continuously improving.

1. Evaluating and testing LLMs for scoring

  • Assess the accuracy and reliability of LLMs used for automated scoring, such as structured grids, job-specific skills and multilingual proficiency (written and spoken).

  • Benchmark closed- and open-source models across assessment types and languages.

  • Test robustness to prompt variation, input noise and adversarial behaviour such as prompt injection.

  • Explore prompt engineering and model optimisation, and recommend which models and prompts go into production.

2. Building our own scoring models

  • Move our scoring onto self-hosted open-weight models, and fine-tune them on expert-rated data where they can beat frontier models.

  • Check every model switch keeps accuracy, reliability and fairness at least as good as today's.

3. Comparing AI with human judgement, and calibrating psychometric models

  • Design studies that compare AI scores with expert ratings, against both individual experts and expert averages, to establish agreement and validity.

  • Develop hybrid scoring pipelines with engineering that combine human oversight with AI scoring.

  • Evaluate and calibrate psychometric models for validity and comparability across populations and test forms.

4. Bias and fairness analysis

  • Detect and analyse bias in AI-generated and psychometric scores across demographic groups, including subgroup analysis.

  • Apply mitigation techniques such as reweighting and calibration without losing model performance.

  • Contribute to fairness dashboards and compliance documentation that support transparent model governance.

5. Predictive analytics and performance insights

  • Model the relationship between candidate scores and job performance and retention, using large assessment and performance datasets.

  • Develop and test models that estimate success probabilities or identify behavioural and linguistic predictors of performance.

  • Work with data science, implementation and customer success teams to turn findings into recommendations for clients and colleagues.

6. Research output

  • Publish reports, papers and conference presentations, internal and external.

  • Explain technical findings clearly to non-technical colleagues and clients.

As one of Maki People's early employees, you'll help shape the team's future.

Our Ideal Candidate

Required

  • PhD or MSc in machine learning, computational linguistics, data science or psychology.

  • A track record of publications, such as peer-reviewed papers or conference papers.

  • Hands-on experience evaluating LLMs: designing benchmarks, measuring agreement with human raters, and testing consistency and robustness of model output.

  • Experience testing AI-driven decisions for fairness and bias.

  • Proficiency in Python or R, with experience in statistical analysis.

  • Clear communication of results to scientific and business audiences.

Preferred

  • Experience with NLP techniques for automated scoring.

  • Experience fine-tuning language models, for example with LoRA.

  • Experience working with cloud databases and large datasets.

  • Working knowledge of psychometric concepts such as reliability, validity and IRT, and how they apply to assessment design and validation.

  • Working knowledge of ethical AI, data governance and compliance, such as the EU AI Act.

  • Experience working with engineering, product and content teams.

Application Process
  • Stage 1 - Screening assessment (25 min)

  • Stage 2 - Power skill assessment with our AI agent (15 min)

  • Stage 3 - Hiring manager interview (45 min)

  • Stage 4 - Executive interview (45 min)

  • Stage 5 - Deep-dive technical interview (60 min)

  • Stage 6 - Interview with Co-founder (30 min)

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI Research Scientist
AI Research Scientist

Maki • Paris (TX)

On-site
USD 120,000 - 180,000
AI Applied Scientist
AI Applied Scientist

Maki • Paris (TX)

On-site
USD 120,000 - 180,000
Senior Business Psychologist
Senior Business Psychologist

MakiPeople • New York (NY)

On-site
USD 120,000 - 190,000
Senior Business Psychologist
Senior Business Psychologist

Maki • New York (NY)

On-site
USD 120,000 - 180,000
AI Implementation Intern
AI Implementation Intern

Maki • New York (NY)

On-site
USD 37,000 - 58,000
Business Development Representative
Business Development Representative

Maki • Paris (TX)

On-site
USD 44,000 - 78,000
Competitive salary
Structured onboarding
Ramp buddy from BDR team
+3
Business Development Representative
Business Development Representative

Maki People • Paris (TX)

On-site
USD 39,000 - 56,000
Competitive salary
Structured onboarding
Clear growth paths
+2
Business Development Representative
Business Development Representative

Maki • New York (NY)

On-site
USD 60,000 - 90,000
Competitive salary
Structured onboarding
Ramp buddy from BDR team
+1
Strategic Account Executive - US
Strategic Account Executive - US

Maki • New York (NY)

On-site
USD 150,000 - 230,000
Equity
Best Sales tools
Market-leading compensation
+1
AI Engineer
AI Engineer

Nortiksoftware • United States

Remote
USD 150,000 - 210,000
Fully remote across Europe
22 paid vacation days plus tenure days
Private health insurance
+3