Research Engineer — AI Safety Tools & Model Audits

Meridian Labs; Boston, Northern (MA, KY); Hybrid; USD 175,000 - 325,000; Full time

Job summary

Meridian Labs is hiring a research engineer to build the model-driven parts of Inspect AI tools, including auditors, scanners, transcript summarizers, and LLM judges. You will measure effectiveness and collaborate with AI safety institutes on real evaluations.

You will work in a hybrid setup based in Boston, contributing to state-of-the-art evaluations and publishing findings with researchers from partner organizations.

Qualifications

• Strong Python and ability to work in large codebases.
• Hands-on experience with LLMs: prompting, evaluation, judges or classifiers.
• Experience in AI safety or security research and testing: capability or alignment evaluations, red teaming, pre-deployment testing, alignment audits, or similar.
• Experience designing and running experiments: you pick sensible baselines and controls, check your results, look for confounds, and say how confident you are.
• Attention to detail with model outputs: you read the transcripts, not just the scores, and you notice when something looks off.
• Clear writing for both technical and non-technical readers.

Responsibilities

• Contribute to Petri, including auditor strategies, judging, and new kinds of audits.
• Build scanners and monitors that work out of the box for common problems like refusals, evaluation awareness, reward hacking, and broken environments.
• Build transcript summarizers and other tools that help researchers make sense of large numbers of agent transcripts.
• Work on LLMs as judges: prompts, tooling, and infrastructure, plus research into how reliable they are.
• Improve the scorers and metrics built into Inspect.
• Measure how well each of these works, using human labels, agreement studies, and error analysis, and publish what you find.
• Collaborate directly with researchers at AISIs and partner organizations on their evaluations and audits.

Skills

Python

LLMs prompting

Experiment design

Technical writing

Attention to detail

Clear communication

Tools

Inspect AI

Petri

Scout

Job description

Meridian Labs is hiring a research engineer to build the model-driven parts of Inspect AI tools, including auditors, scanners, transcript summarizers, and LLM judges. You will measure effectiveness and collaborate with AI safety institutes on real evaluations.

You will work in a hybrid setup based in Boston, contributing to state-of-the-art evaluations and publishing findings with researchers from partner organizations.