Member of Technical Staff, Research

Meridian Labs; Boston, Northern (MA, KY); Hybrid; USD 175,000 - 325,000; Full time

Job summary

Meridian Labs is hiring a research engineer to build the model-driven parts of Inspect AI tools, including auditors, scanners, transcript summarizers, and LLM judges. You will measure effectiveness and collaborate with AI safety institutes on real evaluations.

You will work in a hybrid setup based in Boston, contributing to state-of-the-art evaluations and publishing findings with researchers from partner organizations.

Qualifications

• Strong Python and ability to work in large codebases.
• Hands-on experience with LLMs: prompting, evaluation, judges or classifiers.
• Experience in AI safety or security research and testing: capability or alignment evaluations, red teaming, pre-deployment testing, alignment audits, or similar.
• Experience designing and running experiments: you pick sensible baselines and controls, check your results, look for confounds, and say how confident you are.
• Attention to detail with model outputs: you read the transcripts, not just the scores, and you notice when something looks off.
• Clear writing for both technical and non-technical readers.

Responsibilities

• Contribute to Petri, including auditor strategies, judging, and new kinds of audits.
• Build scanners and monitors that work out of the box for common problems like refusals, evaluation awareness, reward hacking, and broken environments.
• Build transcript summarizers and other tools that help researchers make sense of large numbers of agent transcripts.
• Work on LLMs as judges: prompts, tooling, and infrastructure, plus research into how reliable they are.
• Improve the scorers and metrics built into Inspect.
• Measure how well each of these works, using human labels, agreement studies, and error analysis, and publish what you find.
• Collaborate directly with researchers at AISIs and partner organizations on their evaluations and audits.

Skills

Python

LLMs prompting

Experiment design

Technical writing

Attention to detail

Clear communication

Tools

Inspect AI

Petri

Scout

Job description

Meridian Labs builds open source tools for evaluating and understanding AI models, including Inspect AI, Inspect Scout, Inspect Petri, and Inspect Flow. These tools are used by AI Safety Institutes in the UK, US, EU, Japan, and Korea, and by research organizations such as METR, Apollo, and Epoch. We also use them ourselves: Petri alignment audits run by our team appear in recent Anthropic model cards.

We’re hiring a research engineer to build the model-driven parts of these tools: alignment auditing in Petri, built-in scanners and monitors, transcript summarizers, LLM judges, and better scorers and metrics in Inspect. Part of the job is measuring how well these work, so that users know how much to trust the results.

Much of this work happens in direct collaboration with researchers at government AI Safety Institutes and other organizations that use our tools. You’ll work alongside them on real evaluations, see how your work gets used, and hear quickly when something falls short.

Responsibilities

• Contribute to Petri, including auditor strategies, judging, and new kinds of audits
• Build scanners and monitors that work out of the box for common problems like refusals, evaluation awareness, reward hacking, and broken environments
• Build transcript summarizers and other tools that help researchers make sense of large numbers of agent transcripts
• Work on LLMs as judges: prompts, tooling, and infrastructure, plus research into how reliable they are
• Improve the scorers and metrics built into Inspect
• Measure how well each of these works, using human labels, agreement studies, and error analysis, and publish what you find
• Collaborate directly with researchers at AISIs and partner organizations on their evaluations and audits

Minimum Qualifications

• Strong Python, and comfort working in large existing codebases
• Hands-on experience building with LLMs: prompting, evaluation, judges or classifiers, or similar
• Experience in AI safety or security research and testing: capability or alignment evaluations, red teaming, pre-deployment testing, alignment audits, or similar
• Experience designing and running experiments: you pick sensible baselines and controls, check your results, look for confounds, and say how confident you are
• Attention to detail with model outputs: you read the transcripts, not just the scores, and you notice when something looks off
• Clear writing for both technical and non-technical readers

Preferred Qualifications

• Published research in AI safety, security, alignment, or interpretability
• Experience measuring LLM judge or classifier reliability, such as agreement with human labels, calibration, or bias
• Background in statistics or evaluation metrics
• Experience with Inspect AI, Petri, Scout, or similar tools
• Papers, blog posts, or open source projects you can point to

If you meet most of these but not all, we’d still like to hear from you.

Location

Meridian Labs is based in Boston, MA. We prefer in-person or hybrid work in Boston, the San Francisco Bay Area, or London, and we’re open to remote for the right person.

Compensation

$175,000 – $325,000 with benefits, depending on experience and level.