Platform Engineer Intern — Open-Source AI Evaluation Platform

METR; Berkeley (CA); On-site; USD 192,864 - 220,416; Part time

Benefits offered by this job

Catered meals

In-office gym

Shower facilities

Job summary

METR is seeking platform engineering interns to help build and maintain the world-leading open-source LLM evaluation platform. You will join the infrastructure team and learn largely on your own, with guidance as needed, reporting to Mischa Spiegelmock who leads the team.

You will contribute to building and debugging LLM evaluations, deploying cloud infrastructure, and enhancing observability while supporting researchers challenging top AI models.

Qualifications

• Experience with UNIX/Linux systems.
• Some low-level programming experience (C, C++, or Rust).
• Knowledge of LLMs, transformers, embeddings; ML work desirable.
• Basic web development experience (full-stack, React, TypeScript).
• Nice to have: AWS IAM/ECS/Lambda, PostgreSQL, infrastructure-as-code (Terraform/Pulumi/CDK), Kubernetes, or LLM evaluation tooling like Inspect.

Responsibilities

• Build and maintain the open-source LLM evaluation platform.
• Debug and improve LLM evaluations.
• Deploy and manage cloud infrastructure.
• Develop observability and monitoring for the cloud platform.
• Build agents to automate tasks.
• Support AI researchers in evaluating models.

Skills

UNIX/Linux

Low-level programming (C/C++, Rust)

LLM concepts (transformers, embeddings

Web development (React, TypeScript)

Tools

AWS

PostgreSQL

Terraform

Pulumi

CDK

Kubernetes

Inspect

Job description

METR is seeking platform engineering interns to help build and maintain the world-leading open-source LLM evaluation platform. You will join the infrastructure team and learn largely on your own, with guidance as needed, reporting to Mischa Spiegelmock who leads the team.

You will contribute to building and debugging LLM evaluations, deploying cloud infrastructure, and enhancing observability while supporting researchers challenging top AI models.