Remote Software Engineer: AI Benchmarking & New Benchmarks

Remote; San Francisco (CA); Remote; USD 125,000 - 200,000; Full time

Benefits offered by this job

Remote work

Generous PTO

Staff retreats

Office access Berkeley

Job summary

Epoch AI is seeking a Software Engineer to expand our AI Benchmarking Hub, evaluating frontier AI models and improving our benchmarking infrastructure. You will run benchmarks, integrate with AI providers, and design brand-new benchmarks for our research platform.

This fully remote role welcomes applicants worldwide; you will collaborate with researchers and engineers, implement benchmarks with the Inspect library, and contribute to research products and publications.

Qualifications

• Solid software engineering background with 2+ years building and maintaining complex systems.
• Ability to generate ideas for new benchmarks, experiments, and projects.
• Motivation to contribute to rigorous, independent AI evaluations and publication-ready results.

Responsibilities

• Implement benchmarks within evaluation infrastructure using Inspect library.
• Develop new benchmarks and prototype ideas.
• Collaborate with researchers, analysts, and engineers to ensure accurate, insightful evaluation data.

Skills

Engineering experience

Idea generation

Mission-driven mindset

Tools

Inspect library

Job description

Epoch AI is seeking a Software Engineer to expand our AI Benchmarking Hub, evaluating frontier AI models and improving our benchmarking infrastructure. You will run benchmarks, integrate with AI providers, and design brand-new benchmarks for our research platform.

This fully remote role welcomes applicants worldwide; you will collaborate with researchers and engineers, implement benchmarks with the Inspect library, and contribute to research products and publications.