Frontier AI Evaluations Engineer — Build & Automate Evals

COL Limited; City Of London; On-site; GBP 100,000 - 200,000; Full time

Job summary

A leading research organization in London is hiring Evaluation Engineers to oversee evaluation campaigns for cutting-edge AI models. The role involves automating pipelines, improving evaluation processes, and working closely with frontier labs. Ideal candidates will have a strong background in Python and data analysis, and be passionate about AI model testing. This full-time, in-person position offers a competitive salary, flexible hours, and numerous benefits.

Benefits

Flexible work hours and schedule

Unlimited vacation

Lunch, dinner, and snacks provided

Paid work trips and conferences

Yearly professional development budget

Qualifications

• Strong software engineering experience in Python.
• Comfortable with quantitative analysis and qualitative assessment.
• Ability to convey findings succinctly to various audiences.

Responsibilities

• Run and own evaluation campaigns for unreleased models.
• Automate evaluation pipeline and improve infrastructure.
• Develop a larger vision for evaluation processes.

Skills

Software engineering skills

Process optimisation

Data Analysis & Pattern Recognition

Writing and communication

AI power-user

Job description

A leading research organization in London is hiring Evaluation Engineers to oversee evaluation campaigns for cutting-edge AI models. The role involves automating pipelines, improving evaluation processes, and working closely with frontier labs. Ideal candidates will have a strong background in Python and data analysis, and be passionate about AI model testing. This full-time, in-person position offers a competitive salary, flexible hours, and numerous benefits.