ML Model Evaluation Engineer

Symbolica; London; On-site; GBP 50,000 - 85,000; Full time

Job summary

A leading AI research lab in London is seeking an ML Model Evaluation Engineer to help design experiments and evaluate model capabilities. This role involves collaborating with researchers and engineers to ensure real-world performance and requires proficiency in Python and machine learning. With a focus on experimentation and collaboration, you'll contribute to an inclusive environment while working on cutting-edge AI technologies.

Benefits

Competitive salary and equity package

High trust, low bureaucracy environment

Opportunity to shape model development direction

Qualifications

• Proficient experience in machine learning with a focus on evaluation.
• Strong engineering skills in Python and PyTorch (or JAX).
• Deep understanding of large-scale deep learning model training.

Responsibilities

• Design and implement experiments for model evaluation.
• Build and maintain evaluation pipelines using PyTorch or JAX.
• Analyze evaluation outputs and identify improvement areas.

Skills

Machine Learning

Evaluation Design

Python

PyTorch

Data Analysis

Job description

About Us
Symbolica is an AI research lab pioneering the application of category theory to enable logical reasoning in machines.

We’re a well-resourced, nimble team of experts on a mission to bridge the gap between theoretical mathematics and cutting-edge technologies, creating symbolic reasoning models that think like humans – precise, logical, and interpretable. While others focus on scaling data-hungry neural networks, we’re building AI that understands the structures of thought, not just patterns in data.

Our approach combines rigorous research with fast-paced, results-driven execution. We’re reimagining the very foundations of intelligence while simultaneously developing product-focused machine learning models in a tight feedback loop, where research fuels application.

Founded in 2022, we’ve raised over $30M from leading Silicon Valley investors, including Khosla Ventures,General Catalyst, Abstract Ventures, and Day One Ventures, to push the boundaries of applying formal mathematics and logic to machine learning.

Our vision is to create AI systems that transform industries, empowering machines to solve humanity’s most complex challenges with precision and insight.JoinustoredefinethefutureofAIbyturninggroundbreakingideasintoreality.

About the Role

As an ML Model Evaluation Engineer, you’ll play a critical role in helping us measure progress, design rigorous experiments, and surface meaningful signals as we build models with structured reasoning capabilities. You’ll work alongside researchers and ML engineers to design benchmarks, run large-scale evaluations, and analyse model behaviour — ensuring we’re focused on real-world performance, not just proxy metrics.

This is a role for someone who thrives on experimentation, iteration, and tight feedback loops — someone who loves discovering what works (and what doesn’t) and can design systems to test hypotheses at scale.

This is an onsite role based in our London office (66 City Rd).

Your Focus

• Design and implement robust experiments to evaluate specific model capabilities
• Build and maintain high-frequency evaluation pipelines using PyTorch or JAX
• Engineer benchmark datasets — collecting, filtering, and decontaminating data for meaningful evals
• Create evaluation protocols that measure the right capabilities and avoid metric gaming
• Research and implement strong baselines from literature or current frontier models
• Scale experiments and data analysis to match the demands of large model training runs
• Analyse outputs from eval runs, identify bottlenecks, and present findings clearly to the team
• Collaborate with researchers and engineers to refine evaluation design and keep feedback loops tight
• Contribute to the development of a general-purpose evaluation suite integrated into infra and tooling

About You

• Proficient hands-on experience in machine learning, ideally with a focus on experimental design or evaluation
• Strong engineering skills in Python and PyTorch (or JAX)
• Deep understanding of training and evaluating large-scale deep learning models
• A scientific mindset — you know how to design a clean experiment and what makes a result trustworthy
• Comfortable building infrastructure for benchmark automation and eval pipelines
• Excellent analytical and data-mining skills; comfortable summarising experimental insights to inform team direction
• Familiarity with recent literature and capability evaluations in the frontier AI space
• Collaborative and thoughtful communicator — excited to work closely with both researchers and engineers
• Bonus: experience building benchmark suites, red-teaming evals, or integrating eval infra into full-stack ML pipelines

What We Offer

• Competitive salary and early-stage equity package
• High trust, low bureaucracy environment focused on real impact
• Opportunity to build foundational research tools and shape model development direction
• Work closely with top-notch researchers and ML engineers pushing the edge of machine reasoning

We are able to sponsor a Skilled Worker visa for qualified candidates applying to this position. This specific role exceeds the minimum salary threshold set by the UK government for Skilled Worker visa sponsorship. Please note that English language proficiency at B2 level or higher is required for this role.

Symbolica is an equal opportunities employer. We celebrate diversity and are committed to creating an inclusive environment for all employees, regardless of race, gender, age, religion, disability, or sexual orientation.

Apply for this job

*

indicates a required field

First Name *

Last Name *

Email *

Phone

Resume/CV *

Enter manually

Accepted file types: pdf, doc, docx, txt, rtf

Enter manually

Accepted file types: pdf, doc, docx, txt, rtf

Are you available to work onsite at our brand-new London office? * Select...