AI Evaluation Scientist

Cambridge University Press & Assessment; Cambridge; Hybrid; GBP 40,000 - 70,000; Full time

Job summary

An established industry player is seeking an AI Evaluation Scientist to join their innovative Applied AI team. This role involves evaluating AI-enhanced assessment technologies to ensure accuracy and fairness, contributing to the development of validation frameworks, and applying cutting-edge evaluation methods. The ideal candidate will possess a strong background in inferential statistics and experience with AI model evaluation. Join a collaborative environment that values ethical AI principles and offers flexible working options, along with a competitive rewards package. If you're passionate about advancing AI in education, this opportunity is for you.

Benefits

28 days annual leave plus bank holidays

Private medical insurance

Permanent health insurance

Discretionary annual bonus

Life assurance up to 4 x annual salary

Qualifications

• Strong knowledge of inferential statistics and multivariate analysis.
• Experience evaluating AI models using various methods.

Responsibilities

• Conduct statistical evaluations for AI-enhanced assessment technologies.
• Develop validation frameworks to ensure compliance with industry standards.

Skills

Inferential Statistics

AI Evaluation Methods

Statistical Analysis

Project Management

Ethical AI Principles

Education

Degree in Statistics, Data Science, AI, or Mathematics

Tools

Python

R

Job description

Contract: Permanent, Full Time, 35 hours per week

Are you passionate about AI and data science, as well as ensuring fairness, transparency, and accuracy in AI-enhanced assessment?Are you excited about expanding language learning opportunities and contributing to the best of international practice? If so, we want to hear from you!

We are Cambridge University Press & Assessment, a world-leading academic publisher and assessment organisation and a proud part of the University of Cambridge. Our Applied AI team is looking for a talented AI evaluation specialist to join our expanding team of validation specialists and machine learning engineers.

As an AI Evaluation Scientist, you will play a critical role in evaluating and improving our AI-enhanced capabilities. Your expertise in statistical analysis and AI evaluation will ensure that our next-generation assessment solutions meet the highest validity, fairness, and regulatory standards. This role is central to maintaining Cambridge's reputation as a global leader in AI-enhanced education and assessment.

About The Role

As part of our Applied AI team, you will be responsible for helping drive the evaluation of our AI-enhanced assessment technologies to ensure their accuracy, fairness, and explainability. You will work closely with validation specialists and machine learning engineers to implement robust statistical analyses and help develop innovative evaluation methods to help shape the future of AI-powered second language learning.

Key responsibilities include:

• Conducting statistical evaluations to robustly assess the fairness, accuracy, and transparency of our AI-enhanced assessment technologies, ensuring they align with Cambridge's educational assessment principles.
• Contributing to the development and design of validation frameworks and wider quality assurance processes to ensure AI models meet regulatory requirements and industry best practices.
• Identifying and applying cutting-edge AI evaluation methods, helping advance both our AI capabilities and the science of AI evaluation.
• Working with validation specialists and machine learning engineers to help develop next generation AI-driven educational technologies.
• Publishing findings in academic and industry contexts, presenting insights to both technical and non-technical stakeholders, and representing Cambridge as an expert in AI evaluation.

About You

We are looking for an analytical thinker with practical and theoretical knowledge of inferential statistics and AI evaluation methods who enjoys working as part of a diverse team of specialists. You will excel in this role if you have:

• A degree in a quantitative field (e.g., Statistics, Data Science, AI, Mathematics) or equivalent work experience, ideally with a focus on AI model evaluation.
• Strong theoretical and practical knowledge of inferential statistics and multivariate analysis.
• Practical experience of evaluating AI models using a range of methods, including methods relating to the domain of explainable AI.
• Proficiency in either Python or R for conducting complex statistical analyses (Python preferred).
• An ability and willingness to learn and apply new statistical approaches for addressing particular evaluation challenges.
• An ability to manage projects in a structured, deadline-driven environment, ensuring timely delivery of evaluation results.
• An ability to translate technical findings into actionable insights for cross-functional teams, influencing decision-making at senior levels.
• A strong commitment to ethical AI principles and educational fairness, aligned with Cambridge's mission to expand learning opportunities for all.
• Experience of evaluating AI-enhanced assessment and learning technologies.
• Experience of collaborating with ML engineers in the development and evaluation of AI models.
• Experience of assessment principles and evaluation methods.
• Experience of embedding standardized AI evaluation processes in an industry setting.

Rewards and benefits

We will support you to be at your best in work and to live well outside of it. In addition to competitive salaries, we offer a world-class, flexible rewards package , featuring family-friendly and planet-friendly benefits including:

• 28 days annual leave plus bank holidays
• Private medical and Permanent Health Insurance
• Discretionary annual bonus
• Life assurance up to 4 x annual salary

We are a hybrid working organisation, and we offer a range of flexible working options from day one. We expect most hybrid-working colleagues to spend 40-60% of their time at their dedicated office or location. We will also consider other work arrangements if you wish to work more flexibly or require adjustments due to a disability.

Ready to pursue your potential? Apply now.

We review applications on an ongoing basis, with a closing date for all applications being Sunday 18 th May 2025. You can expect to receive an update on your application and whether you have been shortlisted for interview within the following two weeks.

We reserve the right to close this vacancy once we have received sufficientapplications for the role. Therefore, if you are interested in this exciting opportunity, we encourage you to submit your application as early as possible.

We are committed to an equitable recruitment process. As such, all applications must be submitted via our official online application form. Please refrain from sending your CV directly to our recruiters.

Please note that successful applicants will be subject to satisfactory background checks including DBS due to working in a regulated industry.

Cambridge University Press & Assessment is an approved UK employer for the sponsorship of eligible roles and applicants under the Skilled Worker visa route. Please refer to the gov.uk website for guidance to understand your own eligibility based on the role you are applying for.

Joining us is your opportunity to pursue potential. You'll belong to a collaborative team that's exploring new and better ways to serve students, teachers and researchers across the globe – for the benefit of individuals, society and the world. Sharing our mission will inspire your own growth, development and progress, in an environment which embraces difference, change and aspiration.

Cambridge University Press & Assessment is committed to being a place where anyone can enjoy a successful career, where it's safe to speak up, and where we learn continuously to improve together. We welcome applications from all candidates, regardless of demographic characteristics (age, disability, educational attainment, ethnicity, gender, marital status, neurodiversity, religion, sex, gender identityand sexual identity), cultural, or social class/background.

We believe better outcomes come through diversity of thought, background and approach. We welcome applications from people from all backgrounds and communities, actively seeking to employ people from a wide range of different communities.