Simulation Validation and Uncertainty Engineer

CommonAI C.I.C.; Town of Cambridge; Hybrid; AUD 144,000 - 211,000; Full time

Benefits offered by this job

Vibrant Cambridge office

Hybrid work model

Competitive salary + pension

Job summary

CommonAI CIC in Cambridge is seeking a researcher to develop methods for calibrating AI simulators and quantifying uncertainty in design comparisons. You will combine mathematical reasoning with software engineering to calibrate models against measurements and test predictions on independent cases.

The role welcomes applicants from AI systems, HPC and related fields. You will work with measurement and workload engineers to design experiments, integrate uncertainty outputs into repeatable

Qualifications

• Mathematical maturity: ability to reason about models, data, variation and the assumptions behind a quantitative claim.
• Software engineering experience to turn numerical methods into readable, tested, reusable tools.
• Experience in one or more: quantitative systems performance analysis, statistical modelling, simulation validation, uncertainty quantification, scientific/HPC software or the theory of experimental design.
• Ability to collaborate with specialists and explain results and limits of evidence.

Responsibilities

• Develop and implement methods to calibrate simulator models against measurements and evaluate predictions on unseen cases.
• Distinguish uncertainty from measurements, model assumptions, parameters, and variation; represent it appropriately.
• Analyse sensitivity and uncertainty in comparisons between alternatives and their shared uncertainty sources.
• Develop outputs showing prediction uncertainty, ranking stability, and evidence limits.

Skills

Mathematical maturity

Software engineering

Uncertainty quantification

Bayesian methods

Experimental design

Job description

About CommonAI

CommonAI CIC is a non-profit organisation built around collaborative engineering and is backed by a mix of UK Government and private funding in order to design, build and deploy innovative AI systems.

Our Scaling Inference Lab (SIL) is developing simulation grounded in real measurements to help the UK invest in promising AI infrastructure. We connect proposed hardware designs to realistic workloads, helping investors and engineers understand their potential, their limitations and the next experiment worth doing.

The Opportunity

Make uncertainty a first-class output of an AI systems simulator. When a simulation predicts that one design will outperform another, you will help establish how much confidence the evidence supports, which assumptions could change the conclusion, and which experiments would reduce the uncertainty that matters to the decision.

You will combine mathematical reasoning with practical software engineering to calibrate models against measurements, test predictions on independent cases and build uncertainty analysis into the simulation workflow. Your methods and software will help engineers and investors understand what the simulator predicts and the limits of those predictions.

We welcome people from AI systems, HPC and other fields where simulation is tested against experiments. You might bring systems engineering experience and an interest in developing uncertainty methods, or experience in mathematical modelling and scientific computing that you want to apply to AI infrastructure.

What you will do

• Develop and implement methods to calibrate simulator models against measurements and evaluate predictions on cases that were not used for calibration.
• Distinguish uncertainty arising from measurements, model assumptions and parameters, operational variation and limited simulation runs, and represent it appropriately.
• Analyse sensitivity and uncertainty in comparisons between alternatives, accounting for shared sources of uncertainty and their effects on the comparison.
• Develop outputs that show how uncertain a prediction is, when a ranking could change, and when a result extends beyond the evidence used to validate it.
• Work with measurement and workload engineers to design experiments that test modelling assumptions and reduce uncertainty that matters to the decision.
• Integrate these methods into repeatable simulation workflows and communicate their assumptions and limitations to engineers and the technical analyst.
• An example early project might be to establish a calibration and validation workflow for the a comparison between two architectures, with uncertainty outputs that help explain the result. You will develop and review the methodology with the technical lead and Scientific Director, SIL.
• Mathematical maturity: the ability to reason carefully about models, data, variation and the assumptions behind a quantitative claim.
• Software engineering experience sufficient to turn numerical methods or analysis into readable, tested, reusable tools.
• Relevant experience in one or more of: quantitative systems performance analysis, statistical modelling, simulation validation, uncertainty quantification, scientific/HPC software or the theory of experimental design.
• The ability to collaborate with specialists and explain both a result and the limits of the evidence supporting it.
• Practical experience using AI coding agents and checking their output.
• Experience with model discrepancy, sensitivity analysis, Bayesian methods, surrogate models or sequential experimental design would be useful. So would familiarity with computer architecture or inference systems. We welcome complementary backgrounds: strong systems engineers developing uncertainty expertise, and engineers with modelling or uncertainty expertise moving into AI infrastructure.

Why join us

• Help build UK AI capability and improve how investment and compute serve the public good.
• Build a network spanning UK hardware SMEs, major technology companies, venture and public investors, and leading universities.
• Shape a growing team with startup pace and close academic links.
• Use frontier models and coding agents with a substantial tooling budget.
• A collaborative and supportive work environment
• The opportunity to have a high impact in a growing organisation
• Competitive salary package, including pension and benefits
• Professional development opportunities
• Networking opportunities with influential figures from across the tech sector and academia
• A vibrant office environment located a few minutes’ walk away from Cambridge train station

Working with us

Hybrid, with Cambridge attendance agreed individually around collaboration and the work; applicants across the UK are welcome. We are open to permanent employment, part-time arrangements and consultancy. We would like you to start as soon as practical and can be flexible for the right person.

We particularly welcome experienced engineers and can also consider earlier-career candidates with strong evidence of ability. Relevant postgraduate study is useful, but qualifications are secondary to experience and the quality of your work.

CommonAI CIC is an equal opportunity employer and is committed to creating an inclusive and diverse workplace.