AI Assurance Lead (LLM Evaluation, Fine Tuning and Model Risk)

Intelance; United States; Remote; USD 160,000 - 197,000; Part time

Job summary

Intelance is a UK-based independent technology, AI and cyber assurance firm seeking a role focused on due diligence of AI systems. You will assess how AI sits in products, how models are trained and governed, and how data is used and contracts align with compliance requirements.

The position emphasizes practical fixes, clear client communication and experience with regulated data. Remote UK-based work with occasional London meetings, freelance or through a limited company, and day rates of

Qualifications

• At least five years building, evaluating or assuring machine learning or language model systems.
• Hands on experience of fine tuning open source language models and designing evaluation suites.
• Working knowledge of ISO/IEC 42001, the NIST AI Risk Management Framework or the EU AI Act.
• The ability to explain model risk in plain words to founders, buyers and lawyers.
• Experience with health, clinical or other regulated data is a strong plus.
• Experience of due diligence, audit or model validation work is a strong plus.

Responsibilities

• Map where AI sits in a product: which steps are plain code, which use machine learning and which use language models.
• Review how open source models are fine tuned, what data they learn from, and whether client contracts allow that use.
• Test the evaluation approach: test suites, accuracy measures, human review rules and how failures are caught and fixed.
• Check controls against prompt injection, data leaking between clients and tampering with models.
• Check reliance on outside AI providers, and what data, if any, they receive.
• Rate each finding by how it could affect the deal, and write clear, practical fixes.

Skills

ML system evaluation

Fine-tuning open source LMs

Evaluation suites design

Plain-language risk explanation

AI risk frameworks knowledge

Tools

Python

Job description

About Intelance

Intelance is an independent technology, AI and cyber assurance firm based in the UK. We help organisations check, govern and improve their technology through enterprise architecture, AI governance, cyber assurance and data governance work. We are building a small panel of senior associates to deliver technology and AI due diligence on company sales and investments.

The role

You will check how AI systems are built, tested and governed, for clients who are selling, buying or investing in AI companies. You will test whether the AI does what the company claims, whether it is safe to rely on, and whether the data behind it is used lawfully. Our first engagements are with AI workflow platforms in regulated sectors that combine plain code, classic machine learning and fine tuned open source language models.

You report to an Intelance engagement partner and work with our technology due diligence lead, who brings your findings into the final report.

What you will do

• Map where AI sits in a product: which steps are plain code, which use machine learning and which use language models.

• Review how open source models are fine tuned, what data they learn from, and whether client contracts allow that use.

• Test the evaluation approach: test suites, accuracy measures, human review rules and how failures are caught and fixed.

• Check controls against prompt injection, data leaking between clients and tampering with models.

• Check reliance on outside AI providers, and what data, if any, they receive.

• Rate each finding by how it could affect the deal, and write clear, practical fixes.

What you will bring

• At least five years building, evaluating or assuring machine learning or language model systems.

• Hands on experience of fine tuning open source language models and designing evaluation suites.

• Working knowledge of ISO/IEC 42001, the NIST AI Risk Management Framework or the EU AI Act.

• The ability to explain model risk in plain words to founders, buyers and lawyers.

• Experience with health, clinical or other regulated data is a strong plus.

• Experience of due diligence, audit or model validation work is a strong plus.

How the work is set up

• Remote within the UK, with occasional meetings in London.

• Around 3 to 5 days between late November 2026 and the end of January 2027, with further engagements as the practice grows.

• Freelance, or through your own limited company, at £650 to £800 a day depending on experience. Reasonable travel costs for client meetings are covered.

• A panel agreement, NDA and statement of work are signed before any client details are shared.

• You work from an Intelance mailbox and use our evidence log and report templates, so every report meets the same standard.