AI Safety & Assurance Lead – Hands-On ML Engineer

Fruition IT; Greater London; Hybrid; GBP 100.000 - 120.000; Vollzeit

Zusammenfassung

Fruition IT in London offers a hybrid, permanent AI Safety & Assurance Lead role. It combines hands-on machine learning engineering with safety, model evaluation, and assurance across regulated AI products.

The role emphasizes testing models in real-world environments, building end-to-end evaluation frameworks, and contributing to broader assurance programs. The position focuses on hands-on technical work (about 75%), with collaboration across engineering, product and data science teams.

Qualifikationen

• Production Python is used for building scalable ML systems.
• Hands-on experience with LLMs and modern ML frameworks.

Aufgaben

• Design evaluation frameworks and test harnesses for AI models.
• Evaluate and benchmark LLMs against real-world use cases.
• Measure precision, recall, grounding, calibration and hallucination.
• Red team systems for prompt injection, jailbreaks and data leakage.
• Stress-test models with adversarial and out-of-distribution data.
• Test agent guardrails, permissions and containment.
• Build automated evaluation into CI/CD pipelines.
• Monitor deployed models for drift and regression.
• Support fine-tuning and model adaptation.
• Produce technical assurance evidence and present findings to stakeholders.

Kenntnisse

Production Python

PyTorch

LLM experience

RAG

Vector search

Prompting

Agent orchestration

CI/CD

Cloud deployment

Containers

Statistics

Adversarial ML

AI safety

Tools

PyTorch

Hugging Face Transformers

Jobbeschreibung

Fruition IT in London offers a hybrid, permanent AI Safety & Assurance Lead role. It combines hands-on machine learning engineering with safety, model evaluation, and assurance across regulated AI products.

The role emphasizes testing models in real-world environments, building end-to-end evaluation frameworks, and contributing to broader assurance programs. The position focuses on hands-on technical work (about 75%), with collaboration across engineering, product and data science teams.