Quality Engineer for Agentic AI Platforms & Governance

London Stock Exchange; Greater London; On-site; GBP 60,000 - 90,000; Full time

Job summary

London Stock Exchange’s CE AI team is responsible for enabling scalable, safe AI across the enterprise, providing shared platforms, governance, and delivery support. The Quality Engineer will define evaluation frameworks, build automated test pipelines, and mitigate risks in MCP tools and Skills, collaborating with ML engineers to ensure governance and security compliance.

This role supports regulated, enterprise-grade AI platforms within LSEG’s internal ecosystem.

Qualifications

• Strong Python experience for test harnesses and automation.
• Experience with LLM and RAG evaluation frameworks or custom evaluation pipelines.
• Test automation expertise covering unit, integration, and regression testing.
• Understanding of agentic system risks and failure modes.
• Ability to assess solutions against governance, security, and audit expectations.
• Experience working in regulated or highly governed engineering environments.

Responsibilities

• Define and implement evaluation frameworks for MCP tools and Skills covering correctness, safety, and regression impact.
• Build and maintain automated test pipelines for agentic behaviours and multi-step workflows.
• Evaluate and mitigate agentic failure modes such as hallucination, tool misuse, and latency amplification.
• Produce testing evidence required for PTB and PTO.
• Partner with ML Engineers to embed testability and evaluation hooks into MCP servers and Skills.
• Help define the long-term quality and governance model for federated MCP contributions across LSEG.

Skills

Strong Python

LLM evaluation frameworks

Test automation

Agentic system risks

Governance & security

Regulated engineering experience

Job description

London Stock Exchange’s CE AI team is responsible for enabling scalable, safe AI across the enterprise, providing shared platforms, governance, and delivery support. The Quality Engineer will define evaluation frameworks, build automated test pipelines, and mitigate risks in MCP tools and Skills, collaborating with ML engineers to ensure governance and security compliance.

This role supports regulated, enterprise-grade AI platforms within LSEG’s internal ecosystem.