AI Ops Engineer

Berry Bros. & Rudd; Greater London; Hybrid; GBP 65,000 - 90,000; Full time

Benefits offered by this job

Private Medical cover & Health plan

Life Insurance

Pension

Employee Discount (up to 40%)

Job summary

Berry Bros. & Rudd in London offers a hybrid role to monitor AI agent performance across the estate, tuning routing and logic, and supporting internal dotnet platforms.

You will own orchestrators and monitoring agents, coordinate with AI Engineering Lead on delivery tasks, and work with internal teams including Buying, Offer Admin and Customer Contact. The ideal candidate has QA and monitoring experience, API integration expertise, and hands-on .NET/Azure knowledge.

Qualifications

• Experience in QA and output monitoring with clear exception reporting.
• Proven ability to manage escalations and recognize patterns in data.
• Hands-on AI agent development and optimisation experience.
• Strong: API and MCP integration; deep knowledge of .NET and Azure.

Responsibilities

• Owns Orchestrators and Monitoring agents (central, cross-process).
• Monitors agent performance across the estate and reports issues.
• Tunes routing and logic; optimises model costs for production.
• Defines new AI agent capabilities and interfaces to modernise processes.
• Maintains and enhances in-house dotNet apps as directed.

Skills

Quality assurance

Output monitoring

Escalation management

Pattern recognition

Analytical thinking

AI agent development

API integration

MCP integration

Microsoft .NET

Azure services

Written communication

Attention to detail

Tools

Microsoft Foundry

Azure

Job description

Hybrid working available (a minimum of 3 days working from the office, 2 days working from home)

The job in a nutshell

Monitors agent performance across the BB&R estate - reporting and flagging on performance, whilst tuning the routing and logic. Works with AI Engineering Lead on delivery tasks. Owns the Orchestrators and Monitoring agents shared across all process areas. Also provides support on internal dotnet platforms.

Who you will work with

Internal: AI Lead Engineer (reports to); process area leads and agent-owning roles across Buying, Offer Admin and Customer Contact processes whose agents this role monitors. Application Support team for 3rd party app integrations

External: Agentic platform vendors; Suppliers & SaaS Providers, 3rd party business application partners

What you will do

• Owns Orchestrators and Monitoring agents (shared/central, cross-process)
• Monitors agent performance across the estate
• Reports and flags on performance issues
• Tunes routing and logic
• Optimise agent costs by testing and validating model performance using optimised lower cost models and progressing these into production
• Define new AI agent capabilities and interfaces to replace legacy processes
• Maintain and Enhance in house dotNet applications as directed

What you will bring to the role

Experience / Achievements

• Demonstrable experience in quality assurance and output monitoring
• Proven experience in escalation management and pattern recognitionExperience of analytical thinking applied to high-volume outputs
• Experience producing clear written communication for exception reports
• Demonstrable experience in AI agent development and optimisation
• Deep experience with API and MCP integration and agentic platform system architecture
• Understanding of Microsoft dotnet development platform and Azure+ services

Your Skills, Knowledge and Behaviours

• Strong attention to detail across high-volume agent outputs
• [Desirable] Experience in Lyzr and/or Microsoft Foundry platform
• [Desirable] Dashboard and alerting tools
• [Desirable] Process improvement methodology and data visualisation

We support our colleagues with a wide range of benefits: you will start with 24 days' holiday, and 3 additional days over the Christmas period and 1 Personal Day, Private Medical cover & Health plan, Life Insurance, Pension, Employee Assistance Programme (EAP), generous Employee discount (up to 40% off products), and many more.

Closing Date: 22nd September 2026

N.B. We reserve the right to close a vacancy before the closing date in the event of an overwhelming response or a change in business priorities.