Model Validation Senior Manager

Monzo; Cardiff, Greater London; Hybrid; GBP 85,000 - 105,000; Full time

Job summary

Monzo is seeking a Model Validation Senior Manager - Gen AI in London or remote within the UK. You will lead oversight of Generative AI models across operations, ensuring safe and customer-friendly implementations.

Key tasks include embedding the risk framework, coordinating with 1LoD, and developing validation playbooks and reports for governance bodies. A strong ML background and Python expertise are essential.

Qualifications

• Strong Data Science or ML background with Python proficiency.
• Experience designing Generative AI evaluation frameworks.
• Experience managing modelling projects through full lifecycle.

Responsibilities

• Embed the Model Risk Framework for Generative AI use cases across Monzo.
• Lead collaboration with 1LoD teams to strengthen policies for Gen AI models and systems.
• Embed best practices of Generative AI across the business.
• Develop and maintain validation methods, evidence, and reusable assets for Gen AI.
• Produce clear validation reports and present findings to relevant committees.

Skills

Python

Generative AI

Model risk

Job description

We’re on a mission to make money work for everyone. We’re waving goodbye to the complicated and confusing ways of traditional banking. After starting as a prepaid card, our product offering has grown a lot in the last 10 years in the UK. As well as personal and business bank accounts, we offer joint accounts, accounts for 16-17 year olds, a free kids account and credit cards in the UK, with more exciting things to come beyond. Our UK customers can also save, invest and combine their pensions with us. With our hot coral cards and get-paid-early feature, combined with financial education on social media and our award winning customer service, we have a long history of creating magical moments for our customers! We’re not about selling products - we want to solve problems and change lives through Monzo.

London or Remote (UK)

£85,000 - £105,000 + Incentive Awards tied to your performance + Benefits

Model Validation Senior Manager - Gen AI

The second line of defence Model Oversight team (that’s us!) provides support to our colleagues in Borrowing, Fincrime & Fraud, Finance and Operations to help Monzo grow safely. We’re looking for a Model Validation Senior Manager - Gen AI to lead the teams’ oversight of Generative AI models/use cases (primarily built across Monzo’s Operations squads) to provide assurance that the models/use cases we use are not only good for Monzo but what's best for our customers.

Key Responsibilities

You’ll play a key role by:

• Driving the embedding of the Model Risk Framework for Generative AI use cases across Monzo
• Lead the collaboration with the first line of defence teams to strengthen policies, standards and procedures for Generative AI model, use case, system design and build, implementation, governance, monitoring and use.
• Helping to embed best practice of Generative AI across the business.
• Own the setting and maintenance of Generative AI validation methods, evidence expectations and reusable technical assets with relevant 1LoD modelling teams, applicable across all model risk areas.
• Convert modelling/validation experience into controlled playbooks, tests and working examples that can be applied to 2LoD Validation standards and processes for all Generative AI models, use cases, systems overseen by the 2LoD Model Oversight team.
• Coach Generative AI validation methodologies, to 2LoD Model Oversight colleagues, whom already have transferable Machine Learning skills, to reduce reliance on any single specialist for the validation of Generative AI models, use cases, systems so it can be consistently applied across Financial, Credit and Fincrime/Fraud models.
• Leading the 2LoD Oversight of Operational Model Risk Perform, and oversee, independent model validations of new Operational models (primarily Generative AI models, use cases, systems), and changes to existing, using proportionate modelling techniques depending on the level of complexity applied. This includes writing clear and impactful model validation reports and presenting the findings in relevant committees.
• Build relationships with first line of defence modelling teams that develop Operational Models, and wider risk teams, to support the embedding of the model risk framework to help Monzo grow safely.
• Review ongoing model performance monitoring to ensure Operational models are performing within risk thresholds and providing support to the relevant model development team when further action is required Monitoring and measuring the overall level of Operational model risk that Monzo operates, and driving action plans with first line teams and appropriate governance forums to mitigate residual model risk within desired risk appetite.
• Being a senior advocate of Model Risk Management best practices Support the effective operation of the Model Risk Oversight Committee as a regular attendee, contributing to the group discussion to improve model risk management.
• Promote and help embed a strong risk culture, making risk meaningful and relevant to everyone at Monzo. You will share best practice that is easy for everyone to understand.

Desired Candidate Attributes

• You are highly motivated and energised by the idea of bringing positive impact to our customers and Monzo
• You are self-motivated and thrive in a fast-paced environment
• You have great attention to detail while keeping an eye on the big picture
• You are a great communicator, able to articulate complex problems
• You can provide effective challenge whilst building strong, respectful relationships
• You are keen to grow your knowledge in both business and technology

And, from a technical and knowledge perspective, you should have:

Technical and Knowledge Requirements

• Robust Data Science or Machine Learning background with strong Python capability
• Experience with design of Generative AI evaluation frameworks, architectures, methods, practical tooling and platform alignment.
• Knowledge of Gemini implementations and change assessment
• Experience of managing a modelling project through its full lifecycle
• If possible, some understanding of some of the…