Risk Models Testing Lead – AI/ML (Banking)

Vriti; India; Remote; INR 2,800,000 - 4,500,000; Full time

Job summary

Vriti is seeking an experienced AI/ML Testing & Validation Lead to design and execute comprehensive testing strategies for banking AI/ML models. The role covers risk models, IFRS 9, Basel II/III, AML, and fraud detection, with focus on explainability, data validation, and governance.

The candidate will supervise testing across credit risk, underwriting, pricing, and customer segmentation, evaluate prompts for GenAI, and ensure compliance with regulatory expectations.

Qualifications

• Minimum 10+ years in Banking Technology, Risk, or Digital Transformation.
• Experience testing or validating AI/ML models with Data Scientists and Model Risk teams.
• Strong understanding of model governance and regulatory expectations.

Responsibilities

• Lead AI/ML testing, validation, and performance assessment across datasets.
• Validate models against business/regulatory requirements and risk constraints.
• Evaluate model outputs with metrics like accuracy, ROC-AUC, and bias checks.
• Test GenAI/LMM outputs, prompt strategies, and RAG pipelines for banking use cases.
• Ensure traceable documentation and governance compliance for model risk management.

Skills

AI/ML testing

Banking risk domain

Prompt engineering

GenAI testing

RAG validation

XAI explainability

Bias & fairness testing

Data validation

SQL

Python

Jupyter Notebook

Git

Azure DevOps/Jira

REST APIs

Snowflake

Databricks

MLflow

Selenium

Postman

PyTest

Great Expectations

Education

Bachelor’s degree in CS/Engineering/Math/Finance

Master’s degree an advantage

Certifications in AI/ML or Banking

Tools

SQL

Python

Jupyter Notebook

Git

Azure DevOps/Jira

REST APIs

Snowflake

Databricks

MLflow

Selenium

Postman

PyTest

Great Expectations

Robot Framework

Job description

Key Responsibilities

AI/ML Model Testing & ValidationDesign and execute comprehensive testing strategies for AI and Machine Learning models.Validate model performance across training, validation, and production datasets.Perform functional, non-functional, regression, and end-to-end testing of AI-driven banking solutions.Validate model outputs against business expectations and regulatory requirements.Assess model accuracy, precision, recall, F1 score, ROC-AUC, and other relevant performance metrics.Test model robustness under edge cases, stress scenarios, and adverse conditions.Banking Risk ModelsLead testing and validation of models supporting:Credit RiskProbability of Default (PD)Loss Given Default (LGD)Exposure at Default (EAD)IFRS 9 Expected Credit Loss (ECL)Early Warning SystemsCovenant MonitoringFraud DetectionAML Transaction MonitoringCollections PrioritizationCustomer Risk RatingCredit DecisioningCustomer SegmentationPricing and Limit ManagementGenAI & LLM TestingValidate Large Language Model (LLM) outputs for banking use cases.Test prompt engineering strategies and dynamic prompting frameworks.Evaluate hallucinations, bias, consistency, explainability, and confidence scoring.Perform retrieval validation for Retrieval-Augmented Generation (RAG) applications.Test AI copilots supporting credit underwriting, policy interpretation, and customer servicing.Model GovernanceSupport Model Risk Management (MRM) lifecycle.Ensure compliance with model governance frameworks and validation standards.Validate model documentation, assumptions, limitations, and approval criteria.Ensure adherence to regulatory expectations from central banks and internal governance teams.Data ValidationValidate data quality, completeness, lineage, and feature engineering.Test data pipelines feeding AI models.Verify data transformations and feature calculations.Identify data drift, concept drift, and model performance degradation.Automation & Quality EngineeringDevelop automated testing frameworks for AI models.Integrate AI testing into CI/CD pipelines.Maintain reusable test cases and regression suites.Produce detailed defect reports and testing dashboards.Stakeholder ManagementWork with Risk, Business, Data Science, Compliance, Audit, Technology, and Product teams.Present testing findings and model readiness assessments to governance forums.Support User Acceptance Testing (UAT) and production readiness reviews.Required SkillsBanking DomainCorporate BankingRetail BankingCredit RiskBasel II / Basel IIIIFRS 9Regulatory ReportingLending LifecycleCredit UnderwritingRisk Rating ModelsFraud & AMLAI/MLMachine Learning lifecycleModel ValidationExplainable AI (XAI)Responsible AIPrompt EngineeringGenAI TestingLLM EvaluationRAG ValidationHallucination DetectionConfidence Score ValidationBias & Fairness TestingTechnical SkillsSQLPythonJupyter NotebookGitAzure DevOps / JiraREST APIsSnowflakeDatabricks (preferred)MLflow (preferred)Testing ToolsSelenium (optional)PostmanPyTestGreat ExpectationsRobot FrameworkJMeterAPI TestingData Validation ToolsExperience Required10+ years in Banking Technology, Risk, or Digital Transformation.Minimum 5 years of experience testing or validating AI/ML models.Experience working with Data Scientists and Model Risk teams.Strong understanding of model governance and regulatory expectations.Experience in Agile and DevSecOps environments.Preferred QualificationsBachelor’s degree in Computer Science, Engineering, Mathematics, Statistics, or Finance.Master’s degree is an advantage.Certifications in AI/ML, Data Science, Risk Management, or Banking.ISTQB Certification (preferred).

Key Competencies

Strong analytical and critical thinking skills.Excellent stakeholder management and communication.Ability to translate complex AI concepts into business outcomes.High attention to detail and risk awareness.Strong problem-solving and decision-making capabilities.Leadership experience managing cross-functional teams and complex banking transformation initiatives