AVP/VP, AI/ML Model Validation Engineer

SUMITOMO MITSUI BANKING CORPORATION SINGAPORE BRANCH; Singapore; On-site; SGD 140,000 - 190,000; Full time

Job summary

SUMITOMO MITSUI BANKING CORPORATION SINGAPORE BRANCH is seeking an AVP/VP to lead AI/ML model validation, designing comprehensive test strategies for statistical, ML, LLM, and agentic models. You will conduct rigorous evaluations, bias checks, drift detection, and end-to-end workflow assessments in production-like environments.

The role requires strong Python, R and SQL skills, deep knowledge of ML testing pipelines, and collaboration with data science, IT and governance functions to strengthen

Qualifications

• Minimum 4+ years in model testing, QA/QC, AI/ML evaluation, CI/CD, MLOps, data engineering, or related roles.
• Proficiency in Python, R, SQL; knowledge of additional languages is a plus.
• Experience designing test strategies for ML/AI models, including automated pipelines and regression frameworks.

Responsibilities

• Define and execute comprehensive test strategies covering statistical, ML, LLM and agentic AI models.
• Perform functional, regression and scenario-based testing of model behaviours and workflows.
• Conduct AI/ML evaluations including accuracy checks, bias/fairness assessment, robustness analysis and drift detection.
• Assess end-to-end model workflows including data inputs, feature transformations, task completion, tool-use accuracy and multi-step reasoning.
• Design and maintain automated test and evaluation pipelines, including benchmarking and regression frameworks.
• Validate API and tool-integration behaviour in production-like environments, identifying dependency or orchestration issues.
• Diagnose issues using observability, logging, tracing and debugging tooling, and document findings clearly.
• Collaborate with data scientists across departments to understand modelling intent, feature logic and expected behaviours.
• Perform data-management tasks to support AI/ML model testing, including maintaining metadata, documenting key datasets and ensuring clarity of data inputs.
• Contribute to AI/ML proof-of-concept (POC) initiatives to strengthen evaluation methodologies and support innovation.
• Support data-management/analytics initiatives such as the Analytics Workbench and contribute to AI/ML/data analytics enablement.

Skills

Python

PySpark

MLlib

pytest

R

SQL

Scala

Rust

Java

JS

C++

Tools

CI/CD

MLOps

Job description

AVP/VP, AI/ML Model Validation Engineer

Responsibilities

• Define and execute comprehensive test strategies covering statistical, ML, LLM and agentic AI models.
• Perform functional, regression and scenario-based testing of model behaviours and workflows.
• Conduct AI/ML evaluations including accuracy checks, bias/fairness assessment, robustness analysis and drift detection.
• Assess end-to-end model workflows including data inputs, feature transformations, task completion, tool-use accuracy and multi-step reasoning.
• Design and maintain automated test and evaluation pipelines, including benchmarking and regression frameworks.
• Validate API and tool-integration behaviour in production-like environments, identifying dependency or orchestration issues.
• Diagnose issues using observability, logging, tracing and debugging tooling, and document findings clearly.
• Collaborate with data scientists across departments to understand modelling intent, feature logic and expected behaviours.
• Perform data-management tasks to support AI/ML model testing, including maintaining metadata, documenting key datasets and ensuring clarity of data inputs.
• Contribute to AI/ML proof-of-concept (POC) initiatives to strengthen evaluation methodologies and support innovation.
• Support data-management/analytics initiatives such as the Analytics Workbench and contribute to AI/ML/data analytics enablement.

Requirements

• Minimum 4 years of relevant experience in model testing, QA/QC, AI/ML evaluation, CI/CD, MLOps, data engineering, or related technical roles.
• Proficiency in Python (especially PySpark, MLlib, pytest), R and SQL; knowledge of Scala, Rust, Java, JS or C++ is a plus.
• Experience designing and executing test strategies for ML/AI models, including automated pipelines and regression frameworks.
• Ability to evaluate statistical, ML and LLM models using performance, bias, robustness and drift metrics.
• Strong ability to assess feature engineering logic, dataset integrity, workflow reliability and tool-integration behaviours.
• Experience troubleshooting using logs, traces and debugging tools to identify root-cause issues.
• Strong documentation and communication skills to articulate findings, risks and remediation requirements.
• Ability to collaborate effectively with data science, engineering, IT and governance functions.
• Understanding of Responsible AI concepts and quality expectations for production-ready AI/ML systems.