LLM Research & Validation Specialist

Abu Dhabi Islamic Bank; Abu Dhabi; On-site; AED 300,000 - 600,000; Full time

Job summary

Abu Dhabi Islamic Bank is seeking an LLM Research & Validation Specialist in Abu Dhabi to lead validation of large language models, multimodal systems, and retrieval-augmented setups, building validation toolchains and governance-ready evidence.

You will design evaluation harnesses, assess model behaviour, and communicate limitations to technical teams and governance forums. Strong Python, ML theory, and statistics are required, with banking domain interest a plus.

Qualifications

• Master's degree in Theoretical Physics, Applied Physics, Mathematics, or related quantitative field is required; PhD strongly preferred.
• 1–3 years of AI research, ML, model validation or related field; exceptional profiles considered.
• Strong understanding of probability, statistics, linear algebra, optimisation and uncertainty quantification.
• Deep knowledge of transformers, LLMs, embeddings, RAG, alignment and AI safety failure modes.
• Advanced Python with PyTorch, Hugging Face, evaluation frameworks, SQL, Git and cloud AI platforms.
• Ability to reproduce methods from papers and convert findings into bank-grade validation evidence.

Responsibilities

• Lead independent validation of LLMs, multimodal, RAG and agentic AI use cases across design to monitoring.
• Assess transformer components such as tokenisation, embeddings, attention and context-window behaviour.
• Design reproducible evaluation harnesses, golden datasets and statistical acceptance criteria.
• Evaluate accuracy, hallucination, calibration, robustness and grounding; assess uncertainty.
• Perform deep testing for prompt injection, data leakage and tool-use safety.
• Build Python-based validation tooling, automated pipelines and experimentation tracking.
• Prepare validation reports and management briefings; communicate residual uncertainty clearly.
• Mentor junior validators and improve team methodology within Model Risk.

Skills

Python proficiency

Statistics knowledge

ML research

Adversarial testing

Model validation

Communication skills

Education

Master's degree in a quantitative field

PhD preferred

Tools

PyTorch

Hugging Face

SQL

Git

Experiment tracking

Cloud platforms

Job description

JOB DESCRIPTION

Job title: LLM Research & Validation Specialist

Location: Abu Dhabi, UAE

Role purpose:

• Lead frontier research, quantitative evaluation and independent validation of large language models, multimodal models, retrieval-augmented generation systems and agentic AI used or proposed by ADIB. Translate mathematical and scientific methods into reproducible validation tests, challenger analyses, runtime controls and decision-useful evidence for model governance.
• The role combines deep technical research with second-line effective challenge.
• It is expected to build validation toolkits and evaluation harnesses, independently assess conceptual soundness and production behaviour, and communicate material limitations clearly to technical teams, senior management and governance forums.
• The role does not own model development or production approval.

Key accountabilities /responsibilities:

• Lead independent validation of LLM, multimodal, RAG and agentic AI use cases across design, implementation, deployment and ongoing monitoring.
• Assess transformer architecture, tokenisation, embeddings, attention, context-window behaviour, decoding, fine-tuning, alignment, quantisation and inference configuration.
• Design reproducible evaluation harnesses, golden datasets, adversarial suites, counterfactual tests, canary sets and statistically defensible acceptance criteria.
• Evaluate task performance, hallucination and factuality, calibration, robustness, stability, long-context behaviour, retrieval quality, grounding, citation faithfulness and uncertainty.
• Perform deep testing of prompt injection, indirect injection, data leakage, tool-use safety, excessive agency, multi-step failure propagation, kill-switches and human oversight.
• Apply probability, statistics, optimisation, information theory, numerical methods and experimental design to develop challenger tests and quantify uncertainty.
• Review data provenance, representativeness, contamination, benchmark validity, leakage, drift and limitations of synthetic or LLM-generated evaluation data.
• Build and maintain reusable Python-based validation tooling, automated test pipelines, experiment tracking, results repositories and technical documentation.
• Conduct structured research on emerging model architectures, interpretability, mechanistic analysis, scalable oversight, model evaluation and AI safety methods.
• Independently challenge model owners, vendors and developers, document findings, propose risk-based restrictions and track remediation without assuming first-line ownership.
• Prepare validation reports, research notes, standards, committee papers and senior-management briefings that clearly distinguish evidence, judgement and residual uncertainty.
• Mentor junior validators, improve team methodology and support knowledge transfer across Model Risk

Education and experience:

• Master's degree in Theoretical Physics, Applied Physics, Mathematics, Applied Mathematics or a closely related quantitative discipline is required. A PhD or research-intensive master's is strongly preferred.
• Typically, one to three years of relevant experience in AI research, machine learning, quantitative modelling, model validation, scientific computing or a closely related field. Exceptional research profiles may be considered based on demonstrated capability.
• Deep understanding of probability, statistics, linear algebra, optimisation, numerical computation, experimental design and uncertainty quantification.
• Strong understanding of transformers, LLM training and inference, embeddings, RAG, fine-tuning, alignment, evaluation, agentic systems and AI safety failure modes.
• Advanced Python proficiency and experience with scientific and ML libraries. Exposure to PyTorch, Hugging Face, evaluation frameworks, experiment tracking, SQL, Git and cloud AI platforms is expected.
• Ability to read research papers critically, reproduce methods, design-controlled experiments and convert findings into bank-grade validation evidence.
• Experience with red teaming, adversarial testing, interpretability, calibration, robustness, privacy, security or model risk management is strongly advantageous.
• Excellent technical writing and communication, including the ability to explain mathematical concepts, assumptions and limitations to non-specialist stakeholders.
• Banking experience is advantageous but not mandatory. The role requires willingness to develop knowledge of financial services, Islamic banking, CBUAE expectations and ADIB governance.

Indicative success measures:

• Validation conclusions are reproducible, evidence-based and proportionate to use-case risk.
• Reusable evaluation assets and automation measurably improve validation coverage, consistency and efficiency.
• Material LLM and agentic risks are identified early, clearly communicated and translated into actionable controls or use restrictions.
• Research outputs strengthen ADIB validation methodology and remain traceable to tested evidence rather than unsupported claims.
• Stakeholders receive constructive, independent challenges while second-line ownership and decision rights remain clear.