Ai Model Governance Specialist

ICICI Securities; Mumbai; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

ICICI Securities seeks a seasoned technical professional to oversee the risk management, validation and control lifecycle for Generative AI and Large Language Model deployments across enterprise systems. The role ensures safe, compliant, explainable and reliable AI models, notably RAG architectures, by bridging engineering with governance.

Responsibilities include evaluating LLM outputs, maintaining evaluation tools, and coordinating data, model validation, deployment and monitoring to minimize

Qualifications

• Experience with AI/ML taxonomy
• Understanding of RAG architectures and risk points in ingestion/embedding/retrieval/ grounding/generation
• Familiarity with LLM evaluation methods and metrics (hallucination, toxicity, robustness, calibration, fairness)
• Knowledge of evaluation tools (hallucination detectors, accuracy tools)
• Awareness of major cloud/AI platforms for enterprise LLM deployment (e.g., AWS Bedrock)
• Ability to design control frameworks spanning data, model validation, deployment and monitoring
• Proficiency in Python to inspect evaluation outputs

Responsibilities

• Oversee risk management, validation and control lifecycle for Generative AI and LLM deployments across enterprise systems
• Bridge technical engineering with governance to ensure AI models are safe, compliant, explainable and reliable
• Identify risk points across RAG architecture and define mitigation strategies
• Evaluate and monitor model outputs, metrics and datasets to ensure quality and safety
• Collaborate with stakeholders to implement governance, documentation and standards

Job description

We require a seasoned technical person who has experience and understanding of the AI Model who would assist us in smooth implementation of the AI models into our systems. The person will oversee the risk management, validation, and control lifecycle for Generative AI and Large Language Model (LLM) deployments across enterprise systems. This role bridges technical engineering with governance to ensure AI models specifically Retrieval-Augmented Generation (RAG) architectures are safe, compliant, explainable, and reliable.

Roles and Responsibilities:

• Working knowledge of AI/ML taxonomy
• Solid understanding of RAG architecture Knowledge Base ingestion, embedding/retrieval, grounding, and generation stages, and where risk arises at each
• Familiarity with LLM evaluation methods and metrics hallucination detection, toxicity, robustness, calibration, fairness/bias testing, LLM-as-judge approaches, and reference-answer/golden dataset construction
• Understanding of the logic behind evaluation tools like hallucination detectors, accuracy tools, etc.
• Awareness of major cloud/AI platforms used for enterprise LLM deployment (e.g. Amazon bedrock or equivalent)
• Ability to design control frameworks spanning data, model validation, deployment and monitoring
• Working knowledge of Python to independently inspect evaluation outputs