Responsible AI / Eval Specialist

McCain Foods; Gurugram District; Hybrid; INR 4,000,000 - 6,800,000; Full time

Job summary

McCain Foods India invites a Responsible AI / Eval Specialist to design, implement, and operate evaluation frameworks that decide when AI use cases are ready for production. You will collaborate with AI Platform & Engineering, Knowledge Engineering, Forward Deployment Engineers, and Security to assess accuracy, fairness, robustness, and governance across the AI ecosystem.

You will lead bias testing, red‑team exercises, and risk assessments, and help embed evaluation capabilities into development

Qualifications

• Bachelor’s degree in Computer Science, Statistics, Engineering, or a related field; advanced degree preferred.
• 10+ years of experience in AI/ML evaluation, model validation, model risk management, or a related discipline.
• Hands-on experience with AI/ML evaluation methodologies and performance metrics, including accuracy, calibration, fairness, and robustness.
• Experience evaluating Large Language Models (LLMs) and AI agents using frameworks such as HELM, Eleuther LM Evaluation Harness, or equivalent platforms.
• Strong Python programming skills and experience with common machine learning evaluation and testing tools.
• Experience working with Databricks or comparable enterprise data platforms.
• Familiarity with Responsible AI and AI governance frameworks, including NIST AI RMF, EU AI Act, ISO/IEC 42001, or similar standards.
• Strong analytical, problem-solving, and critical-thinking skills.
• Ability to communicate complex technical findings effectively to both technical and non-technical stakeholders.
• Experience conducting bias testing, red-team exercises, adversarial testing, and AI risk assessments is highly desirable.

Responsibilities

• Design and implement evaluation strategies for AI use cases, including assessments of accuracy, reliability, hallucination rates, fairness, and robustness.
• Build, maintain, and operate evaluation harnesses, benchmark suites, and golden datasets across the McCain AI technology stack.
• Conduct stage-gate evaluations and provide clear, actionable findings to support governance decisions.
• Run bias and fairness testing across protected attributes and business-relevant user groups.
• Identify, quantify, and communicate fairness concerns while recommending practical remediation approaches.
• Lead red-team and adversarial testing exercises to identify vulnerabilities, failure modes, prompt injection risks, and jailbreak scenarios.
• Partner with Enterprise Security teams to strengthen AI resilience through broader adversarial testing initiatives.
• Support risk classification activities by providing evaluation evidence and recommendations that inform governance decisions.
• Collaborate with AI Platform & Engineering and Knowledge Engineering teams to embed evaluation capabilities into AI development workflows.
• Maintain model cards, evaluation reports, decision logs, and other governance documentation while contributing to Responsible AI standards and playbooks.
• Coach AI Engineers and Forward Deployment Engineers on evaluation best practices and quality disciplines.
• Share evaluation insights, frameworks, and reusable patterns across the AI organization.

Skills

AI/ML evaluation

Python programming

LLMs evaluation

Governance frameworks

Analytical thinking

Communication skills

Bias/adversarial testing

Education

Bachelor's degree in CS/Stats/Engineering

Advanced degree preferred

Tools

HELM

EleutherLM

Databricks

Azure

Anthropic

SAP

Salesforce

Job description

Position Title: Responsible AI / Eval Specialist

Position Type: Regular - Full-Time

In every role, McCainers are ambitious, curious, and passionate about creating exceptional work experiences - together. With a customer-first mindset, we make doing business with McCain easy.

About The Role.

The Responsible-AI / Eval Specialist is responsible for designing, implementing, and operating evaluation frameworks that determine whether AI use cases are ready to progress through governance stage-gates and into production. This role serves as the hands-on execution arm of McCain’s Responsible AI standards, working closely with AI Platform & Engineering, Knowledge Engineering, Forward Deployment Engineers, and Security teams to evaluate accuracy, reliability, fairness, robustness, and overall quality across the McCain AI ecosystem. The role supports AI solutions built on Azure, Anthropic foundation models, Distyl technologies, Databricks, SAP, Salesforce, and other enterprise platforms.

What You’ll Be Doing.

• Design and execute evaluation strategies for AI use cases, including assessments of accuracy, reliability, hallucination rates, fairness, and robustness.
• Build, maintain, and operate evaluation harnesses, benchmark suites, and golden datasets across the McCain AI technology stack.
• Conduct stage-gate evaluations and provide clear, actionable findings to support governance decisions.
• Run bias and fairness testing across protected attributes and business-relevant user groups.
• Identify, quantify, and communicate fairness concerns while recommending practical remediation approaches.
• Lead red-team and adversarial testing exercises to identify vulnerabilities, failure modes, prompt injection risks, and jailbreak scenarios.
• Partner with Enterprise Security teams to strengthen AI resilience through broader adversarial testing initiatives.
• Support risk classification activities by providing evaluation evidence and recommendations that inform governance decisions.
• Collaborate with AI Platform & Engineering and Knowledge Engineering teams to embed evaluation capabilities into AI development workflows.
• Maintain model cards, evaluation reports, decision logs, and other governance documentation while contributing to Responsible AI standards and playbooks.
• Coach AI Engineers and Forward Deployment Engineers on evaluation best practices and quality disciplines.
• Share evaluation insights, frameworks, and reusable patterns across the AI organization.

What You’ll Need To Be Successful.

• Bachelor’s degree in Computer Science, Statistics, Engineering, or a related field; advanced degree preferred.
• 10+ years of experience in AI/ML evaluation, model validation, model risk management, or a related discipline.
• Hands-on experience with AI/ML evaluation methodologies and performance metrics, including accuracy, calibration, fairness, and robustness.
• Experience evaluating Large Language Models (LLMs) and AI agents using frameworks such as HELM, Eleuther LM Evaluation Harness, or equivalent platforms.
• Strong Python programming skills and experience with common machine learning evaluation and testing tools.
• Experience working with Databricks or comparable enterprise data platforms.
• Familiarity with Responsible AI and AI governance frameworks, including NIST AI RMF, EU AI Act, ISO/IEC 42001, or similar standards.
• Strong analytical, problem-solving, and critical-thinking skills.
• Ability to communicate complex technical findings effectively to both technical and non-technical stakeholders.
• Experience conducting bias testing, red-team exercises, adversarial testing, and AI risk assessments is highly desirable.

About The Team.

The Responsible-AI / Eval Specialist is part of McCain’s AI governance and delivery ecosystem and works closely with the Use Case Governance & Operations Lead. The role partners across AI Platform & Engineering, Knowledge Engineering, Forward Deployment Engineering, Enterprise Security, and business stakeholders to ensure AI use cases meet quality, risk, and Responsible AI standards before deployment. This position plays a critical role in advancing safe, reliable, and trustworthy AI solutions across the organization.

About McCain.

Leadership principles.

At McCain, our leadership principles guide how we engage with customers, collaborate as a team, and achieve success. We focus on understanding customer needs, driving innovation, empowering people, and taking ownership to clear obstacles and deliver results.

AI & Digital Capabilities.

At McCain, AI and digital capabilities are part of how we work and innovate. We value curiosity toward emerging technologies and the thoughtful, responsible use of AI-enabled tools to improve outcomes as part of our future-ready culture.

The McCain Experience.

We are McCain. This statement is a testament to our collective strength and our individual value. Your contributions play a vital role in our success. Our winning culture is rooted in authenticity and trust, empowering us to bring out the best in one another. Join us and experience why we’re better together.

Our purpose is grounded in building meaningful relationships. We’re big believers in the power of working together in person—it helps us stay connected, collaborate more effectively, and grow as a team. At the same time, we recognize the importance of flexibility. Most office-based roles follow a hybrid model, with the option to work remotely two days a week. There may be exceptions depending on the role and location, so we encourage you to speak with your recruiter for more details.

McCain Foods is an equal opportunity employer. As a global family-owned company, we strive to be the employer of choice in the diverse communities around the world in which we live and work. We recognize that inclusion drives our creativity, resilience, and success and makes our business stronger. All qualified applicants will receive consideration for employment without regard to race, religion, color, national origin, sex, age, veteran status, disability, or any other protected characteristic under applicable law.

McCain is an accessible employer. If you require an accommodation throughout the recruitment process (including alternate formats of materials or accessible meeting rooms), please let us know and we will work with you to find appropriate solutions.

Your privacy is important to us. By submitting personal data or information to us, you agree this will be handled in accordance with McCain’s Global Privacy Policy and Global Employee Privacy Policy, as applicable. McCain leverages AI in the hiring process, though all final decisions are made by humans. You can understand our approach to AI and how your personal information is being handled here.

Job Family: Information Technology

Location(s): IN - India : Haryana : Gurgaon

Company: McCain Foods(India) P Ltd