Responsible AI Eval Specialist

McCain Foods; Gurugram District; Hybrid; INR 3,500,000 - 6,000,000; Full time

Job summary

McCain Foods in Gurgaon, Haryana, India seeks a Responsible AI / Eval Specialist to design, implement, and operate evaluation frameworks for AI governance. You will assess accuracy, fairness, reliability, and robustness across the McCain AI stack, collaborating with security and engineering teams.

You will guide stage-gate decisions, maintain governance artifacts, and coach engineers on best practices in evaluation and risk management, with hybrid work options in a global, innovation-driven

Qualifications

• 10+ years of experience in AI/ML evaluation, model validation, or risk management.
• Hands-on experience with AI/ML evaluation methodologies and performance metrics.
• Experience evaluating LLMs and AI agents using HELM or Eleuther LM Evaluation Harness.
• Strong Python programming skills and ML evaluation/testing tooling familiarity.
• Experience with Databricks or comparable enterprise data platforms.
• Familiarity with Responsible AI and governance frameworks (NIST AI RMF, EU AI Act, ISO/IEC 42001).
• Strong analytical and communication skills to convey technical findings.

Responsibilities

• Design and execute evaluation strategies for AI use cases including accuracy and fairness.
• Build and operate evaluation harnesses, benchmarks, and golden datasets.
• Conduct stage-gate evaluations and provide actionable governance findings.
• Run bias and fairness testing across protected attributes and user groups.
• Lead red-team and adversarial testing to identify vulnerabilities and risks.
• Collaborate with Security to strengthen AI resilience and risk assessments.
• Embed evaluation capabilities into AI development workflows with platform teams.
• Maintain governance artifacts like model cards and evaluation reports.

Skills

AI/ML evaluation

LLMs evaluation

Python

Databricks

Responsible AI

Communication

Bias testing

Adversarial testing

Education

Bachelor’s degree in CS/Statistics/Engineering

Job description

Job Description:

Position Title

Responsible AI / Eval Specialist

Position Type

Regular - Full-Time

Requisition ID

43986

In every role, McCainers are ambitious, curious, and passionate about creating exceptional work experiences - together. With a customer-first mindset, we make doing business with McCain easy.

About The Role

The Responsible-AI / Eval Specialist is responsible for designing, implementing, and operating evaluation frameworks that determine whether AI use cases are ready to progress through governance stage-gates and into production. This role serves as the hands‑on execution arm of McCain’s Responsible AI standards, working closely with AI Platform & Engineering, Knowledge Engineering, Forward Deployment Engineers, and Security teams to evaluate accuracy, reliability, fairness, robustness, and overall quality across the McCain AI ecosystem. The role supports AI solutions built on Azure, Anthropic foundation models, Distyl technologies, Databricks, SAP, Salesforce, and other enterprise platforms.

What You’ll Be Doing

• Design and execute evaluation strategies for AI use cases, including assessments of accuracy, reliability, hallucination rates, fairness, and robustness.
• Build, maintain, and operate evaluation harnesses, benchmark suites, and golden datasets across the McCain AI technology stack.
• Conduct stage‑gate evaluations and provide clear, actionable findings to support governance decisions.
• Run bias and fairness testing across protected attributes and business‑relevant user groups.
• Identify, quantify, and communicate fairness concerns while recommending practical remediation approaches.
• Lead red‑team and adversarial testing exercises to identify vulnerabilities, failure modes, prompt injection risks, and jailbreak scenarios.
• Partner with Enterprise Security teams to strengthen AI resilience through broader adversarial testing initiatives.
• Support risk classification activities by providing evaluation evidence and recommendations that inform governance decisions.
• Collaborate with AI Platform & Engineering and Knowledge Engineering teams to embed evaluation capabilities into AI development workflows.
• Maintain model cards, evaluation reports, decision logs, and other governance documentation while contributing to Responsible AI standards and playbooks.
• Coach AI Engineers and Forward Deployment Engineers on evaluation best practices and quality disciplines.
• Share evaluation insights, frameworks, and reusable patterns across the AI organization.

What You’ll Need To Be Successful

• Bachelor’s degree in Computer Science, Statistics, Engineering, or a related field; advanced degree preferred.
• 10+ years of experience in AI/ML evaluation, model validation, model risk management, or a related discipline.
• Hands‑on experience with AI/ML evaluation methodologies and performance metrics, including accuracy, calibration, fairness, and robustness.
• Experience evaluating Large Language Models (LLMs) and AI agents using frameworks such as HELM, Eleuther LM Evaluation Harness, or equivalent platforms.
• Strong Python programming skills and experience with common machine learning evaluation and testing tools.
• Experience working with Databricks or comparable enterprise data platforms.
• Familiarity with Responsible AI and AI governance frameworks, including NIST AI RMF, EU AI Act, ISO/IEC 42001, or similar standards.
• Strong analytical, problem‑solving, and critical‑thinking skills.
• Ability to communicate complex technical findings effectively to both technical and non‑technical stakeholders.
• Experience conducting bias testing, red‑team exercises, adversarial testing, and AI risk assessments is highly desirable.

About The Team

The Responsible‑AI / Eval Specialist is part of McCain’s AI governance and delivery ecosystem and works closely with the Use Case Governance & Operations Lead. The role partners across AI Platform & Engineering, Knowledge Engineering, Forward Deployment Engineering, Enterprise Security, and business stakeholders to ensure AI use cases meet quality, risk, and Responsible AI standards before deployment. This position plays a critical role in advancing safe, reliable, and trustworthy AI solutions across the organization.

About McCain

Click Here to learn more about McCain and how we provide opportunities to make an impact that matters.

Leadership principles

At McCain, our leadership principles guide how we engage with customers, collaborate as a team, and achieve success. We focus on understanding customer needs, driving innovation, empowering people, and taking ownership to clear obstacles and deliver results.

AI & Digital Capabilities

At McCain, AI and digital capabilities are part of how we work and innovate. We value curiosity toward emerging technologies and the thoughtful, responsible use of AI‑enabled tools to improve outcomes as part of our future‑ready culture.

The McCain Experience

We are McCain. This statement is a testament to our collective strength and our individual value. Your contributions play a vital role in our success. Our winning culture is rooted in authenticity and trust, empowering us to bring out the best in one another. Here, you’ll find opportunities to learn, grow, and thrive. Join us and experience why we’re better together.

Our purpose is grounded in building meaningful relationships. We’re big believers in the power of working together in person—it helps us stay connected, collaborate more effectively, and grow as a team. At the same time, we recognize the importance of flexibility. Most office‑based roles follow a hybrid model, with the option to work remotely two days a week. There may be exceptions depending on the role and location, so we encourage you to speak with your recruiter for more details.

McCain Foods is an equal opportunity employer. As a global family‑owned company, we strive to be the employer of choice in the diverse communities around the world in which we live and work. We recognize that inclusion drives our creativity, resilience, and success and makes our business stronger. All qualified applicants will receive consideration for employment without regard to race, religion, color, national origin, sex, age, veteran status, disability, or any other protected characteristic under applicable law.

McCain is an accessible employer. If you require an accommodation throughout the recruitment process (including alternate formats of materials or accessible meeting rooms), please let us know and we will work with you to find appropriate solutions.

Your privacy is important to us. By submitting personal data or information to us, you agree this will be handled in accordance with McCain’s Global Privacy Policy and Global Employee Privacy Policy, as applicable. McCain leverages AI in the hiring process, though all final decisions are made by humans. You can understand our approach to AI and how your personal information is being handled here.

Job Family

Information Technology

Location(s)

IN - India : Haryana : Gurgaon

Company

McCain Foods(India) P Ltd