AI Evaluation Lead

QualityKiosk Technologies; Bengaluru; On-site; INR 3,000,000 - 4,200,000; Full time

Job summary

QualityKiosk Technologies is seeking an AI Evaluation Lead / Principal Consultant to own the strategy, methodology, and growth of the AI Evaluation practice in Bengaluru, India. This senior role focuses on scaling AI quality capabilities, building flagship accounts, and shaping enterprise AI evaluation programs.

You will define commercial models, lead strategic initiatives, and guide leadership teams across LLMs, safety evaluation, and compliance-focused AI assurance.

Qualifications

• Expert-level understanding of AI evaluation and enterprise AI assurance.
• Knowledge of hallucination evaluation, robustness testing, safety testing, red teaming, responsible AI and compliance-oriented evaluation.
• Experience in enterprise AI delivery models, AI lifecycle risks, governance, and quality assurance.

Responsibilities

• Define and own the AI Evaluation practice strategy, operating model, methodology direction, and quality standards.
• Build and scale enterprise AI evaluation offerings across voice AI, chatbots, LLM applications, agentic AI, and multimodal AI.
• Own flagship client relationships and engage senior executives on AI quality, AI risk, and AI assurance.
• Shape high-value AI evaluation programs for enterprise clients.
• Lead solutioning, proposals, presales discussions, and strategic account expansion.

Skills

AI evaluation

LLM evaluation

AI quality

Conversational AI

Enterprise AI assurance

Education

BE / BTech / MTech / MS in AI/CS/CE/IT

Tools

AI testing platforms

AI evaluation platforms

Job description

JOB DESCRIPTION – AI Evaluation Lead / Principal Consultant

Role Summary

We are looking for an AI Evaluation Lead / Principal Consultant to own the strategy, methodology, commercial growth, and leadership direction of the AI Evaluation practice. This is a senior leadership role focused on scaling AI quality capabilities, building flagship accounts, shaping enterprise AI evaluation programs, and establishing thought leadership in AI assurance.

Experience Required

12+ years of total experience. Candidates should have 5+ years of experience in AI, ML, NLP, LLMs, AI quality, enterprise AI delivery, AI evaluation, or AI assurance, with 3+ years in team leadership, practice leadership, or senior client advisory roles.

Key Responsibilities

• Define and own the AI Evaluation practice strategy, operating model, methodology direction, and quality standards.


• Build and scale enterprise AI evaluation offerings across voice AI, chatbots, LLM applications, agentic AI, and multimodal AI.


• Own flagship client relationships and engage senior executives on AI quality, AI risk, and AI assurance.


• Shape high-value AI evaluation programs for enterprise clients.


• Define commercial models, effort estimation approaches, delivery structures, and margin discipline for evaluation engagements.


• Lead solutioning, proposals, presales discussions, and strategic account expansion.


• Build reusable frameworks for hallucination testing, robustness testing, red teaming, safety evaluation, compliance evaluation, and regression testing.


• Guide L3/L2 leaders and develop the leadership bench for the evaluation practice.


• Own hiring strategy, senior talent development, and succession planning.


• Identify new market opportunities, domains, modalities, partnerships, and adjacent offerings.


• Represent the practice externally through industry forums, client workshops, publications, advisory discussions, and thought leadership.


• Drive methodology adoption internally and externally.


• Ensure delivery quality, client satisfaction, revenue growth, and long-term practice scalability.

Required Knowledge and Skills

Expert-level understanding of AI evaluation, LLM evaluation, AI quality, conversational AI, and enterprise AI assurance.

• Strong knowledge of hallucination evaluation, robustness testing, safety testing, red teaming, responsible AI, and compliance-oriented evaluation.


• Strong understanding of enterprise AI delivery models, AI lifecycle risks, governance, and quality assurance.


• Strong executive stakeholder management and consultative selling ability.


• Proven experience building teams, practices, offerings, or large-scale delivery capabilities.


• Ability to define commercial strategy, pricing approach, delivery economics, and account growth plans.

Strong methodology design and thought leadership capability.

• Ability to mentor senior practitioners and create successors.


• Strong communication, executive presentation, negotiation, and strategic planning skills.


• Ability to work across delivery, product, sales, client leadership, and partner ecosystems.

Preferred Skills

• Experience leading an AI testing, AI quality, AI assurance, AI governance, or AI evaluation practice.


• Experience with AI evaluation and testing platforms.


• Strong network in AI, digital transformation, quality engineering, or enterprise technology.


• Experience publishing thought leadership, white papers, case studies, or speaking at industry events.


• Experience in regulated industries such as BFSI, healthcare, insurance, automotive, or telecom.


• Exposure to Responsible AI frameworks, model risk management, AI governance, and regulatory compliance.

Educational Qualification

B.E. / B.Tech / M.Tech / M.S. in Artificial Intelligence, Computer Science, Computer Engineering, Information Technology, or related Computer discipline.