Exciting Opportunity - AI Evaluation Testing Lead / Expert - Bengalaru

QualityKiosk

Navi Mumbai, Mumbai, Bengaluru

On-site

INR 4,000,000 - 8,000,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

QualityKiosk Technologies is seeking an AI Evaluation Lead Expert (12+ years) to own the strategy, methodology, commercial growth, and leadership direction of the AI Evaluation practice. This is a senior leadership role focused on scaling AI quality capabilities, building flagship accounts, shaping enterprise AI evaluation programs, and establishing thought leadership in AI assurance.

This role requires 12+ years total experience with 5+ years in AI/ML/NLP/LLMs and 3+ years in leadership, plus

Qualifications

  • Total 12+ years of experience in AI-related fields.
  • 5+ years in AI/ML/NLP/LLMs/AI quality and 3+ years in leadership roles.

Responsibilities

  • Define and own AI Evaluation practice strategy, model, and standards.
  • Scale enterprise AI evaluation offerings across voice AI, chatbots, and LLM apps.
  • Own flagship client relationships and advise on AI quality and risk.
  • Shape programs, define commercial models, delivery margins.
  • Lead solutioning, proposals, presales, and strategic account growth.
  • Develop reusable frameworks for hallucination, robustness, safety, and compliance testing.
  • Guide L3/L2 leaders and build leadership bench in the practice.
  • Oversee hiring strategy and senior talent development.

Skills

AI evaluation
Leadership
Strategy
Stakeholder management
Presales discussions

Education

B.E. / B.Tech / M.Tech / M.S. in AI/CS/CE/IT

Tools

AI evaluation platforms

Job description

We're Hiring | AI Evaluation Lead | Bangalore | Mumbai

QualityKiosk Technologies is looking for an experienced AI Evaluation Lead Expert (12+ Years) to join our growing team and drive transformation initiatives.

Website: www.qualitykiosk.com

Location: Bangalore/ Mumabi
Preference: Immediate Joiners

What We're Looking For

Role Summary We are looking for an AI Evaluation Lead / Principal Consultant to own the strategy, methodology, commercial growth, and leadership direction of the AI Evaluation practice. This is a senior leadership role focused on scaling AI quality capabilities, building flagship accounts, shaping enterprise AI evaluation programs, and establishing thought leadership in AI assurance.

Experience Required

12+ years of total experience. Candidates should have 5+ years of experience in AI, ML, NLP, LLMs, AI quality, enterprise AI delivery, AI evaluation, or AI assurance, with 3+ years in team leadership, practice leadership, or senior client advisory roles.

Key Responsibilities
  • Define and own the AI Evaluation practice strategy, operating model, methodology direction, and quality standards.
  • Build and scale enterprise AI evaluation offerings across voice AI, chatbots, LLM applications, agentic AI, and multimodal AI.
  • Own flagship client relationships and engage senior executives on AI quality, AI risk, and AI assurance.
  • Shape high-value AI evaluation programs for enterprise clients. Define commercial models, effort estimation approaches, delivery structures, and margin discipline for evaluation engagements.
  • Lead solutioning, proposals, presales discussions, and strategic account expansion.
  • Build reusable frameworks for hallucination testing, robustness testing, red teaming, safety evaluation, compliance evaluation, and regression testing.
  • Guide L3/L2 leaders and develop the leadership bench for the evaluation practice.
  • Own hiring strategy, senior talent development, and succession planning.
  • Identify new market opportunities, domains, modalities, partnerships, and adjacent offerings.
  • Represent the practice externally through industry forums, client workshops, publications, advisory discussions, and thought leadership.
  • Drive methodology adoption internally and externally.
  • Ensure delivery quality, client satisfaction, revenue growth, and long-term practice scalability.
Required Knowledge and Skills

Expert-level understanding of AI evaluation, LLM evaluation, AI quality, conversational AI, and enterprise AI assurance.

  • Strong knowledge of hallucination evaluation, robustness testing, safety testing, red teaming, responsible AI, and compliance-oriented evaluation.
  • Strong understanding of enterprise AI delivery models, AI lifecycle risks, governance, and quality assurance.
  • Strong executive stakeholder management and consultative selling ability.
  • Proven experience building teams, practices, offerings, or large-scale delivery capabilities.
  • Ability to define commercial strategy, pricing approach, delivery economics, and account growth plans.

Strong methodology design and thought leadership capability.

Preferred Skills
  • Experience leading an AI testing, AI quality, AI assurance, AI governance, or AI evaluation practice.
  • Experience with AI evaluation and testing platforms.
  • Strong network in AI, digital transformation, quality engineering, or enterprise technology.
  • Experience publishing thought leadership, white papers, case studies, or speaking at industry events.
  • Experience in regulated industries such as BFSI, healthcare, insurance, automotive, or telecom.
  • Exposure to Responsible AI frameworks, model risk management, AI governance, and regulatory compliance.

Educational Qualification B.E. / B.Tech / M.Tech / M.S. in Artificial Intelligence, Computer Science, Computer Engineering, Information Technology, or related Computer discipline.


Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Quality Engineer – R01573517
Lead Quality Engineer – R01573517

Brillio • Bengaluru

On-site
INR 3,000,000 - 5,400,000
Associate Principal - Quality Engineering (AI-Driven Test Automation)
Associate Principal - Quality Engineering (AI-Driven Test Automation)

Vivantify Technology Solutions India Private Limited • Bengaluru

Hybrid
INR 3,500,000 - 6,000,000
Competitive salary
Professional growth opportunities
Collaborative work environment
+2
Senior AI Testing Engineer (Generative AI)
Senior AI Testing Engineer (Generative AI)

Wfnen • Bengaluru

Hybrid
INR 1,200,000 - 1,600,000
Lead Software Development Engineer in Test - AI
Lead Software Development Engineer in Test - AI

Jobgether • India

Remote
INR 4,000,000 - 7,000,000
Fully remote (India)
Competitive salary
Leadership opportunities
+2
Associate Principal - Quality Engineering (AI-Driven Test Automation)
Associate Principal - Quality Engineering (AI-Driven Test Automation)

Vivantify Technology Solutions India Pvt. Ltd. • Bengaluru

On-site
INR 4,200,000 - 6,600,000
AI Senior QA
AI Senior QA

Hiver, Inc. • Bengaluru

On-site
INR 2,200,000 - 3,400,000
Senior Quality & AI Evaluation Engineer (AIRE)
Senior Quality & AI Evaluation Engineer (AIRE)

BMW TechWorks India Private Limited • Pune District, Bengaluru

On-site
INR 3,000,000 - 4,200,000
AI Evaluation Engineer - Proofline
AI Evaluation Engineer - Proofline

Fermi AI • Bengaluru

On-site
INR 1,500,000 - 2,100,000
QA Engineer(AI)
QA Engineer(AI)

National e-Governance Division • New Delhi

On-site
INR 3,500,000 - 5,000,000
AI QA Engineer
AI QA Engineer

NewVision Software • Pune District

Hybrid
INR 1,200,000 - 1,800,000