AI Quality & Model Evaluation Lead

Adecco India; Bengaluru; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

Adecco India is seeking an AI Quality & Model Evaluation Lead in Bengaluru to drive QA, healthcare audit validation and AI/ML evaluation for production-ready AI-powered healthcare solutions.

The role requires hands-on leadership to establish quality standards, perform audits, evaluate model performance, and provide Go/No-Go decisions for deployments. Strong SQL, Python, and testing expertise are essential.

Qualifications

• Bachelor's degree in CS, statistics, data science, health informatics, healthcare mgmt, engineering or related field.
• Master's degree is an added advantage.

Responsibilities

• Drive QA, healthcare audit validation, AI/ML evaluation, and production readiness for AI healthcare solutions.
• Establish quality standards, audit methodologies, evaluate model performance and provide Go/No-Go recommendations.
• Lead QA processes, test management, data validation, and stakeholder reporting.

Skills

QA & Software Testing

AI/ML Model Evaluation

LLM & Generative AI Evaluation

Healthcare RCM

Statistical Sampling

Data Analysis & SQL

Python

LLM Evaluation Tools

API Testing

Test Management & Defect Tracking

QA Automation

Quality Metrics & Reporting

Go/No-Go & Risk Assessment

Agile & QA Process

Leadership & Stakeholder Management

Education

Bachelor's Degree in CS/Statistics/Data Science/Health Informatics/Healthcare Management/Engineering

Master's Degree (added advantage)

Tools

DeepEval

Ragas

LangSmith

TruLens

Promptfoo

Postman

Job description

Job Title: AI Quality & Model Evaluation Lead
Department: Engineering
Location: Bengaluru
Employment Type: Full-Time
Experience: 5+ Years
Role Level: Technical Lead

Job Overview

We are looking for an experienced AI Quality & Model Evaluation Lead to drive quality assurance, healthcare audit validation, AI/ML evaluation, and production readiness for AI-powered healthcare solutions. The ideal candidate will bring strong expertise in software testing, healthcare revenue cycle management (RCM), statistical quality methodologies, AI/ML model evaluation, and Generative AI assessment.

The role requires a hands-on leader who can establish quality standards, define audit methodologies, evaluate AI model performance, perform healthcare coding audits, and provide independent Go/No-Go recommendations for production deployments.

Required Skill Set

• QA & Software Testing (Functional, Integration, Regression, API, Data Validation, Test Planning and Execution)
• AI/ML Model Evaluation (Model Accuracy, Precision, Recall, F1-Score, Sensitivity, Specificity, Benchmark and Ground-Truth Validation)
• LLM & Generative AI Evaluation (Response Accuracy, Relevance, Groundedness, Hallucination Detection, Factuality, Consistency, and Prompt Evaluation)
• Healthcare RCM (Medical Coding Validation, Clinical Documentation Review, AR Denial Management, Prior Authorization, and Audit Methodology)
• Statistical Sampling & Audit (Random/Stratified Sampling, Sample Size Determination, Quality Audits, Confidence Levels, and Error Analysis)
• Data Analysis & SQL (Data Extraction, Validation, Reconciliation, Quality Analysis, and Complex SQL Queries)
• Python (Data Analysis, Statistical Analysis, Test Automation, and Evaluation Scripts)
• LLM Evaluation Tools (DeepEval, Ragas, LangSmith, TruLens, Promptfoo, or Equivalent Frameworks)
• API Testing (Postman, REST APIs, Request/Response Validation, Authentication, and API Automation)
• Test Management & Defect Tracking (JIRA, TestRail, TestLink, Zephyr, Xray, Defect Lifecycle Management, and Reporting)
• QA Automation (Automation Frameworks, Regression Automation, Data Validation, and Test Execution Optimization)
• Quality Metrics & Reporting (Test Coverage, Defect Trends, Model Accuracy, Audit Results, Quality KPIs, and Dashboards)
• Go/No-Go & Risk Assessment (Release Readiness, Quality Risk Evaluation, Acceptance Criteria, and Data-Driven Decisions)
• Agile & QA Process (Sprint Planning, Estimation, QA Processes, Continuous Improvement, and Cross-Functional Collaboration)
• Leadership & Stakeholder Management (Team Mentoring, Task Planning, Communication, Issue Escalation, and Stakeholder Coordination)

Qualifications

• Bachelor's Degree in Computer Science, Statistics, Data Science, Health Informatics, Healthcare Management, Engineering, or a related field.
• Master's Degree is an added advantage.

Candidate Profile

5+ years of experience in Software QA, Healthcare Audit, or AI/ML Quality Evaluation
Strong hands-on experience in QA and Testing activities
Experience evaluating AI/ML Models and Generative AI/LLM Applications
Healthcare RCM, Medical Coding Validation, and Clinical Documentation Audit experience
Strong SQL, Data Analysis, and Python skills
Experience with LLM Evaluation Tools such as DeepEval, Ragas, or LangSmith
Ability to make independent Go/No-Go decisions based on quality, audit, and statistical evidence
Excellent leadership, stakeholder management, and communication skills