Principal QA – AI & Conversational Systems (Pune)

Codvo Private Limited; Pune District; On-site; INR 1,500,000 - 2,000,000; Full time

Job summary

A global technology services company is seeking a Principal QA – AI & Conversational Systems to lead evaluations of AI-driven voice bots and agent assist systems. This role requires designing validation frameworks to ensure compliance with accuracy, safety, and latency standards. The ideal candidate will possess experience in LLM testing and knowledge of AI safety and bias validation. Join a team that values respect, fairness, and innovation within the technology sector.

Qualifications

• Experience in LLM or conversational AI testing.
• Hands-on exposure to AI evaluation metrics.
• Strong understanding of AI safety and bias validation.
• Experience designing AI testing frameworks.

Responsibilities

• Lead evaluation of AI-driven voice bots, agent assist, and summarization systems.
• Design LLM validation frameworks ensuring accuracy, safety, and latency compliance.
• Drive responsible AI and bias testing standards.
• Hallucination detection and response accuracy scoring.
• RAG and knowledge grounding validation.
• Synthetic call testing and AI load benchmarking.
• Prompt robustness and fallback validation.
• Sensitive data leakage and AI compliance testing.

Skills

Experience in LLM or conversational AI testing

Hands-on exposure to AI evaluation metrics

Strong understanding of AI safety and bias validation

Experience designing AI testing frameworks

Job description

Principal QA – AI & Conversational Systems

Company Overview

AtCodvo, software and people transformations gohand-in-hand. We are a global empathy-led technology services company. Product innovation and mature software engineering are part of our core DNA. Respect, Fairness, Growth, Agility, and Inclusiveness are the core values that we aspire to live by each day. We continue to expand our digital strategy, design, architecture, and product management capabilities to offer expertise, outside-the-box thinking, and measurable results

Role Overview

Lead evaluation of AI-driven voice bots, agent assist, and summarization systems.

Design LLM validation frameworks ensuring accuracy, safety, and latency compliance.

Drive responsible AI and bias testing standards.

Core Responsibilities

• Hallucination detection and response accuracy scoring.
• RAG and knowledge grounding validation.
• Synthetic call testing and AI load benchmarking.
• Prompt robustness and fallback validation.
• Sensitive data leakage and AI compliance testing.

Ideal Background

• Experience in LLM or conversational AI testing.
• Hands-on exposure to AI evaluation metrics.
• Strong understanding of AI safety and bias validation.
• Experience designing AI testing frameworks.