GenAI / Agent Testing Engineer(Bangalore only)

PwC; Bengaluru; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

PwC Bengaluru is seeking a GenAI / LLM Agent Test Engineer to ensure quality and responsible behavior of LLM-based and RAG-based agents, including single and multi-agent workflows.

You will focus on prompt engineering, hallucination detection, and automated evaluation using modern LLM testing frameworks, collaborating with platform, data, and GenAI teams.

Qualifications

• Experience testing LLM-based systems and RAG architectures.
• Ability to design prompts and analyze model behavior.
• Familiarity with ethics and responsible AI principles.

Responsibilities

• Test and validate LLM-based agents across single and multi-agent workflows.
• Design prompt engineering and adversarial tests to uncover hallucinations.
• Evaluate hallucinations, relevance, and factual accuracy.
• Implement unit tests for LLMs and integration with testing pipelines.
• Ensure ethical and responsible AI testing coverage.
• Collaborate with platform, data, and GenAI teams for quality improvement.

Skills

GenAI testing

LLM testing

Prompt engineering

Adversarial testing

Quality assurance

Education

Bachelor's degree in Computer Science or related field

Tools

Python

PyTest

LLM frameworks

Job description

Job Summary: We are looking for a GenAI / LLM Agent Test Engineer to assure quality, reliability, and responsible behavior of LLM-based and RAG-based agents, including single-agent and multi-agent workflows. The role focuses on prompt engineering, hallucination detection, agent reasoning validation, and automated evaluation of GenAI outputs using modern LLM testing frameworks.

Key Responsibilities

• Test and validate LLM-based and RAG-based agents, including reasoning, memory, and decision-making behavior
• Design and execute prompt engineering and adversarial testing to uncover hallucinations, bias, and edge cases
• Perform hallucination testing, relevance checks, and factual accuracy validation
• Implement unit tests for LLMs using LLM-as-a-Judge techniques
• Validate agent reasoning chains, memory states, and conversation persistence
• Embed LLMs within testing pipelines to:
  • Detect hallucinations
  • Run adversarial prompt testing
• Perform automated evaluation of agent outputs
• Ensure AI Ethics and Responsible AI compliance in testing coverage
• Collaborate with platform, data, and GenAI teams for continuous quality improvement