Test Engineer

Persistent Systems; Bengaluru; Hybrid; INR 2,000,000 - 4,000,000; Full time

Benefits offered by this job

Hybrid work

Long Service awards

Education programs

Insurance coverage

Competitive salary

Job summary

Persistent Systems in Bengaluru seeks an AI Test Engineer with 8-14 years of experience to validate AI-powered applications across the software development lifecycle, focusing on LLMs, Agentic AI, prompt workflows, and AI integrations. You will design test strategies, automate tests, and ensure accuracy, safety, and bias checks while collaborating with AI engineers, developers, and product teams.

Hybrid work and growth opportunities are offered.

Qualifications

• 8–14 years of software testing and QA experience in AI/Generative AI.
• Hands-on testing of LLM outputs, safety, bias, and hallucinations.
• Experience with test automation frameworks and AI validation pipelines.
• Proficient in API testing and automation using Postman/REST Assured.
• Familiarity with CI/CD integration and agile/scrum environments.

Responsibilities

• Design and execute test strategies for Generative AI and Agentic AI apps.
• Validate LLM outputs for accuracy, bias, and safety.
• Develop and maintain automated testing frameworks for AI apps.
• Collaborate with developers, AI engineers, and product teams to define quality gates.
• Create test datasets and monitor model drift risks.
• Contribute to testing governance and responsible AI practices.

Skills

AI Testing

Automation

LLMs

Selenium

Playwright

API testing

CI/CD

Agile/Scrum

Tools

Selenium

Playwright

Postman

REST Assured

Job description

We are seeking an AI Test Engineer with 8-14 years of experience in Software Testing, Test Automation, and AI/Generative AI validation. The ideal candidate will be responsible for ensuring the quality, reliability, safety, and correctness of AI-powered applications across the software development lifecycle. This role requires expertise in testing Large Language Models (LLMs), Agentic AI solutions, prompt engineering workflows, AI integrations, and automated testing frameworks.

• Experience: 8 to 14 years
• Job Type: Full Time Employment

What You'll Do:

• Design and execute comprehensive test strategies for Generative AI and Agentic AI applications.
• Validate LLM outputs for accuracy, bias, hallucinations, safety, and consistency.
• Test prompt versions and evaluate AI behavior changes across model updates.
• Develop and maintain automated test frameworks for AI-enabled applications.
• Execute functional, regression, integration, performance, and non-functional testing.
• Test AI integrations, APIs, workflows, and external service connections.
• Perform regression testing for AI model updates, prompt changes, and version upgrades.
• Collaborate with developers, AI engineers, and product teams to establish quality gates.
• Create and manage test datasets for AI validation and benchmarking.
• Monitor AI application performance and identify risks related to model drift and output quality.
• Support CI/CD quality gates and automated validation pipelines.
• Document test cases, test results, defects, and quality metrics.
• Contribute to responsible AI practices and testing governance frameworks.
• Drive continuous improvement in AI testing methodologies and automation practices.

Expertise You'll Bring:

• 8-14 years of experience in Software Testing and Quality Engineering.
• Strong experience in Manual and Automation Testing.
• Hands-on experience with AI/Generative AI testing.
• Strong understanding of Large Language Models (LLMs) and AI application behavior.
• Experience validating AI outputs for accuracy, safety, bias, and hallucinations.
• Knowledge of Agentic AI systems and workflow testing.
• Experience with Selenium and Playwright automation frameworks.
• Strong API testing experience using tools such as Postman, REST Assured, or equivalent.
• Understanding of model variability, non-deterministic outputs, and AI evaluation methods.
• Experience with AI prompt testing and prompt optimization validation.
• Strong knowledge of test data management practices for AI systems.
• Familiarity with Responsible AI principles and AI risk assessment.
• Experience integrating testing into CI/CD pipelines.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and stakeholder collaboration capabilities.
• Experience working in Agile/Scrum environments.
• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting-edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."