AI/ML Quality & Automation Engineer (GenAI & Agentic Systems)

Virtusa Corporation

Hyderabad, Chennai, Bengaluru

Hybrid

INR 900,000 - 1,300,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Virtusa Corporation in India seeks an AI/ML Engineer to lead GenAI and LLM testing, designing automated validation frameworks to ensure reliability, accuracy and AI security of autonomous workflows and conversational systems. You will build test suites for Generative AI, RAG pipelines, perform NLP benchmarking, and collaborate with ML/DL teams to address model vulnerabilities and implement guardrails.

This role emphasizes hands-on coding in Python and Java, API testing with Postman, and using

Qualifications

  • Experience designing automated validation frameworks for AI/ML products.
  • Hands-on with Generative AI, Agentic AI, RAG pipelines.
  • Proficient in LLM Testing, GenAI Testing and AI Security best practices.
  • Solid understanding of ML, DL and NLP concepts.
  • Proficiency in API testing with Postman; agile tracking with JIRA.

Responsibilities

  • Build and execute automated test suites for Generative AI, Agentic AI and RAG pipelines.
  • Perform rigorous RAG Evaluation and NLP benchmarking to minimize hallucinations and ensure quality.
  • Develop and test integrations using MCP, validating context sharing across tools and models.
  • Design and automate API testing workflows using Postman, Python and Java.
  • Collaborate with ML and DL teams to identify vulnerabilities, enforce AI security guardrails, and track defects in JIRA.

Skills

Python
Java
AIML
GenAI Testing
LLM Testing
RAG Evaluation
NLP
Machine Learning
Deep Learning
API testing

Tools

MCP
Postman
JIRA

Job description

We are seeking a skilled AI/ML Engineer to lead GenAI Testing and LLM Testing for our next-generation Artificial Intelligence products. You will design automated validation frameworks to ensure the reliability, accuracy, and AI Security of autonomous workflows and conversational systems.

Key Responsibilities:
  • Build and execute automated test suites for Generative Artificial Intelligence (Generative AI), Agentic AI, and Retrieval Augmented Generation (RAG) pipelines.
  • Perform rigorous RAG Evaluation and Natural Language Processing (NLP) benchmarking to minimize hallucinations and ensure response quality.
  • Develop and test integrations using the Model Context Protocol (MCP), validating context sharing across tools and models.
  • Design and automate API testing workflows using Postman, Python, and Java.
  • Collaborate with Machine Learning and Deep Learning teams to identify model vulnerabilities, enforce AI Security guardrails, and track defects using JIRA.
Required Skills & Qualifications:
  • Strong programming experience in Python and Java for test automation and AIML application development.
  • Hands-on experience with Generative AI, Agentic AI, Retrieval Augmented Generation, and Model Context Protocol (MCP).
  • Proven expertise in LLM Testing, GenAI Testing, RAG Evaluation, and AI Security best practices.
  • Solid understanding of core Machine Learning, Deep Learning, and Natural Language Processing concepts.
  • Proficiency in API testing with Postman and agile project tracking with JIRA.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

GenAI Test Lead
GenAI Test Lead

QualiZeal • Hyderabad

On-site
INR 3,500,000 - 6,500,000
Flexible work hours
GenAI project exposure
Senior AI Testing Engineer (Generative AI)
Senior AI Testing Engineer (Generative AI)

Wfnen • Bengaluru

Hybrid
INR 1,200,000 - 1,600,000
Hiring For GenAI QA
Hiring For GenAI QA

1point1 • Navi Mumbai

On-site
INR 1,000,000 - 1,800,000
AI or ML Engineer - Python, QA Testing
AI or ML Engineer - Python, QA Testing

Optum • Hyderabad

On-site
INR 1,800,000 - 3,200,000
Lead AI Test Engineer
Lead AI Test Engineer

Qentelli LLC • Hyderabad

On-site
INR 1,800,000 - 2,400,000
AI QA Lead Engineer
AI QA Lead Engineer

Qentelli LLC • Hyderabad

On-site
INR 1,200,000 - 1,800,000
Gen AI QA Engineer
Gen AI QA Engineer

Deqode • Bengaluru

On-site
INR 1,200,000 - 2,100,000
AI/LLM QA Engineer - Agentic and Multi-Agent System Testing
AI/LLM QA Engineer - Agentic and Multi-Agent System Testing

Crew Kraftorz LLP • Hyderabad

Hybrid
INR 1,500,000 - 2,200,000
AI Test Engineer
AI Test Engineer

AXA Global Business Services (AXA GBS) • Pune District

On-site
INR 900,000 - 1,500,000
QA Lead
QA Lead

Nura Analytics • Chennai District

On-site
INR 1,200,000 - 1,500,000