Hiring For GenAI QA

1point1

Navi Mumbai

On-site

INR 1,000,000 - 1,800,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

1Point1 in Navi Mumbai is seeking a GenAI QA Engineer to test and validate Generative AI applications, ensuring accuracy, reliability, and performance.

You will perform functional and API testing, develop automation scripts in Python/Java, and validate AI-generated responses, prompts, and multi-turn conversations. Collaboration with data scientists and AI engineers is essential.

Qualifications

  • Bachelor's degree in Computer Science or related field.
  • 1+ years of software testing experience.
  • 1+ years exposure to Generative AI / LLM testing.
  • Strong understanding of software testing methodologies.
  • Hands-on Python-based testing & API automation.
  • Experience testing chatbot or AI-powered applications.

Responsibilities

  • Test GenAI apps, chatbots, and AI-powered solutions.
  • Validate LLM responses for accuracy, relevance, and completeness.
  • Test prompt engineering scenarios and prompt-response behavior.
  • Validate multi-turn conversations and context understanding.
  • Evaluate LLM outputs for hallucinations, bias, toxicity, and factuality.
  • Assess RAG workflows: ingestion, embedding, and retrieval accuracy.
  • Test semantic search and vector database integration.
  • Ensure outputs are grounded in retrieved docs and citations.
  • Develop API tests with Postman/REST Assured and automation scripts.
  • Integrate tests into CI/CD pipelines and regression suites.
  • Assess performance: latency, throughput, and concurrency.
  • Evaluate security and data privacy for GenAI apps.
  • Prepare test plans, reports, and collaborate with team.

Skills

Functional testing
API testing
Automation
Prompt testing
RAG architectures
LLM testing

Education

Bachelor's degree in Computer Science or related field

Tools

Postman
REST Assured
Python
Java

Job description

Job Title

Generative AI QA Engineer

Experience

1-2 Years (including relevant QA experience)

Domain

Generative AI / LLM / AI Testing

Employment Type

Full-Time

Job Summary

We are looking for an experienced GenAI QA Engineer responsible for testing, validating, and ensuring the quality, reliability, accuracy, and performance of Generative AI applications and Large Language Models (LLMs).

The candidate should have experience in functional testing, API testing, automation, prompt testing, and validating AI-generated responses. Familiarity with RAG architectures, vector databases, AI agents, and LLM evaluation frameworks is highly desirable.

Key Responsibilities
GenAI Application Testing
  • Test Generative AI applications, chatbots, virtual assistants, and AI-powered solutions.
  • Validate LLM-generated responses for accuracy, relevance, consistency, and completeness.
  • Test prompt engineering scenarios and prompt-response behavior.
  • Validate multi-turn conversations and contextual understanding.
LLM Testing & Evaluation
  • Evaluate LLM responses for hallucinations, bias, toxicity, and factual correctness.
  • Perform prompt injection and jailbreak testing.
  • Validate response quality, relevance, groundedness, and context retention.
  • Define test datasets and evaluation benchmarks.
RAG Testing
  • Test Retrieval-Augmented Generation (RAG) applications.
  • Validate document ingestion, chunking, embedding, and retrieval accuracy.
  • Test semantic search and vector database integration.
  • Validate whether generated responses are grounded in retrieved documents.
  • Test citation accuracy and source relevance.
API & Automation Testing
  • Perform API testing using Postman or REST Assured.
  • Develop automation scripts using Python or Java.
  • Automate GenAI test scenarios and regression test suites.
  • Integrate automated tests into CI/CD pipelines.
Performance & Security Testing
  • Test response latency, throughput, scalability, and token consumption.
  • Validate application behavior under concurrent user loads.
  • Test security vulnerabilities specific to LLM applications.
  • Validate data privacy and sensitive information leakage.
Reporting & Collaboration
  • Prepare test strategies, test plans, test cases, and test reports.
  • Collaborate with developers, data scientists, and AI engineers.
  • Identify defects, analyze root causes, and track resolutions.
Experience & Qualifications
  • Bachelor's degree in Computer Science, IT, or a related field.
  • 1+ years of software testing experience.
  • 1+ years of exposure to Generative AI / LLM testing.
  • Strong understanding of software testing methodologies.
  • Hands-on experience with Python-based testing and API automation.
  • Experience testing chatbot or AI-powered applications.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI QA Engineer
AI QA Engineer

Aventra.AI • Chennai

On-site
INR 1,200,000 - 2,100,000
Senior AI Testing Engineer (Generative AI)
Senior AI Testing Engineer (Generative AI)

Wfnen • Bengaluru

Hybrid
INR 1,200,000 - 1,600,000
GenAI Test Lead
GenAI Test Lead

QualiZeal • Hyderabad

On-site
INR 3,500,000 - 6,500,000
Flexible work hours
GenAI project exposure
Gen AI QA Engineer
Gen AI QA Engineer

Deqode • Bengaluru

On-site
INR 1,200,000 - 2,100,000
Lead AI Test Engineer
Lead AI Test Engineer

Qentelli LLC • Hyderabad

On-site
INR 1,800,000 - 2,400,000
QA Engineer
QA Engineer

TigerHire • India

Remote
INR 1,200,000 - 2,000,000
Generative AI Engineer
Generative AI Engineer

Tekskills Inc. • Pune District

On-site
INR 1,200,000 - 1,600,000
AI Test Engineer
AI Test Engineer

Deqode • Bengaluru

On-site
INR 900,000 - 1,500,000
Generative AI Developer - Senior Associate
Generative AI Developer - Senior Associate

PwC • Bengaluru

On-site
INR 2,400,000 - 3,400,000
AI QA Lead Engineer
AI QA Lead Engineer

Qentelli LLC • Hyderabad

On-site
INR 1,200,000 - 1,800,000