Quality Assurance Analyst (Gen AI and RAG)

Infor Inc.; Hyderabad; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Infor is seeking a Quality Assurance Analyst focused on Gen AI and RAG to ensure high-quality enterprise software. You will collaborate with developers, product managers, and customer success teams to validate functionality, performance, and scalability.

Responsibilities include designing test cases for AI Studio components, validating schema contracts, testing content submissions, and conducting OpenSearch relevance evaluations across multiple indices to guarantee accurate results across

Qualifications

• Bachelor's degree in Computer Science, IT, Engineering or related field.
• 3–5 years of software QA experience.
• Experience with REST API testing and tools like Postman or REST Assured.
• Python automation experience for API and integration tests.
• Familiarity with search systems and relevance scoring.

Responsibilities

• Design and execute test cases for AI Studio's Orchestrator Agent.
• Validate gap map outputs and schema contracts.
• Test Content API submissions for structure and types.
• Run OpenSearch relevance tests across indices.
• Verify multi-agent packet assembly and output contracts.
• Ensure proper execution tier ordering and upstream dependencies.
• Perform tenant isolation testing across operations.
• Analyse audit logs and traces to identify root causes.

Skills

REST API testing

Python scripting

Analytical thinking

Bug reporting

Documentation

Education

Bachelor's degree in CS/IT/Engineering

Tools

Postman

REST Assured

OpenSearch

Elasticsearch

Job description

Quality Assurance Analyst (Gen AI and RAG)

Department: Development

Employment Type: Full Time

Location: Hyderabad

Description

As a Quality Assurance Analyst at Infor, you will have a critical role in ensuring the delivery of high-quality enterprise applications that meet the evolving needs of our global customers. You’ll collaborate closely with developers, product managers, and customer success teams to validate functionality, performance, and scalability—helping us deliver smarter, faster, and more intuitive software.

A Typical Day in the Life Includes:

• Design and execute test cases for AI Studio's Orchestrator Agent — verifying that the correct components are selected, the right questions are asked, and assumptions are applied accurately for a given user input.
• Validate gap map outputs — confirming that fields are correctly classified as confirmed, assumed, auto-resolved, or missing based on schema contracts and domain defaults.
• Test schema and knowledge document submissions through the Content API — verifying structure validation, field type enforcement, controlled vocabulary checks, and embedding generation.
• Execute search relevance tests across all six OpenSearch indices — verifying that queries return the expected documents with acceptable relevance scores.
• Test multi-agent task packet assembly — confirming that confirmed values, assumed values, upstream context, and output contracts are correctly populated before specialist agents are invoked.
• Verify execution tier ordering — ensuring components with blocking dependencies are never dispatched before their upstream dependencies have completed.
• Perform tenant isolation testing — confirming that session data, project history, and search results are strictly scoped to the correct tenant across all operations.
• Analyse audit logs and search traces to identify the root cause of failures in the multi-agent pipeline and report findings with clear reproduction steps.

Basic Qualifications:

• Bachelor's degree in Computer Science, Information Technology, Engineering, or related field. 3–5 years of experience in software quality assurance.
• Solid understanding of REST API testing and hands-on experience with tools such as Postman, REST Assured, or equivalent.
• Experience writing automated test scripts in Python — ability to build repeatable test cases for API-level and integration-level scenarios.
• Familiarity with search systems — understanding of how relevance scoring works and the ability to design queries that validate expected search behaviour.
• Ability to define acceptability criteria for non-deterministic outputs — understanding that LLM-driven responses require semantic evaluation rather than exact-match assertions.
• Experience with integration testing across multi-service or distributed architectures — ability to trace a failure across service boundaries to its root cause.
• Strong analytical skills and attention to detail — particularly important when validating structured JSON outputs such as task packets, gap maps, and deployment blueprints.
• Good written communication skills — ability to write clear, reproducible bug reports and test documentation.

Preferred Qualifications:

• Exposure to RAG systems, vector search, or LLM-driven applications — either through professional experience or personal projects.
• Familiarity with OpenSearch, Elasticsearch, or similar search platforms.
• Exposure to LLM evaluation tools such as RAGAS, DeepEval, or equivalent frameworks for assessing AI output quality.
• Experience with multi-tenant SaaS testing — particularly data isolation and session boundary validation.
• Familiarity with CI/CD pipelines and the ability to integrate automated test suites into deployment workflows.
• Knowledge of JSON Schema validation and the ability to write schema-level assertions against complex nested JSON structures.
• Exposure to Infor OS, ERP systems, or enterprise workflow automation products.
• Experience designing synthetic test data for knowledge-driven systems where the quality of test documents directly affects the coverage and reliability of test results.