Senior Manual QA Engineer (AI / LLM & API Testing)

Description Ciklum; India; On-site; INR 1,500,000 - 2,100,000; Full time

Benefits offered by this job

Medical Insurance

Mental Health Support

Financial & Legal Consultations

Udemy Access

Language Courses

Company-paid Certifications

Job summary

Ciklum is seeking a Senior Manual QA Engineer in India to lead testing of an AI-driven knowledge platform. You will verify document processing, AI outputs, and API integrations across portals and chatbots.

You’ll ensure data privacy, assess RAG retrievals, and contribute to QA automation for pipelines, with a focus on both traditional and AI-specific testing.

Qualifications

• Strong foundation in software QA processes.
• Hands-on testing of web apps and REST APIs (manual & automated).
• Experience testing deterministic and probabilistic AI behavior (LLMs).
• Ability to evaluate RAG architecture outputs and AI grounding.
• Experience with document processing pipelines (OCR, parsers).
• Knowledge of data privacy in AI apps.
• SQL knowledge.
• Azure-based systems experience is a plus.
• Experience with ERP and enterprise integrations (CRM, SAP, SharePoint, Jira).
• UI automated testing experience.

Responsibilities

• E2E testing of multi-format document processing and OCR accuracy.
• Refine acceptance criteria, create test scenarios, execute, report, and manage defects.
• Validate RAG retrieval quality and metadata filtering.
• Test AI outputs: grounding, hallucination risk, references, confidence scores.
• End-to-end testing of content workflows and vector DB publishing sync.
• Test APIs and enterprise integrations across portals and chatbots.
• Verify data protection controls and PII scanning.
• Support QA automation at API and UI levels for data pipelines.

Skills

QA fundamentals

Web testing

REST API testing

AI/LLM testing

RAG testing

Data privacy

SQL

UI automation testing

Azure experience

Boost.ai

CRM integrations testing

Jira

Tools

Azure

Boost.ai

Jira

SAP

SharePoint

Job description

Job Description

Ciklum is looking for a Senior Manual QA Engineer (AI / LLM & API Testing) to join our team full-time in India.

We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and product owners, we engineer technology that redefines industries and shapes the way people live.

About the role:

As a Senior Manual QA Engineer (AI / LLM & API Testing), become a part of a cross-functional development team engineering experiences of tomorrow.

Client is looking for a QA Engineer to test an AI-powered knowledge management and retrieval platform. The solution includes document processing, RAG-based search, LLM-generated answers and integrations with external systems. An optimal candidate should be able to test traditional software features and AI-specific behavior (e.g. hallucination detection, referencing sources, confidence scores, PII handling).

Responsibilities:

• E2E testing of multi-format document processing (PDFs, Word, PPT, images/screenshots) and parsing accuracy (OCR/Vision extraction)
• Performing standard test activities: acceptance criteria refinement, test scenario creation, execution, reporting, defect management, and regression testing
• Validation of RAG retrieval quality, including hybrid search (semantic + BM25 keyword search), metadata filtering, and retrieval scoping
• Verification of probabilistic AI outputs for answer grounding, hallucination risk, source references, and confidence scores
• E2E testing of human-in-the-loop content workflows (AI draft generation, confidence score thresholds, reviewer approvals, and vector DB publishing sync)
• API and enterprise integration testing (portals, chatbots, ticketing systems) across deterministic and non-deterministic flows
• Validation of data protection controls, including PII scanning and redaction
• Support QA automation at both API and UI levels for critical data pipelines

Requirements:

• Strong foundation in software QA processes
• Hands-on experience with web application testing and REST API testing (manual and automated)
• Practical understanding of testing both deterministic logic (pass/fail, APIs, DB sync) and probabilistic AI behavior (LLM outputs, context edge cases)
• Ability to evaluate RAG architecture outputs
• Experience verifying document processing pipelines (data extraction, OCR, parsers)
• Solid understanding of data privacy principles in AI applications
• Working knowledge of SQL
• Experience working with Azure-based systems is a strong plus
• Experience with Boost.ai
• Experience testing enterprise integrations (CRMs, SAP, SharePoint, Jira)
• Experience with UI automated testing

What’s in it for you?

• Strong community: Work alongside top professionals in a friendly, open-door environment
• Growth focus: Take on large-scale projects with a global impact and expand your expertise
• Tailored learning: Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications
• Endless opportunities: Explore diverse domains through internal mobility, finding the best fit to gain hands-on experience with cutting-edge technologies
• Care: We’ve got you covered with company-paid medical insurance, mental health support, and financial & legal consultations

About us:

At Ciklum, we are always exploring innovations, empowering each other to achieve more, and engineering solutions that matter. With us, you’ll work with cutting-edge technologies, contribute to impactful projects, and be part of a One Team culture that values collaboration and progress.

India is a strategic innovation hub for Ciklum, with growing teams in Chennai and Pune leading advancements in EdgeTech, AR/VR, IoT, and beyond. Join us to collaborate on game-changing solutions and take your career to the next level.

Explore, empower, engineer with Ciklum!