Senior Automation Quality Analyst

Ciklum; Pune District, Chennai District; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

Ciklum is seeking a QA Automation professional in Pune to lead E2E testing for document processing pipelines, OCR/vision extraction, and AI-driven workflows. The role includes validating RAG retrieval, reviewing probabilistic AI outputs, and ensuring data protection checks across API and UI layers.

The candidate will work with Azure-based systems, Boost.ai, and enterprise integrations (CRM/SAP/SharePoint/Jira), contributing to robust QA automation and regression testing.

Qualifications

• E2E testing of multi-format documents (PDFs, Word, PPT) and OCR/vision extraction.
• Validate RAG retrieval quality with hybrid search and metadata filters.
• Verify AI outputs for grounding, hallucination risk, and source references.

Responsibilities

• E2E testing of document processing and parsing accuracy (OCR/vision).
• Refine acceptance criteria, create test scenarios, execute, report, and manage defects.
• Test API, UI, and enterprise integrations across portals and chatbots.

Skills

UI automated testing

API testing

QA automation

Defect management

Tools

Azure

Boost.ai

SAP

SharePoint

Jira

Job description

Role & responsibilities

• E2E testing of multi-format document processing (PDFs, Word, PPT, images/screenshots) and parsing accuracy (OCR/Vision extraction).
• Performing standard test activities: acceptance criteria refinement, test scenario creation, execution, reporting, defect management, and regression testing.
• Validation of RAG retrieval quality, including hybrid search (semantic + BM25 keyword search), metadata filtering, and retrieval scoping.
• Verification of probabilistic AI outputs for answer grounding, hallucination risk, source references, and confidence scores.
• E2E testing of human-in-the-loop content workflows (AI draft generation, confidence score thresholds, reviewer approvals, and vector DB publishing sync).
• API and enterprise integration testing (portals, chatbots, ticketing systems) across deterministic and non-deterministic flows.
• Validation of data protection controls, including PII scanning and redaction.
• Support QA automation at both API and UI levels for critical data pipelines.

Preferred candidate profile

• Experience working with Azure-based systems is a strong plus.
• Experience with Boost.ai.
• Experience testing enterprise integrations (CRMs, SAP, SharePoint, Jira).
• Experience with UI automated testing.