Lead QA - Agentic AI

Sycamore Informatics Inc.; India; On-site; INR 3,500,000 - 6,000,000; Full time

Job summary

Sycamore Informatics Inc. in India seeks a Senior Agentic AI QA Lead Principal to own the quality strategy for agentic AI platforms. You will define test strategies, validate autonomous behaviors, and drive production readiness.

Lead cross-functional teams, design complex test scenarios for planning, testing for biases and safety, review validation evidence, and mentor QA staff while ensuring risk-based release recommendations.

Qualifications

• Design end-to-end testing strategy for agentic and autonomous AI systems
• Design complex test scenarios for reasoning chains, tool orchestration, and multi-agent collaboration
• Lead testing for hallucinations, bias, ethical risks, and unsafe behaviors
• Validate human-in-the-loop controls, auditability, and traceability
• Review and approve AI test cases, defect reports, and validation evidence
• Partner with engineering, product, and compliance teams on risk mitigation
• Mentor QA teams on agentic AI testing techniques
• Provide release recommendations based on risk-based quality assessments
• Define and conduct any scale/performance testing for AI Agents built as well
• Train Sycamore resources with Agentic AI skills where applicable

Responsibilities

• Define end-to-end testing strategy for agentic and autonomous AI systems
• Design test scenarios for reasoning chains and multi-agent collaboration
• Lead testing for hallucinations, bias, and unsafe behaviors
• Validate human-in-the-loop controls and traceability
• Review AI test cases, defects, and validation evidence
• Collaborate with engineering, product, and compliance on risk mitigation
• Mentor QA teams on agentic AI testing techniques
• Provide release recommendations based on risk assessments
• Conduct scale/performance testing for AI agents
• Train resources in Agentic AI skills

Skills

Agent frameworks

Autonomous workflows

Testing AI systems

API automation

Python

Robot Framework

Telemetry analysis

Governance & guardrails

Education

Bachelor’s degree in CS/EE/Physics/Math

Tools

Robot Framework

APIs

Job description

Experience (in years) - 9 – 15 Years Experience

Education - e.g., Bachelor’s degree, preferably in Computer Science, Electrical Engineering, Physics, Math or any other related discipline. (Education requirements to be entered here)

Summary

The Senior Agentic AI QA Lead Principle owns the quality strategy for Agentic AI platforms. This role focuses on defining AI test strategies, evaluating systemic risks, validating autonomous behaviors, and ensuring production readiness of agentic systems.

• Define end-to-end testing strategy for agentic and autonomous AI systems
• Design complex test scenarios for reasoning chains, tool orchestration, and multi-agent collaboration
• Lead testing for hallucinations, bias, ethical risks, and unsafe behaviors
• Validate human-in-the-loop controls, auditability, and traceability
• Review and approve AI test cases, defect reports, and validation evidence
• Partner with engineering, product, and compliance teams on risk mitigation
• Mentor QA teams on agentic AI testing techniques
• Provide release recommendations based on risk-based quality assessments
• Define and conduct any scale/performance testing for AI Agents built as well
• Train Sycamore resources with Agentic AI skills where applicable

Technical Skills – Advanced Agentic AI Testing

• Deep understanding of agent frameworks and orchestration layers
• Expertise in testing autonomous workflows and decision policies
• Experience evaluating planning failures, looping behaviors, and dead-ends
• Advanced prompt evaluation, adversarial testing, and red-teaming
• Strong experience testing API-driven and distributed agent architectures
• Ability to design evaluation metrics for accuracy, reliability, and safety
• Experience reviewing logs, traces, and telemetry for agent behavior analysis
• Knowledge of AI governance, model risk management, and guardrails
• Exposure to automation (Python, Robot Framework, API automation

Deep-Dive Testing Experience (Senior)

• Designing AI test strategies under uncertainty and non-determinism
• Validating autonomous escalation and self-correction mechanisms
• Testing long-running agents and stateful workflows
• Defining acceptance criteria for probabilistic outputs
• Assessing business and compliance risks of agent decisions
• Establishing testing standards and review processes for AI systems

Desired Experience

• Experience defining AI validation approaches under CSA or GxP
• Automation framework ownership or strategy experience
• Experience in AI governance, audits, or regulatory inspections
• Working with Any LLMs like Claude with aws bedrock