Lead QA - Agentic AI

Sycamore Informatics Inc.; India; On-site; INR 450,000 - 900,000; Full time

Job summary

Sycamore Informatics Inc. seeks a Senior Agentic AI QA Lead Principle to define AI test strategies, validate autonomous behaviors, and ensure production readiness of agentic systems.

You will design end-to-end tests for multi-agent collaboration, review risk controls, mentor QA teams, and collaborate with engineering, product and compliance to mitigate risks.

Qualifications

• Experience defining AI validation approaches for agentic systems.
• Experience in testing autonomous AI workflows and decision policies.
• Ability to design end-to-end AI test strategies and metrics.
• Experience auditing logs, traces, and telemetry for agent behavior.
• Familiarity with AI governance and risk management.

Responsibilities

• Define end-to-end testing strategy for agentic and autonomous AI systems.
• Design complex test scenarios for reasoning chains, tool orchestration, and multi-agent collaboration.
• Lead testing for hallucinations, bias, ethical risks, and unsafe behaviors.
• Validate human-in-the-loop controls, auditability, and traceability.
• Review and approve AI test cases, defect reports, and validation evidence.
• Partner with engineering, product, and compliance teams on risk mitigation.
• Mentor QA teams on agentic AI testing techniques.
• Provide release recommendations based on risk-based quality assessments.
• Define and conduct scale/performance testing for AI Agents built as well.
• Train Sycamore resources with Agentic AI skills where applicable.

Skills

Agentic AI testing

QA leadership

Test strategy design

Performance testing

Log analysis

Python automation

Education

Bachelor's degree

Tools

Python

Robot Framework

API automation

Job description

Experience

Experience (in years) - 9 – 15 Years Experience

Education

Education - e.g., Bachelor’s degree, preferably in Computer Science, Electrical Engineering, Physics, Math or any other related discipline. (Education requirements to be entered here)

Summary

The Senior Agentic AI QA Lead Principle owns the quality strategy for Agentic AI platforms. This role focuses on defining AI test strategies, evaluating systemic risks, validating autonomous behaviors, and ensuring production readiness of agentic systems.

• Define end-to-end testing strategy for agentic and autonomous AI systems
• Design complex test scenarios for reasoning chains, tool orchestration, and multi-agent collaboration
• Lead testing for hallucinations, bias, ethical risks, and unsafe behaviors
• Validate human-in-the-loop controls, auditability, and traceability
• Review and approve AI test cases, defect reports, and validation evidence
• Partner with engineering, product, and compliance teams on risk mitigation
• Mentor QA teams on agentic AI testing techniques
• Provide release recommendations based on risk-based quality assessments
• Define and conduct any scale/performance testing for AI Agents built as well
• Train Sycamore resources with Agentic AI skills where applicable

Technical Skills – Advanced Agentic AI Testing

• Deep understanding of agent frameworks and orchestration layers
• Expertise in testing autonomous workflows and decision policies
• Experience evaluating planning failures, looping behaviors, and dead-ends
• Advanced prompt evaluation, adversarial testing, and red-teaming
• Strong experience testing API-driven and distributed agent architectures
• Ability to design evaluation metrics for accuracy, reliability, and safety
• Experience reviewing logs, traces, and telemetry for agent behavior analysis
• Knowledge of AI governance, model risk management, and guardrails
• Exposure to automation (Python, Robot Framework, API automation

Deep-Dive Testing Experience (Senior)

• Designing AI test strategies under uncertainty and non-determinism
• Validating autonomous escalation and self-correction mechanisms
• Testing long-running agents and stateful workflows
• Defining acceptance criteria for probabilistic outputs
• Assessing business and compliance risks of agent decisions
• Establishing testing standards and review processes for AI systems

Desired Experience

• Experience defining AI validation approaches under CSA or GxP
• Automation framework ownership or strategy experience
• Experience in AI governance, audits, or regulatory inspections
• Working with Any LLMs like Claude with aws bedrock