AI QA ENGINEER

Opkey; Bengaluru; On-site; INR 600,000 - 900,000; Full time

Job summary

Opkey is seeking an AI | QA Engineer to join our ERP automation team in Bengaluru, India. In this role, you will collaborate closely with AI developers to ensure the quality of our AI-powered agents, which are critical for enterprise ERP workflows.

The ideal candidate will have 2–4 years of experience in QA Engineering, a strong grasp of SDLC/STLC methodologies, and proficiency in Python for automation tasks. Experience with AI agents and automation tools is essential.

Opkey offers a dynamic work environment with opportunities to work at the forefront of AI technology.

Qualifications

• 2–4 years of hands-on experience as a QA Engineer in software or automation products.
• Strong understanding of SDLC and STLC methodologies.
• Experience testing API-based systems, microservices, and complex backend workflows.

Responsibilities

• Design, develop, and execute test strategies for AI agents used in automation pipelines.
• Validate the behavior and reliability of AI agent outputs.
• Collaborate with AI developers to understand agent architecture.

Skills

QA Engineering

Python

AI Agents

Test Automation

SDLC/STLC Methodologies

Agile/Scrum

Root Cause Analysis

ERP Platforms

Education

Certification in AI/ML fundamentals

Certification in Generative AI or LLM-related technologies

Tools

Selenium

Pytest

Postman

Streamlit

Jira

Job description

Opkey is the leading Cloud Application Lifecycle Management (CALM) platform for Oracle, Workday, Salesforce, Coupa, and more. It cuts costs and risks that drag down implementations and ongoing change, helping you go live on time, get more from your cloud app investments, and reach AI readiness faster. Opkey’s 20+ AI agents manage all five phases of the cloud application lifecycle—Define, Design, Configure, Test, and Train.

Job Title

AI | QA Engineer

Location

Bangalore, India (On-site)

Department

AI & Automation

Overview

We are looking for a skilled AI QA Engineer to join our ERP automation team at Opkey. You will work closely with AI developers to test, validate, and ensure the quality of AI-powered agents built for enterprise ERP workflows.

Key Responsibilities

• Design, develop, and execute test strategies for AI agents used in ERP automation pipelines
• Validate the behavior, accuracy, and reliability of AI agent outputs across various ERP scenarios
• Collaborate with AI developers to understand agent architecture and identify edge cases
• Build and maintain automated test frameworks tailored for LLM-based and agentic systems
• Perform functional, regression, integration, and performance testing on AI-driven workflows
• Monitor AI agent performance post-deployment and flag anomalies or drift in behavior
• Write detailed test plans, bug reports, and QA documentation specific to AI systems
• Conduct prompt testing and evaluate AI responses for correctness, consistency, and safety
• Build and maintain internal QA dashboards and reporting tools for real-time test visibility
• Participate in Agile/Scrum ceremonies including sprint planning, standups, retrospectives, and backlog grooming as the QA representative
• Conduct thorough Root Cause Analysis (RCA) on defects, test failures, and AI agent misbehaviors and present findings to the team
• Follow and contribute to SDLC and STLC processes ensuring quality gates are met at every stage of development

Required Skills & Experience

• 2–4 years of hands‑on experience as a QA Engineer in software or automation products
• Strong understanding of SDLC and STLC methodologies with experience applying them in fast‑paced product environments
• Hands‑on experience working in Agile/Scrum teams with familiarity in tools like Jira or Azure DevOps
• Proficiency in Python for writing test scripts, automation utilities, and data validators
• Experience building lightweight internal tools or test dashboards using Streamlit
• Solid understanding of AI agents, LLMs, RAG pipelines, and agentic frameworks (LangChain, AutoGen, CrewAI, etc.)
• Experience testing API‑based systems, microservices, and complex backend workflows
• Proficiency with test automation tools such as Selenium, Pytest, Postman, or similar
• Proven ability to perform Root Cause Analysis on complex bugs and systemic failures with clear documentation
• Familiarity with ERP platforms (SAP, Oracle, Salesforce, etc.) is a strong plus
• Knowledge of prompt engineering and how to test AI model behavior systematically
• Experience with CI/CD pipelines and integrating automated tests into DevOps workflows

AI Certifications (Required / Preferred)

• Certification in AI/ML fundamentals — Google, Microsoft (Azure AI), AWS, or IBM preferred
• Certification in Generative AI or LLM‑related technologies (e.g., DeepLearning.AI, Coursera, Udacity)
• Any recognized certification in AI Agent development or Agentic frameworks is a strong plus