QA Engineer – Automation & Agentic Applications

Beroe Inc; India; Remote; INR 1,000,000 - 1,500,000; Full time

Job summary

A technology solutions company located in India is looking for a QA Engineer with strong automation skills and experience with intelligent applications. In this role, you will develop automated test frameworks, validate agent behavior, and ensure compliance and safety in AI-driven environments. Applicants should have over 5 years in QA Engineering, proficiency in Python, and experience with various testing frameworks. This position supports a hybrid work model.

Qualifications

• 5+ years of experience in QA Engineering with a focus on test automation.
• Proficient in Python for writing test scripts and tools.
• Strong understanding of software testing methodologies.

Responsibilities

• Design, develop, and maintain automated test frameworks and test scripts using Python.
• Integrate automated tests into CI/CD pipelines for continuous quality.
• Conduct root cause analysis of production issues.

Skills

Automation testing

Python

Problem-solving

Agile methodologies

Tools

Pytest

Selenium

Playwright

Postman

Job description

QA Engineer – Automation & Agentic Applications

We are looking for a QA Engineer with strong automation expertise and experience testing intelligent/agentic applications to join our growing engineering team. You will play a critical role in ensuring the quality, safety, and reliability of our platform by developing automated test frameworks, validating agent autonomy and decision-making, and collaborating closely with developers, product owners, and DevOps. This role goes beyond traditional QA: you will be working with AI-driven, agentic applications that can plan, act, and adapt. Your job will be to make sure they behave correctly, safely, and predictably under real-world conditions.

Key Responsibilities

• Design, develop, and maintain automated test frameworks and test scripts using Python.
• Write and execute manual and automated test cases to validate product features, agent behavior, and functionality.
• Integrate automated tests into CI/CD pipelines to ensure continuous quality and fast feedback.
• Collaborate with developers and product managers to identify, reproduce, and resolve defects early in the lifecycle.
• Define and document test plans, strategies, and acceptance criteria for both deterministic and agentic features.
• Perform scenario-based, exploratory, and chaos testing to validate agent decision making under uncertainty, failure, and edge conditions.
• Verify observability and explainability of agent behavior (logs, traces, intermediate actions) to ensure compliance, transparency, and auditability.
• Test human-in-the-loop workflows, ensuring proper escalation, overrides, and user safety.
• Conduct root cause analysis of production issues, including agent misbehavior, and contribute to long-term fixes.
• Contribute to the development and enforcement of QA standards and best practices.

Must-Have Qualifications

• 5+ years of experience in QA Engineering with a focus on test automation.
• Proficient in Python for writing test scripts and tools.
• Hands‑on experience with testing frameworks such as Pytest, Selenium, or Playwright.
• Strong understanding of software testing methodologies, including unit, integration, system, and end‑to‑end testing.
• Familiarity with REST APIs and tools like Postman, Swagger, or Bruno for API testing.
• Solid understanding of Agile/Scrum methodologies.
• Strong problem‑solving skills with the ability to think beyond “happy path” testing.

Nice‑to‑Have Qualifications

• Experience testing agentic or AI‑driven applications (e.g., autonomous decision making, multi‑step workflows)
• Experience with scenario‑based or chaos testing frameworks
• Familiarity with observability and monitoring tools (e.g., Datadog, Grafana, AWS CloudWatch) to validate agent behavior
• Experience testing microservices
• Experience with CI/CD tools like GitHub Actions, Jenkins, or GitLab CI
• Knowledge of security and guardrail testing (role‑based access, safe action validation)

One file only. 5 MB limit. Allowed types: pdf, doc, docx.