SDET 3

Bidgely; India; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Bidgely is seeking a Senior SDET to own quality across customer-facing products including web dashboards, APIs, notifications, and GenAI solutions such as chatbots and AI agents. You will define testing methodologies for AI agents and develop automated evaluations integrated with CI/CD.

This hands-on role collaborates with GenAI engineering, CX engineering, and product teams, building UI and API automation, evaluating AI agents, and mentoring other SDETs.

Qualifications

• 6–9 years of software testing experience with hands-on SDET roles.
• Proficiency in Python and Java.
• Experience owning test strategies and enhancing automation.
• UI automation experience with Selenium or Playwright.
• API automation and end-to-end testing within CI/CD.
• Experience testing chatbots, IVRs, GenAI agents.
• Ability to design evaluations, rubrics, and LLM-as-judge assessments.
• Familiarity with LangChain, LangGraph, LangSmith, or similar tools.
• Strong analytical and communication skills.

Responsibilities

• Own the test strategy and release quality criteria for the product area.
• Build and maintain UI, API, and end-to-end regression suites.
• Design evaluation strategies for AI agents with LLM-as-judge.
• Create evaluation datasets and automated evaluation pipelines in Python.
• Test AI agents for accuracy, consistency, tool usage, safety, and data handling.
• Debug agent failures using LangSmith or similar tools.
• Analyze quality across releases and identify regressions.
• Review requirements to identify gaps in testing coverage.
• Review test code and mentor other SDETs; communicate findings to teams.

Skills

Analytical thinking

Communication skills

Mentoring

Tools

Selenium

Playwright

Python

Java

CI/CD

GitHub Actions

Jenkins

AWS

Postman

LangChain

LangGraph

LangSmith

LLM evaluation

Job description

The Role

We are seeking a Senior SDET to take ownership of quality across our customer-facing products, including web dashboards, APIs, notifications, and GenAI solutions such as chatbots, voice IVR, and AI agents. In this role, you will define testing methodologies for AI agents, develop automated evaluations, and establish quality criteria for releases.

You will be responsible for the test strategy in your area and continually build and enhance UI, API, and end-to-end automation and regression coverage. This is a hands-on engineering position working closely with GenAI engineering, CX engineering, and product teams.

Key Responsibilities

• Own the test strategy and release quality criteria for your product area.
• Build and maintain UI automation (using Selenium or Playwright), API automation, and end-to-end regression test suites.
• Design evaluation strategies for AI agents, utilizing LLM-as-judge, rubrics, and other assessment techniques.
• Create evaluation datasets and automated evaluation suites in Python, integrated with CI/CD pipelines.
• Test AI agents for accuracy, consistency, tool usage, safety, and data handling.
• Debug agent failures using LangSmith or similar AI observability tools.
• Analyze quality across releases to identify regressions.
• Review product requirements to pinpoint gaps in requirements and test coverage.
• Review test code, define testing standards, and mentor other SDETs.
• Communicate findings and recommendations with engineering, product, and leadership teams.

Requirements

• 6–9 years of software testing experience, including hands-on SDET roles.
• Proficiency in both Python and Java.
• Experience owning test strategies and enhancing automation and release quality.
• Experience with UI automation tools such as Selenium or Playwright.
• Experience developing API automation and end-to-end frameworks within CI/CD.
• Ability to analyze product requirements and spot testing gaps.
• Previous experience testing chatbots, IVRs, and GenAI agents.
• Experience designing evaluations, rubrics, and LLM-as-judge assessments.
• Working knowledge of LangChain, LangGraph, LangSmith, or similar tools.
• Experience creating evaluation frameworks or tools for AI agents.
• Strong analytical and communication skills.

Tech You Will Work With

Python, Java, Selenium, Playwright, REST API testing, Postman, TestNG, PyTest, LangChain, LangGraph, LangSmith, LLM evaluation, LLM-as-judge, RAG testing, AI agent testing, end-to-end testing, regression testing, CI/CD, GitHub Actions, Jenkins, AWS.