SDET – AI Tools & Test Automation

Anthrobyte; Hyderabad; Sur place; INR 1 200 000 - 1 800 000; Plein temps

Résumé du poste

Anthrobyte is seeking an experienced SDET/automation engineer to ensure AI systems in production meet reliability and safety standards. You will design tests for regression suites, guardrails, and evaluation sets, and build reusable frameworks to accelerate future engagements.

You will mentor teams and own quality gates in the CI/CD pipeline, shaping how we test enterprise AI in real-world data. You’ll work in a small, high-trust team, collaborating with engineering and security stakeholders to

Qualifications

• 7–10 years in software testing, with SDET/automation background.
• Deep automation expertise: Python, Pytest, API testing (Postman).
• Production testing of ML/LLM systems, RAG pipelines, and vector workflows.

Responsabilités

• Define quality for AI in production and set criteria for ship-ready systems.
• Build benchmarks for latency, throughput, and retrieval quality across pipelines.

Connaissances

Python

Pytest

Postman

Playwright

Cypress

Selenium

Locust

Gatling

k6

JMeter

CI/CD

GitHub Actions

Docker

AWS

Azure

LLM testing

Outils

Jest

SpecFlow

pytest-bdd

behave

Description du poste

Anthrobyte wins by putting AI agents into the real operations of enterprise clients — and making them work, in production, where it matters. Production is only trusted when it has been proven. We need an engineer who can prove it: someone who tests AI systems the way they actually behave, not the way a demo suggests, and who builds the tooling that lets the whole team do the same.

This is a build-from-the-front role. You test inside the problem, against the client’s real data and constraints, not from a checklist.

What you’ll own

• Quality for AI in production. You decide what “good enough to ship” means for systems that don’t behave the same way twice. Evaluation sets, regression suites, hallucination and guardrail checks — you design them, run them, and defend the results.
  
• Performance and benchmarking. You build repeatable benchmarks for latency, throughput, and retrieval quality across pipelines, vector search, and agent workflows — and you explain why the numbers moved.
  
• Quality gates in the pipeline. You embed automated tests into CI/CD so nothing reaches a client’s live operations without passing them. The gate is yours, and so is the call when it should stay shut.
  
• The product loop. You turn what you learn in the field into reusable test frameworks, benchmarks, and standards inside AATLAS — so every engagement makes the next one faster and safer, and we compound into a product, not a services treadmill.
  
• Raising the standard. This is the multiplier. You mentor engineers in test automation and performance practice, and codify how we test into a playbook others can run without you. Success is a team that tests the way you do, not you testing everything.
  
• Client trust. You present benchmark and quality findings to engineering leads, client security, and business stakeholders — in architecture reviews, security reviews, and the room where decisions get made — and you hold your position when the answer is “not ready yet.”
  

Requirements

What we’re looking for

• Production track record. 7–10 years in software testing, at least 5 of them as an SDET or automation engineer. You’ve built test frameworks from scratch and lived with what you shipped.
  
• Core automation, with depth. Expert in Python and Pytest — framework design, fixtures, parametrization, reporting — and in Postman, with API suites automated in CI. Production experience in at least one UI framework: Playwright, Cypress, or Selenium.
  
• Performance engineering. Hands-on with Locust, Gatling, k6, or JMeter, and able to interpret results rather than just generate them.
  
• Delivery plumbing. CI/CD in GitLab, GitHub Actions, or Azure DevOps. Docker. At least one cloud platform (AWS or Azure). Backend and database testing.
  
• Range. You’re equally comfortable writing code, designing a test strategy, and explaining a failure to a non-technical stakeholder.
  
• Systems mindset. You think in reliability, not test counts. Enterprise AI is won on auditability and things that keep working. Bonus points
  
• Experience testing ML and LLM systems, RAG pipelines, vector databases, embeddings, or MCP-based agent workflows.
  
• Familiarity with LLM evaluation approaches and tooling.
  
• A second UI framework, Jest, or BDD tooling (SpecFlow, pytest-bdd, behave).
  
• Accessibility and security testing (axe, OWASP ZAP).
  
• Experience in regulated or enterprise environments such as finance, healthcare, or telecom.
  
• A track record of growing junior engineers into independent operators.
  

What this role is not

A manual tester working from scripts. A pure automation engineer who runs suites someone else designed. A researcher who benchmarks in isolation and never sees a client’s reality. If you want to test from a distance rather than from inside the problem, this isn’t it.

How we work

Small, high-trust, founder-close team. Governance-first, because our work runs inside clients’ live operations. You’ll have real authority over what “ready” means, and the mandate to build the testing capability we’ll become known for.