QA Engineer

WebSenor Ltd; United States; Remote; USD 70,000 - 110,000; Full time

Job summary

WebSenor Ltd is seeking a QA Engineer for their AI platform team in a remote capacity. The successful candidate will be proactive and hands-on, owning test coverage to ensure AI-generated outputs are reliable and ready for real-world use.

Responsibilities include writing automated test suites, testing agent orchestration systems, and validating AI outputs. The position requires strong programming skills in Python and experience with testing tools like Playwright.

Salary is as per industry standards, and the role is ideal for those with a passion for quality assurance in an innovative environment.

Qualifications

• 3+ years of QA Engineering experience.
• Strong programming skills in Python.
• Hands-on experience with Playwright and Vitest.

Responsibilities

• Write and maintain automated test suites.
• Test agent orchestration systems and governance workflows.
• Validate AI-generated outputs for accuracy and reliability.

Skills

QA Engineering experience

Programming skills in Python

Hands-on experience with Playwright

Experience with AI/LLM applications

Understanding of API testing

Problem-solving skills

Tools

FastAPI

PostgreSQL

Docker

React

Job description

We’re Hiring: QA Engineer (AI Platform) – Remote

Job Title: QA Engineer
Work Mode: Remote
Shift: 7:30 PM IST – 4:30 AM IST
Experience: 3+ Years

About the Role

We are looking for a proactive and hands‑on QA Engineer to join our AI platform team.

This is an early‑stage role where you will work closely with the Principal QA Engineer, owning test coverage end‑to‑end and ensuring AI‑generated outputs are reliable, accurate, and ready for real‑world business use.

Key Responsibilities

A. Core Testing & Automation

• Write and maintain automated test suites (unit, integration, end‑to‑end)
• Own test domains from design to execution and regression coverage
• Build evaluation pipelines for AI/LLM outputs including prompt testing and quality scoring

B. AI & Agent Testing

• Test agent orchestration systems and governance workflows
• Detect model drift and prompt sensitivity issues
• Validate AI‑generated outputs for accuracy and reliability

C. Platform & Integration Testing

• Test data ingestion pipelines across systems (CRM, email, documents, financial data)
• Validate file ingestion (Word, Excel, PowerPoint, PDF) including edge cases
• Test streaming responses, latency, and fallback mechanisms

D. Data & System Validation

• Validate knowledge graph outputs and data retrieval accuracy
• Ensure system performance under real enterprise data conditions

Required Skills

• 3+ years of QA Engineering experience
• Strong programming skills in Python
• Hands‑on experience with Playwright and Vitest
• Experience testing AI/LLM‑based applications
• Strong understanding of API testing, async systems, and workflows
• Ability to independently own testing responsibilities
• Strong analytical and problem‑solving skills

Tech Stack Exposure

• AI/LLM: Anthropic Claude, OpenAI GPT, multi‑model routing
• Backend: Python (FastAPI), async systems
• Data: Memgraph, Neo4j, Qdrant, PostgreSQL, Redis
• Frontend: React, Next.js, TypeScript
• Cloud: Google Cloud Platform, Azure
• DevOps: GitHub Actions, Docker
• Testing: Playwright, Vitest

Nice to Have

• Experience with LLM evaluation frameworks (LangSmith, PromptFlow, etc.)
• Exposure to agent‑based systems or orchestration frameworks
• Experience in regulated industries (finance, healthcare, etc.)

Salary

As per industry standards