Senior AI QA Engineer for Sora Union

Sora Union

United States

Remote

USD 120,000 - 170,000

Full time

27 hours ago

Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Sora Union Group is a professional services company specializing in brand design and engineering services, with a globally distributed team. We seek an experienced QA engineer to design and execute tests for AI/LLM outputs, build testing agents, and validate MCP tools and integrations.

You will develop test plans, regression suites, and automated tests for web apps, APIs, and end-to-end workflows to ensure data integrity and release confidence.

Qualifications

Degree in computer science or mathematics.
7+ years of professional QA experience.
Hands-on AI/LLM testing experience.
Experience with Claude, ChatGPT, or similar AI tools for testing.
Experience with Playwright and MCP-based automation.
Working knowledge of REST APIs and SQL.
Familiarity with Jira, TestRail, Azure DevOps, or GitHub.
Understanding of Agile/Scrum methodologies.
English upper-intermediate or higher.

Responsibilities

Design and execute tests evaluating LLM-generated responses for accuracy and consistency.
Build and maintain AI-powered testing agents and test harnesses.
Test MCP tools and integrations, including error scenarios.
Develop test plans, regression suites, and manual testing checklists.
Design and maintain automated tests for web apps, APIs, and end-to-end workflows.
Test backend APIs, verify data integrity, and validate system integrations.
Reproduce, document, and triage defects with actionable evidence.
Partner with engineering and product teams to assess risks and improve release confidence.

Skills

AI/LLM testing

QA testing

Automation

Playwright

REST APIs

SQL

Agile/Scrum

English communication

Education

Bachelor's degree in Computer Science or Mathematics

Tools

Playwright

MCP automation

Jira

TestRail

Azure DevOps

GitHub

Job description

Responsibilities

Design and execute tests evaluating LLM-generated responses for accuracy and consistency
Build and maintain AI-powered testing agents and test harnesses
Test MCP tools and integrations, including error scenarios
Develop test plans, regression suites, and manual testing checklists
Design and maintain automated tests for web apps, APIs, and end-to-end workflows
Test backend APIs, verify data integrity, and validate system integrations
Reproduce, document, and triage defects with actionable evidence
Partner with engineering and product teams to assess risks and improve release confidence

Requirements