Staff Quality Engineer - AI/LLM Testing & Automation

Talener; Hoboken (NJ); Hybrid; USD 130,000 - 150,000; Full time

Job summary

Talener is seeking a Staff Quality Engineer to drive QA for AI-powered products and platforms in production. You will focus on testing LLM-based applications, agentic AI systems, and RAG workflows, with strong automation using Playwright and TypeScript.

The role requires 8+ years in Quality Engineering, hands-on AI testing experience in production, and leadership in Agile environments. The position is based in Hoboken, NJ, offering hybrid work (3 days onsite) and a competitive base salary with

Qualifications

• 8+ years of Quality Engineering or Software QA experience.
• Hands-on testing of AI/LLM-powered applications in production environments.
• Experience testing agentic AI systems, multi-agent workflows, and RAG pipelines.
• Strong understanding of AI evaluation methodologies including RAGAS, LLM-as-Judge, regression baselines, and non-deterministic testing.
• Strong automation experience using Playwright or similar testing frameworks.
• Proficiency with TypeScript and/or Java.
• Experience with API testing, backend validation, and test data management.
• Experience building and maintaining automated test frameworks across UI, API, and backend layers.
• Experience designing and executing functional, regression, integration, and end-to-end testing strategies.
• Experience integrating automated testing into CI/CD pipelines.
• Experience leading quality efforts for complex software projects within Agile environments.

Responsibilities

• Develop and execute QA strategies for AI-powered platforms.
• Lead quality efforts for complex software projects in Agile environments.
• Integrate automated testing into CI/CD pipelines across UI, API, and backend layers.

Skills

Quality Engineering

AI/LLM testing

RAG pipelines

Playwright

TypeScript

Java

API testing

CI/CD integration

Agile leadership

Test automation design

Education

Bachelor's degree in CS/Engineering

Tools

Playwright

API testing tools

Job description

Talener is seeking a Staff Quality Engineer to drive QA for AI-powered products and platforms in production. You will focus on testing LLM-based applications, agentic AI systems, and RAG workflows, with strong automation using Playwright and TypeScript.

The role requires 8+ years in Quality Engineering, hands-on AI testing experience in production, and leadership in Agile environments. The position is based in Hoboken, NJ, offering hybrid work (3 days onsite) and a competitive base salary with