Senior Quality Engineer

Talener; Hoboken (NJ); Hybrid; USD 130,000 - 150,000; Full time

Job summary

Talener is seeking a Staff Quality Engineer to drive QA for AI-powered products and platforms in production. You will focus on testing LLM-based applications, agentic AI systems, and RAG workflows, with strong automation using Playwright and TypeScript.

The role requires 8+ years in Quality Engineering, hands-on AI testing experience in production, and leadership in Agile environments. The position is based in Hoboken, NJ, offering hybrid work (3 days onsite) and a competitive base salary with

Qualifications

• 8+ years of Quality Engineering or Software QA experience.
• Hands-on testing of AI/LLM-powered applications in production environments.
• Experience testing agentic AI systems, multi-agent workflows, and RAG pipelines.
• Strong understanding of AI evaluation methodologies including RAGAS, LLM-as-Judge, regression baselines, and non-deterministic testing.
• Strong automation experience using Playwright or similar testing frameworks.
• Proficiency with TypeScript and/or Java.
• Experience with API testing, backend validation, and test data management.
• Experience building and maintaining automated test frameworks across UI, API, and backend layers.
• Experience designing and executing functional, regression, integration, and end-to-end testing strategies.
• Experience integrating automated testing into CI/CD pipelines.
• Experience leading quality efforts for complex software projects within Agile environments.

Responsibilities

• Develop and execute QA strategies for AI-powered platforms.
• Lead quality efforts for complex software projects in Agile environments.
• Integrate automated testing into CI/CD pipelines across UI, API, and backend layers.

Skills

Quality Engineering

AI/LLM testing

RAG pipelines

Playwright

TypeScript

Java

API testing

CI/CD integration

Agile leadership

Test automation design

Education

Bachelor's degree in CS/Engineering

Tools

Playwright

API testing tools

Job description

Our client is seeking a Staff Quality Engineer (Playwright/Typescript/AI - LLMs) to support the quality and validation of AI-powered products and platforms. This role focuses on testing LLM-based applications, agentic AI systems, and Retrieval-Augmented Generation (RAG) workflows in production environments.

Job Title: Staff Quality Engineer (AI & Automation)

Salary: $130,000 - $150,000 Base + 15% Bonus

Location: Hoboken, NJ (Hybrid | 3 Days Per Week Onsite)

Required Skills & Experience

• 8+ years of Quality Engineering or Software Quality Assurance experience.
• Hands-on experience testing AI/LLM-powered applications in production environments.
• Experience testing agentic AI systems, multi-agent workflows, and RAG pipelines.
• Strong understanding of AI evaluation methodologies including RAGAS, LLM-as-Judge, regression baselines, and non-deterministic testing.
• Strong automation experience using Playwright or similar testing frameworks.
• Proficiency with TypeScript and/or Java.
• Experience with API testing, backend validation, and test data management.
• Experience building and maintaining automated test frameworks across UI, API, and backend layers.
• Experience designing and executing functional, regression, integration, and end-to-end testing strategies.
• Experience integrating automated testing into CI/CD pipelines.
• Experience leading quality efforts for complex software projects within Agile environments.

Preferred Qualifications

• Experience with vector databases, embeddings, or prompt engineering.
• AWS or cloud platform experience.
• Experience with AI-assisted development tools such as Cursor or GitHub Copilot.
• Experience with AI observability, monitoring, or evaluation platforms.
• Bachelor's degree in Computer Science, Engineering, or a related technical field.