AI-Native QA Engineer: Automate AI Workflows

Insight Global; Rosemont (IL); On-site; USD 110,000 - 140,000; Full time

Job summary

Insight Global seeks a Quality Engineer, AI-Native to validate AI-driven workflows, agent behavior, and prompt outputs in a non-deterministic environment. You will craft automated test strategies, test database interactions, and ensure accuracy across AI-enabled processes.

Collaboration with engineering teams is essential to maintain reliability as prompts evolve. The role emphasizes testing AI/LLM systems, familiarity with LangChain/LangGraph/LangSmith, and strong communication of QA risks.

Qualifications

• Strong QA/SDET background with automation experience.
• Experience with API and UI testing.
• Hands-on Playwright or similar modern QA automation frameworks.
• Ability to validate complex AI/agentic systems and non-deterministic outputs.
• Experience testing database interactions and backend workflows.
• Understanding of prompt engineering and AI system behavior.

Responsibilities

• Test and validate AI/agentic systems and business workflows.
• Evaluate prompt outputs for consistency, reliability, and decision accuracy.
• Build and execute automated test strategies for probabilistic systems.
• Validate database interactions, API flows, and agent behaviors across workflows.
• Detect inconsistencies caused by prompt or workflow changes.
• Measure deviations in system performance and support regression testing efforts.
• Partner with engineering teams to ensure AI-enabled workflows meet quality targets.
• Help define scalable QA approaches for testing prompts and agent behaviors at scale.

Skills

QA/SDET automation

API testing

UI testing

Playwright

AI/LLM testing

Data validation

Statistical thinking

Communication on QA

Ambiguity handling

Tools

Playwright

LangChain

LangGraph

LangSmith

Job description

Insight Global seeks a Quality Engineer, AI-Native to validate AI-driven workflows, agent behavior, and prompt outputs in a non-deterministic environment. You will craft automated test strategies, test database interactions, and ensure accuracy across AI-enabled processes.

Collaboration with engineering teams is essential to maintain reliability as prompts evolve. The role emphasizes testing AI/LLM systems, familiarity with LangChain/LangGraph/LangSmith, and strong communication of QA risks.