Mid level Quality Engineer, AI-Native

Insight Global; Rosemont (IL); On-site; USD 110,000 - 140,000; Full time

Job summary

Insight Global seeks a Quality Engineer, AI-Native to validate AI-driven workflows, agent behavior, and prompt outputs in a non-deterministic environment. You will craft automated test strategies, test database interactions, and ensure accuracy across AI-enabled processes.

Collaboration with engineering teams is essential to maintain reliability as prompts evolve. The role emphasizes testing AI/LLM systems, familiarity with LangChain/LangGraph/LangSmith, and strong communication of QA risks.

Qualifications

• Strong QA/SDET background with automation experience.
• Experience with API and UI testing.
• Hands-on Playwright or similar modern QA automation frameworks.
• Ability to validate complex AI/agentic systems and non-deterministic outputs.
• Experience testing database interactions and backend workflows.
• Understanding of prompt engineering and AI system behavior.

Responsibilities

• Test and validate AI/agentic systems and business workflows.
• Evaluate prompt outputs for consistency, reliability, and decision accuracy.
• Build and execute automated test strategies for probabilistic systems.
• Validate database interactions, API flows, and agent behaviors across workflows.
• Detect inconsistencies caused by prompt or workflow changes.
• Measure deviations in system performance and support regression testing efforts.
• Partner with engineering teams to ensure AI-enabled workflows meet quality targets.
• Help define scalable QA approaches for testing prompts and agent behaviors at scale.

Skills

QA/SDET automation

API testing

UI testing

Playwright

AI/LLM testing

Data validation

Statistical thinking

Communication on QA

Ambiguity handling

Tools

Playwright

LangChain

LangGraph

LangSmith

Job description

Job Description

Job Description:Insight Global is seeking a Quality Engineer, AI-Native for a top FinTech / intelligent automation client. This candidate will play a critical role in validating AI-driven workflows, agent behavior, and prompt outputs in an environment where testing goes far beyond traditional QA. They will help ensure that business-critical automation systems behave reliably, even when small changes create large output variation. The ideal candidate brings strong automation testing experience, comfort with ambiguity, and a modern approach to quality engineering in AI-powered systems.

Day-to-Day:

• Test and validate AI/agentic systems and business workflows
• Evaluate prompt outputs for consistency, reliability, and decision accuracy
• Build and execute automated test strategies for probabilistic systems
• Validate database interactions, API flows, and agent behaviors across workflows
• Detect inconsistencies caused by prompt or workflow changes
• Measure deviations in system performance and support regression testing efforts
• Partner with engineering teams to ensure AI-enabled workflows meet accuracy and quality targets
• Help define scalable QA approaches for testing prompts and agent behaviors at scale

Skills and Requirements

• Strong QA/SDET background with automation experience
• Experience with API and UI testing
• Hands‑on experience using Playwright or similar modern QA automation frameworks
• Ability to validate complex systems and test ambiguity/non‑deterministic outputs
• Experience testing database interactions and backend workflows
• Understanding of prompt engineering, prompt output validation, or AI system behavior
• Ability to think statistically about validation, regression, variation, and reliability
• Strong communication skills around testing strategy and quality risks
• Experience testing AI/LLM systems or agentic workflows
• Familiarity with LangChain, LangGraph, or LangSmith
• Experience with model evaluation, prompt testing, or AI quality monitoring at scale
• Background in data validation, complex workflow testing, or intelligent automation
• Understanding of regulated or business‑critical workflows such as underwriting, funding, or support operations

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to [e-mail].