Senior AI Quality Engineer - GenAI & Observability

Wolters Kluwer - Financial Services Solutions; Cape Girardeau (MO); On-site; USD 120,000 - 150,000; Full time

Job summary

Wolters Kluwer - Financial Services Solutions seeks a quality engineering lead for enterprise-scale generative AI applications. You will own the AI quality framework, from RAG pipelines to LLM orchestration and AI-driven user experiences.

You will collaborate with developers to validate system architecture, observability, and production readiness while driving responsible AI practices. The role emphasizes building automated regression suites, harnesses for evaluation, and measurable gates for AI

Responsibilities

• Own and implement the quality engineering approach for enterprise-scale generative AI applications.
• Architect evaluation harnesses to measure retrieval accuracy, citation correctness, and latency.
• Design automated regression suites to detect quality drift across prompts and model versions.
• Lead root-cause analysis for AI quality failures using logs, traces, and evaluation outputs.
• Develop test automation in Python for APIs, services, and data pipelines within CI/CD.
• Build observability dashboards tracking model behavior and operational health.

Job description

Wolters Kluwer - Financial Services Solutions seeks a quality engineering lead for enterprise-scale generative AI applications. You will own the AI quality framework, from RAG pipelines to LLM orchestration and AI-driven user experiences.

You will collaborate with developers to validate system architecture, observability, and production readiness while driving responsible AI practices. The role emphasizes building automated regression suites, harnesses for evaluation, and measurable gates for AI