Senior AI Quality & Reliability Engineer

Vizient, Inc; Chicago (IL); On-site; USD 102,000 - 179,000; Full time

Job summary

Vizient, Inc. seeks an experienced AI Quality Engineer to lead hands-on design, development, and execution of AI Quality Engineering initiatives supporting Vizient's enterprise AI transformation.

In this role you will design AI‑native testing, validation processes, and scalable frameworks, mentor quality engineers, and collaborate with AI Engineering, Clinical, Data, Security, and Product teams to ensure responsible deployment within healthcare workflows.

Qualifications

• Bachelor's degree in Computer Science with 5+ years in Quality Engineering.
• Advanced degree in CS or Engineering highly preferred.
• 5+ years experience in Quality Engineering, QA, software testing, or enterprise delivery.
• Experience providing technical leadership for testing initiatives, automation programs, or enterprise technology delivery.
• Experience with AI Quality Engineering, AI‑assisted testing, observability, and reliability engineering preferred.
• Healthcare technology or regulated industries experience is a plus.

Responsibilities

• Design, develop, and execute AI Quality Engineering strategies for AI-powered applications and enterprise AI platforms.
• Build scalable AI Quality Engineering practices, including AI native testing, validation, runtime quality controls, and automated testing frameworks.
• Lead AI validation activities including functional, prompt, workflow, regression, and release validation.
• Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, Data, and Product teams to scale QA processes across enterprise AI initiatives.
• Support runtime reliability through observability, telemetry, distributed tracing, monitoring, drift detection, incident response, and production quality assurance for AI-enabled systems.
• Develop AI evaluation datasets, quality scoring methodologies, and automated testing workflows.
• Collaborate with Clinical, Operational, and Engineering stakeholders to validate healthcare workflows and responsible AI deployment.
• Coordinate testing across Agile delivery teams and manage defects, releases, and production support.
• Mentor Quality Engineers to promote best practices and continuous improvement in AI Quality Engineering.
• Research and recommend emerging AI Quality Engineering technologies to improve enterprise AI quality capabilities.

Skills

Analytical skills

Problem-solving

Leadership

Communication

Collaboration

Education

Bachelor's degree in Computer Science

Advanced degree in CS/Engineering

Job description

When you’re the best, we’re the best. We instill an environment where employees feel engaged, satisfied and able to contribute their unique skills and talents while living and working as their authentic selves. We provide extensive opportunities for personal and professional development, building both employee competence and organizational capability to fuel exceptional performance through an inclusive environment both now and in the future.

Summary In this role, you will drive the hands‑on design, development, and execution of AI Quality Engineering initiatives supporting Vizient's enterprise AI transformation efforts.

You will design and implement AI Quality Engineering practices, AI validation processes, AI‑assisted testing approaches, runtime quality controls, and scalable testing frameworks that support the responsible deployment of AI‑powered business solutions.

This role combines hands‑on quality engineering, AI‑enabled testing modernization, healthcare workflow validation, technical mentorship, and cross‑functional collaboration.

You will help advance Vizient's Quality Engineering capabilities beyond traditional software testing toward AI‑native validation, AI‑assisted testing, runtime observability, reliability engineering, and modern AI Quality Engineering practices.

Responsibilities

• Design, develop, and execute AI Quality Engineering strategies supporting AI‑powered applications, large language model (LLM) solutions, intelligent automation, agentic systems, and enterprise AI platforms.
• Build and implement scalable AI Quality Engineering practices, including AI‑native testing approaches, validation processes, runtime quality controls, reusable testing accelerators, and automated testing frameworks.
• Lead AI validation activities including functional testing, prompt validation, workflow testing, regression testing, release validation, runtime quality assurance, and production reliability support.
• Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, Data, and Product teams to deliver scalable AI Quality Engineering processes across enterprise AI initiatives.
• Support runtime reliability through observability, telemetry, distributed tracing, monitoring, drift detection, incident response, and operational quality assurance for AI‑enabled systems.
• Develop and maintain AI evaluation frameworks, validation datasets, quality scoring methodologies, and automated testing workflows that improve the reliability and scalability of AI solutions.
• Collaborate with Clinical, Operational, and Engineering stakeholders to validate healthcare workflows, payer operations, and AI‑enabled business processes while supporting responsible AI deployment through human‑in‑the‑loop validation practices.
• Coordinate testing activities across Agile delivery teams, including sprint planning, test execution, defect management, issue tracking, release readiness, risk identification, and production support.
• Mentor Quality Engineers and provide technical guidance that promotes engineering excellence, AI‑enabled testing modernization, continuous improvement, and adoption of modern Quality Engineering practices.
• Research, evaluate, and recommend emerging AI Quality Engineering, testing automation, observability, and runtime assurance technologies to continuously improve enterprise AI quality capabilities.

Qualifications

• Relevant degree preferred.
• Advanced degree in Computer Science, Engineering, or a related field highly preferred.
• Bachelor's degree in Computer Science, 5 or more years of experience in Quality Engineering, Quality Assurance, software testing, enterprise application delivery, technology operations, or related technology functions required.
• 2 or more years of experience providing technical leadership for testing initiatives, automation programs, or enterprise technology delivery projects required.
• Experience supporting Quality Engineering or Quality Assurance across enterprise platforms, APIs, healthcare applications, operational workflows, or integrated business systems required.
• Strong knowledge of software development life cycle (SDLC), Agile methodologies, test automation frameworks, defect management, release validation, and production support processes required.
• Experience validating AI‑powered applications, intelligent automation, machine learning, large language model (LLM), or AI‑enabled business workflows preferred.
• Experience with AI Quality Engineering practices, AI‑assisted testing, runtime observability, monitoring, telemetry, or reliability engineering preferred.
• Strong analytical, problem‑solving, organizational, communication, collaboration, and leadership skills required.
• Demonstrated ability to manage multiple priorities and deliver results within fast‑paced, highly collaborative enterprise environments required.
• Experience in healthcare technology, payer operations, clinical workflows, or other regulated industries supporting AI governance and responsible AI deployment preferred.

Estimated Hiring Range

#LI-JB1 Estimated Hiring Range: At Vizient, we consider skills, experience, and organizational needs in our compensation approach. Geographic factors may adjust the range estimate and hires typically fall below the top range. Compensation decisions are tailored to individual circumstances. The current salary range for this role is $102,400.00 to $179,000.00. This position is also incentive eligible. Vizient has a comprehensive benefits plan! Please view our benefits here: http://www.vizientinc.com/about-us/careers

Equal Opportunity Employer

Females/Minorities/Veterans/Individuals with Disabilities

The Company is committed to equal employment opportunity to all employees and applicants without regard to race, religion, color, gender identity, ethnicity, age, national origin, sexual orientation, disability status, veteran status or any other category protected by applicable law.

Working at Vizient means shaping the future of healthcare, every day. Alongside our clients, we're building a healthier, more human healthcare system. We connect people, data, and solutions to enable healthcare organizations to improve financial sustainability, enhance system performance, and deliver better care.

Headquartered in Irving, Texas, with 11 offices across the United States, Vizient has more than 5,000 employees who bring deep expertise and a shared commitment to transforming healthcare. Together, we're solving some of the industry's most complex challenges while creating opportunities to learn, grow, and drive lasting change for healthcare organizations and the communities they serve.

Through our enterprise of companies, we deliver meaningful impact across the continuum of care

• Apexus - The exclusive contractor for 340B Drug Pricing Program, managed by the Health Resources and Services Administration.
• aptitude - The industry's first online direct contracting market, supporting self-contracting activity between hospitals and suppliers.
• Kaufman Hall - Leading healthcare advisory services to support our clients in achieving financial, clinical, and operational excellence.
• Provista - Proven supply chain partner specializing in extended, diagnostic, and preventive care facilities, including surgery centers, home health, and senior living.