Senior QA Engineer — Conversational AI (Contract)

Florida Blue; United States; On-site; USD 120,000 - 160,000; Full time

Job summary

Florida Blue, in partnership with Magnit, seeks a Senior QA Engineer to lead testing of conversational AI applications in a healthcare context. You will design test strategies, validate prompts, and build automation for APIs and UI.

You will collaborate with AI/ML teams to address non-deterministic outputs, hallucinations and bias, and ensure HIPAA-aligned security testing. This contract role offers exposure to GuideWell affiliates.

Qualifications

• 5+ years of QA experience, with AI/ML testing experience preferred.
• Experience designing, implementing, and maintaining automated test suites.
• Knowledge of API testing and CI/CD pipelines.
• Proficiency in at least one programming language (Python/Java/JavaScript).
• Understanding of AI concepts, LLMs, NLP and failure modes.

Responsibilities

• Lead QA efforts for conversational AI applications and ensure reliability, accuracy, and security.
• Design and implement comprehensive test strategies and edge-case scenarios for AI behavior.
• Develop and maintain automated test suites for AI workflows, including API and UI testing.
• Validate prompts and prompt injection/safety considerations with AI/ML teams.
• Document defects and quality metrics; communicate results to stakeholders.

Skills

QA testing

AI/ML testing

Test automation

API testing

CI/CD

Prompt engineering

Python/Java/JS

Education

Bachelor's Degree in IT or related field

Tools

Selenium

Cypress

Postman

REST Assured

Job description

Florida Blue, in partnership with Magnit, seeks a Senior QA Engineer to lead testing of conversational AI applications in a healthcare context. You will design test strategies, validate prompts, and build automation for APIs and UI.

You will collaborate with AI/ML teams to address non-deterministic outputs, hallucinations and bias, and ensure HIPAA-aligned security testing. This contract role offers exposure to GuideWell affiliates.