QA Automation Engineer - GenAI / LLM Testing (Healthcare)

DXC Luxoft; United States; A distancia; USD 110.000 - 170.000; Jornada completa

Descripción de la vacante

Luxoft seeks an experienced AI Quality Engineer to shape quality strategies for AI-enabled applications, platforms, and services in healthcare environments. The role emphasizes automated testing, AI model evaluation, and ensuring safety, reliability, and compliance of GenAI, LLM, and ML solutions across enterprise ecosystems.

You will develop automated testing frameworks, collaborate with product, data science, and engineering teams, and contribute to governance, documentation, and production

Formación

• 3–5+ years of experience in Quality Engineering, Quality Assurance, or Software Testing.
• 3+ years in test automation development and 2+ years testing AI, machine learning, Generative AI, LLM, or intelligent automation solutions.
• Strong experience designing automated test frameworks, validating AI-generated outputs, and performing API, integration, and data testing.

Responsabilidades

• Design and execute comprehensive testing strategies for AI/ML, GenAI, LLM, and intelligent automation solutions. Develop and maintain automated testing frameworks covering functional, non-functional, and AI-specific requirements. Evaluate AI-generated outputs for accuracy, reliability, consistency, bias, fairness, safety, and Responsible AI compliance. Collaborate with engineering, product, data science, and business teams to identify quality risks and establish acceptance criteria.

Conocimientos

Quality engineering

Test automation

AI / ML testing

Generative AI / LLM testing

Python / Java

API testing

Integration testing

Data validation

Responsible AI

AI model evaluation

Test framework development

CI/CD

Agile / SDLC

Descripción del empleo

Project description

Join Our Team: Innovating Health Care with Cutting-Edge TechnologyWe are looking for an experienced AI Quality Engineer to design and implement quality engineering strategies for AI-enabled applications, platforms, and services. The role focuses on automated testing, AI model evaluation, and ensuring the accuracy, reliability, safety, and compliance of Generative AI, LLM, and machine learning solutions within enterprise environments.

Responsibilities

• Design and execute comprehensive testing strategies for AI/ML, GenAI, LLM, and intelligent automation solutions. Develop and maintain automated testing frameworks covering functional, non-functional, and AI-specific requirements. Evaluate AI-generated outputs for accuracy, reliability, consistency, bias, fairness, safety, and Responsible AI compliance. Collaborate with engineering, product, data science, and business teams to identify quality risks and establish acceptance criteria. Produce quality metrics, testing documentation, governance evidence, and recommendations supporting AI deployment and production monitoring.

SKILLS

Must have

• - Quality Engineering- Test Automation- AI / ML Testing- Generative AI / LLM Testing- Python / Java- API Testing- Integration Testing- Data Validation- Responsible AI- AI Model Evaluation- Test Framework Development- CI/CD- Agile / SDLCMandatory Skills Description3–5+ years of experience in Quality Engineering, Quality Assurance, or Software Testing, including 3+ years in test automation development and 2+ years testing AI, machine learning, Generative AI, LLM, or intelligent automation solutions. Strong experience designing automated test frameworks, validating AI-generated outputs, and performing API, integration, and data testing. Proficiency in Python, Java, or similar programming languages, with solid knowledge of Agile, SDLC, CI/CD, and Responsible AI principles, including fairness, bias detection, transparency, explainability, and human oversight. Strong analytical, communication, and cross-functional collaboration skills are required.Experience testing RAG pipelines, chatbots, conversational AI, AI agents, and LLM-based applications, including hallucination detection, prompt robustness, model drift, bias, and adversarial testing. Knowledge of AI evaluation frameworks, test harnesses, MLOps, AI governance, and cloud platforms such as Azure, AWS, or GCP. Experience within healthcare, payer, provider, pharmacy, claims, or other regulated environments is preferred. Familiarity with accessibility testing, audit and compliance processes, and mentoring QA engineers is a plus.AI Tool Expectations: Consistent use of enterprise-approved AI tools, including GitHub Copilot and Microsoft 365 Copilot, targeting a minimum of 90% weekly utilization to enhance testing, documentation, analysis, and delivery efficiency.

Nice to have

Exceptional communication skills.Ability to deliver exceptional customer service with a positive attitude.