AI/ML QA Engineer

Arkhya Tech. Inc.; McLean (VA); On-site; USD 110,000 - 160,000; Full time

Job summary

Arkhya Tech. Inc. seeks an AI/ML QA Engineer to ensure the quality, reliability, and governance of AI and Machine Learning solutions in the healthcare domain.

The role involves validating AI models, testing data pipelines, evaluating model outputs, identifying bias and performance issues, and supporting the end-to-end quality assurance lifecycle for AI-driven applications. The expert will collaborate with data scientists and engineers, develop test plans, automate end-to-end tests, and

Qualifications

• 5–8 years of experience in AI/ML QA within healthcare
• Strong automation and testing mindset
• Experience validating AI/ML models and data pipelines

Responsibilities

• Validate AI/ML models, data pipelines, and outputs
• Evaluate model bias, performance, and hallucinations
• Ensure governance and Responsible AI compliance across the QA lifecycle
• Collaborate with data scientists and engineers to implement end-to-end testing

Skills

AI/ML Testing

Python

SQL

ETL Testing

Snowflake

Databricks

Azure

Healthcare Domain

GenAI Testing

Tools

Selenium

Postman

JIRA

TestNG

PyTest

Job description

Location: McLean, VA 22102 (100% Onsite)

Experience: 5 to 8 YearsPrimary Skills: AI/ML Testing, Python, SQL, ETL Testing, Snowflake, Databricks, Azure, Healthcare Domain, GenAI Testing.

Job Summary:

We are seeking a skilled AI/ML QA Engineer to ensure the quality, reliability, performance, and governance of AI and Machine Learning solutions in the healthcare domain. The role involves validating AI models, testing data pipelines, evaluating model outputs, identifying bias and performance issues, and supporting the end-to-end quality assurance lifecycle for AI-driven applications.

equired Technical Skills

• QA & Automation
• Manual Testing
• Test Planning & Execution
• Test Automation (Selenium, API Testing with Postman)
• JIRA, TestNG / PyTest
• Data Testing (SQL, ETL Testing, Data Validation)
• Programming (Python, PySpark preferred)

Expertise You'll Bring:

• Experience testing Generative AI and LLM-based applications.
• Familiarity with GitHub Copilot, Microsoft 365 Copilot, and enterprise AI tools.
• Understanding of prompt engineering and prompt validation techniques.
• Experience evaluating AI output quality, hallucinations, and response consistency.
• Knowledge of Responsible AI principles, fairness, explainability, and governance.