Remote Senior AI QA Engineer — Healthcare

Harris Computer Corp.

Tennessee

Remote

USD 120,000 - 160,000

Full time

4 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Career growth
Medical, dental, vision benefits
Vacation time (3 weeks) and personal 5
Stock ownership and 401k/RRSP
Flexible work arrangements

Job summary

Harris Computer Corp. is seeking a Staff AI Engineer (QA) to lead the quality and reliability of our AI/ML solutions for healthcare. The role focuses on building robust evaluation frameworks, data pipelines, and production-grade testing for patient-care AI systems.

You will mentor engineers, define accuracy thresholds with clinical input, and leverage AI tools to improve workflows while maintaining safety and fairness across deployments.

Qualifications

  • 7 or more years in software or machine learning engineering, with at least 3 years focused on evaluation, testing, or quality for AI systems.
  • A track record of designing and building complete systems from scratch, taking them from prototype to production.
  • A track record of building and running evaluation systems that teams use in production, not just for one-off analyses.
  • At least 2 years of experience evaluating large language model (LLM) systems in a production environment.
  • Strong statistical skills for measuring accuracy, including sampling, confidence intervals, and error analysis.
  • 1 year + of experience designing evaluation sets for document extraction, classification, or structured output.
  • Deep knowledge of explainable AI (XAI) techniques and AI-powered tools (e.g., SHAP, LIME, Claude).
  • Clear written communication, including design documents, and the ability to explain complex trade-offs to clinicians, engineers, and executives.

Responsibilities

  • Build and own end-to-end AI system evaluation, including data pipelines, service layer, and architecture.
  • Design evaluation infrastructure, dataset management, automated test harnesses, and replay of AI workflows.
  • Create and maintain evaluation datasets with clinician-reviewed answers that evolve with systems.
  • Develop error metrics weighted by severity for clinical or financial impact.
  • Implement production monitoring for accuracy, overrides, and safety with actionable alerts.
  • Write adversarial tests for AI systems and hidden/malicious content handling.
  • Collaborate with product and clinical leads to set accuracy thresholds for capabilities.
  • Adopt emerging AI tools to improve workflows and empower engineers.

Skills

ML engineering
Evaluation testing
Production systems
Statistical analysis
Explainable AI
LLM evaluation
Adversarial testing

Tools

SHAP
LIME
Claude
Great Expectations

Job description

Harris Computer Corp. is seeking a Staff AI Engineer (QA) to lead the quality and reliability of our AI/ML solutions for healthcare. The role focuses on building robust evaluation frameworks, data pipelines, and production-grade testing for patient-care AI systems.

You will mentor engineers, define accuracy thresholds with clinical input, and leverage AI tools to improve workflows while maintaining safety and fairness across deployments.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff AI QA Engineer – Healthcare Quality Lead (Remote)
Staff AI QA Engineer – Healthcare Quality Lead (Remote)

Harris Computer Corp. • Texas

Remote
USD 120,000 - 160,000
Career growth
Medical/Dental/Vision
Vacation and personal days
+4
Remote Staff AI QA Engineer - Healthcare AI Quality Leader
Remote Staff AI QA Engineer - Healthcare AI Quality Leader

Harris Computer Corp. • Michigan

Remote
USD 120,000 - 160,000
Career growth
Comprehensive benefits
Vacation + personal days
+1
Staff AI QA Engineer - Healthcare, Remote, High Impact
Staff AI QA Engineer - Healthcare, Remote, High Impact

Harris Computer Corp. • New Mexico

Remote
USD 120,000 - 160,000
Medical benefits
Dental and vision coverage
Vacation plus personal days
+3
Senior AI Quality Engineer – Healthcare (Remote)
Senior AI Quality Engineer – Healthcare (Remote)

Harris Computer Corp. • Virginia

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Stock ownership and retirement plans
+2
Lead AI Quality Assurance Engineer – Remote Healthcare
Lead AI Quality Assurance Engineer – Remote Healthcare

Harris Computer Corp. • New York (NY)

Remote
USD 120,000 - 160,000
Growth opportunities
Medical benefits
Dental benefits
+7
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • New York (NY)

Remote
USD 120,000 - 160,000
Growth opportunities
Medical benefits
Dental benefits
+7
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Virginia

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Stock ownership and retirement plans
+2
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Texas

Remote
USD 120,000 - 160,000
Career growth
Medical/Dental/Vision
Vacation and personal days
+4
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • New Mexico

Remote
USD 120,000 - 160,000
Medical benefits
Dental and vision coverage
Vacation plus personal days
+3
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Tennessee

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Vacation time (3 weeks) and personal 5
+2