Remote Staff AI QA Engineer - Healthcare AI Quality Leader

Harris Computer Corp.

Michigan

Remote

USD 120,000 - 160,000

Full time

4 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Career growth
Comprehensive benefits
Vacation + personal days
Stock ownership

Job summary

Harris Computer is seeking a Staff AI Engineer (QA) to ensure the reliability, fairness, and safety of our AI solutions for healthcare. You will lead testing strategy, build evaluation infra, and mentor a growing team, defining the gold standard for AI quality in production.

This full-time remote role is open to candidates in the U.S. or Canada. You will collaborate with product and clinical leads to set accuracy targets and drive adoption of AI tools to improve workflows.

Qualifications

  • 7+ years in software or ML engineering with at least 3 years focused on evaluation, testing, or quality for AI systems.
  • Experience designing and building complete systems from scratch, from prototype to production.
  • Experience building and running evaluation systems used in production, not just one-off analyses.
  • At least 2 years of experience evaluating large language model (LLM) systems in a production environment.
  • Strong statistical skills for measuring accuracy, including sampling, confidence intervals, and error analysis.
  • 1 year+ of experience designing evaluation sets for document extraction, classification, or structured output.
  • Deep knowledge of explainable AI (XAI) techniques and AI-powered tools (e.g., SHAP, LIME, Claude).
  • Clear written communication to explain complex trade-offs to clinicians, engineers, and executives.

Responsibilities

  • Build the core AI system end-to-end and own its architecture.
  • Design and build the evaluation infrastructure, including dataset management and test harnesses.
  • Build and maintain evaluation datasets with clinician-reviewed correct answers.
  • Create error metrics weighted by severity to prioritize risk areas.
  • Develop production monitoring for accuracy, overrides, and safety issues.
  • Write adversarial tests for AI systems to catch hidden or malicious content.
  • Collaborate with product and clinical leads to define accuracy thresholds and expansion criteria.
  • Adopt emerging AI tools to improve workflows and drive efficiency.

Skills

Software/ML engineering
Evaluation/QA for AI
Production systems
LLM evaluation
Statistical analysis
Explainable AI
Communication

Tools

Great Expectations
SHAP
LIME
Claude

Job description

Harris Computer is seeking a Staff AI Engineer (QA) to ensure the reliability, fairness, and safety of our AI solutions for healthcare. You will lead testing strategy, build evaluation infra, and mentor a growing team, defining the gold standard for AI quality in production.

This full-time remote role is open to candidates in the U.S. or Canada. You will collaborate with product and clinical leads to set accuracy targets and drive adoption of AI tools to improve workflows.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead AI Quality Assurance Engineer – Remote Healthcare
Lead AI Quality Assurance Engineer – Remote Healthcare

Harris Computer Corp. • New York (NY)

Remote
USD 120,000 - 160,000
Growth opportunities
Medical benefits
Dental benefits
+7
Senior AI Quality Engineer – Healthcare (Remote)
Senior AI Quality Engineer – Healthcare (Remote)

Harris Computer Corp. • Virginia

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Stock ownership and retirement plans
+2
Staff AI QA Engineer - Healthcare, Remote, High Impact
Staff AI QA Engineer - Healthcare, Remote, High Impact

Harris Computer Corp. • New Mexico

Remote
USD 120,000 - 160,000
Medical benefits
Dental and vision coverage
Vacation plus personal days
+3
Staff AI QA Engineer – Healthcare Quality Lead (Remote)
Staff AI QA Engineer – Healthcare Quality Lead (Remote)

Harris Computer Corp. • Texas

Remote
USD 120,000 - 160,000
Career growth
Medical/Dental/Vision
Vacation and personal days
+4
Remote Senior AI QA Engineer — Healthcare
Remote Senior AI QA Engineer — Healthcare

Harris Computer Corp. • Tennessee

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Vacation time (3 weeks) and personal 5
+2
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • New Mexico

Remote
USD 120,000 - 160,000
Medical benefits
Dental and vision coverage
Vacation plus personal days
+3
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Tennessee

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Vacation time (3 weeks) and personal 5
+2
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Michigan

Remote
USD 120,000 - 160,000
Career growth
Comprehensive benefits
Vacation + personal days
+1
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Virginia

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Stock ownership and retirement plans
+2
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • New York (NY)

Remote
USD 120,000 - 160,000
Growth opportunities
Medical benefits
Dental benefits
+7