Staff AI QA Engineer - Healthcare, Remote, High Impact

Harris Computer Corp.

New Mexico

Remote

USD 120,000 - 160,000

Full time

5 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Medical benefits
Dental and vision coverage
Vacation plus personal days
Employee stock ownership
401k matching (US)
Flexible work arrangements

Job summary

Harris Computer Corp. is hiring a Staff AI Engineer (QA) to lead quality assurance for cutting-edge AI and ML solutions in healthcare. You will define testing standards, build evaluation datasets, and create metrics with clinical impact in mind.

The role emphasizes mentoring a team, shaping testing strategies, and adopting emerging AI tools while ensuring safety, fairness, and accuracy across production systems. Remote-friendly within the U.S. and Canada.

Qualifications

  • 7+ years in software or ML with at least 3 years focused on evaluation, testing, or quality for AI systems.
  • Experience designing and building complete systems from scratch, taking them from prototype to production.
  • Experience building and running evaluation systems used by teams in production.
  • At least 2 years evaluating large language model systems in a production environment.
  • Strong statistical skills for measuring accuracy, including sampling, confidence intervals, and error analysis.
  • Experience designing evaluation sets for document extraction, classification, or structured output.

Responsibilities

  • Build the core AI system end-to-end, from service layer and data pipelines to evaluation infra, and own its architecture.
  • Design and build the evaluation infra, including dataset management, test harnesses, and replay against historical workflows.
  • Create and maintain evaluation datasets with clinician-reviewed correct answers that evolve with our systems.
  • Develop error metrics weighted by severity to prioritize clinical or financial impact.
  • Establish production monitoring for accuracy, overrides, and safety with actionable alerts.
  • Write adversarial tests for AI systems, including hidden instructions in documents and invented content.
  • Collaborate with product and clinical leads to define accuracy thresholds and expansion criteria for AI capabilities.

Skills

ML engineering
Quality assurance
Evaluation systems
Statistical analysis
Explainable AI
Documentation

Tools

SHAP
LIME
Claude

Job description

Harris Computer Corp. is hiring a Staff AI Engineer (QA) to lead quality assurance for cutting-edge AI and ML solutions in healthcare. You will define testing standards, build evaluation datasets, and create metrics with clinical impact in mind.

The role emphasizes mentoring a team, shaping testing strategies, and adopting emerging AI tools while ensuring safety, fairness, and accuracy across production systems. Remote-friendly within the U.S. and Canada.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff AI QA Engineer – Healthcare Quality Lead (Remote)
Staff AI QA Engineer – Healthcare Quality Lead (Remote)

Harris Computer Corp. • Texas

Remote
USD 120,000 - 160,000
Career growth
Medical/Dental/Vision
Vacation and personal days
+4
Senior AI Quality Engineer – Healthcare (Remote)
Senior AI Quality Engineer – Healthcare (Remote)

Harris Computer Corp. • Virginia

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Stock ownership and retirement plans
+2
Remote Staff AI QA Engineer - Healthcare AI Quality Leader
Remote Staff AI QA Engineer - Healthcare AI Quality Leader

Harris Computer Corp. • Michigan

Remote
USD 120,000 - 160,000
Career growth
Comprehensive benefits
Vacation + personal days
+1
Lead AI Quality Assurance Engineer – Remote Healthcare
Lead AI Quality Assurance Engineer – Remote Healthcare

Harris Computer Corp. • New York (NY)

Remote
USD 120,000 - 160,000
Growth opportunities
Medical benefits
Dental benefits
+7
Remote Senior AI QA Engineer — Healthcare
Remote Senior AI QA Engineer — Healthcare

Harris Computer Corp. • Tennessee

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Vacation time (3 weeks) and personal 5
+2
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • New York (NY)

Remote
USD 120,000 - 160,000
Growth opportunities
Medical benefits
Dental benefits
+7
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Virginia

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Stock ownership and retirement plans
+2
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Texas

Remote
USD 120,000 - 160,000
Career growth
Medical/Dental/Vision
Vacation and personal days
+4
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Michigan

Remote
USD 120,000 - 160,000
Career growth
Comprehensive benefits
Vacation + personal days
+1
Staff AI Engineer - QA
Staff AI Engineer - QA

Harris Computer Corp. • Tennessee

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Vacation time (3 weeks) and personal 5
+2