Staff AI QA Engineer - Remote, Equity & Growth

Harriscomputer

United States

Remote

USD 120,000 - 160,000

Full time

4 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

medical benefits
vacation & personal days
employee stock ownership
RRSP/401k matching
flexible work arrangements

Job summary

Harris Computer is seeking a Staff AI Engineer focused on Quality Assurance to lead the testing and reliability of healthcare AI/ML solutions. This remote role is open to candidates in the United States or Canada, offering a path to shape how AI quality is defined and measured across clinical workflows.

The position requires extensive experience in software/ML engineering with a proven ability to design end-to-end systems, build evaluation frameworks, and establish rigorous metrics for accuracy

Qualifications

  • 7+ years in software or ML engineering with 3+ years in evaluation or QA for AI systems.
  • Proven track record building end-to-end systems from prototype to production.
  • Experience running evaluation systems in production, not just one-off analyses.
  • 2+ years evaluating large language models (LLMs) in production environments.
  • Strong statistical skills: sampling, confidence intervals, error analysis.
  • 1+ year designing evaluation sets for document extraction, classification, or structured output.
  • Deep knowledge of XAI techniques and AI-powered tools.

Responsibilities

  • Build the core AI system end-to-end, from service layer to evaluation infrastructure.
  • Design and maintain evaluation datasets with clinician-reviewed answers.
  • Create error metrics weighted by severity for clinically/financially impactful mistakes.
  • Develop production monitoring for accuracy, overrides, and safety issues with alerts.
  • Write adversarial tests for AI systems, including hidden instructions in documents.
  • Collaborate with product and clinical leads to define accuracy thresholds for capability expansion.

Skills

System design
Quality testing
Statistical analysis
Explainable AI (XAI)

Tools

SHAP
LIME
Claude
Great Expectations

Job description

Harris Computer is seeking a Staff AI Engineer focused on Quality Assurance to lead the testing and reliability of healthcare AI/ML solutions. This remote role is open to candidates in the United States or Canada, offering a path to shape how AI quality is defined and measured across clinical workflows.

The position requires extensive experience in software/ML engineering with a proven ability to design end-to-end systems, build evaluation frameworks, and establish rigorous metrics for accuracy

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff AI QA Engineer – Healthcare AI Quality & Reliability (Remote)
Staff AI QA Engineer – Healthcare AI Quality & Reliability (Remote)

Harris Computer Corp. • South Carolina

Remote
USD 120,000 - 160,000
Career growth opportunities
Medical, dental, and vision benefits
3 weeks vacation + 5 personal days
+1
Remote Staff AI QA Engineer - Healthcare AI Quality Leader
Remote Staff AI QA Engineer - Healthcare AI Quality Leader

Harris Computer Corp. • Michigan

Remote
USD 120,000 - 160,000
Career growth
Comprehensive benefits
Vacation + personal days
+1
Senior AI Quality Engineer – Healthcare (Remote)
Senior AI Quality Engineer – Healthcare (Remote)

Harris Computer Corp. • Virginia

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Stock ownership and retirement plans
+2
Staff AI QA Engineer - Healthcare, Remote, High Impact
Staff AI QA Engineer - Healthcare, Remote, High Impact

Harris Computer Corp. • New Mexico

Remote
USD 120,000 - 160,000
Medical benefits
Dental and vision coverage
Vacation plus personal days
+3
Lead AI Quality Assurance Engineer – Remote Healthcare
Lead AI Quality Assurance Engineer – Remote Healthcare

Harris Computer Corp. • New York (NY)

Remote
USD 120,000 - 160,000
Growth opportunities
Medical benefits
Dental benefits
+7
Staff AI Quality Engineer – Remote Healthcare
Staff AI Quality Engineer – Remote Healthcare

N. Harris Computer Corporation - USA • Raleigh (NC)

Remote
USD 120,000 - 160,000
Medical, dental, and vision benefits
Flexible work arrangements
Career growth opportunities
+2
Staff AI QA Engineer – Healthcare Quality Lead (Remote)
Staff AI QA Engineer – Healthcare Quality Lead (Remote)

Harris Computer Corp. • Texas

Remote
USD 120,000 - 160,000
Career growth
Medical/Dental/Vision
Vacation and personal days
+4
Staff AI QA Engineer – Healthcare, Remote
Staff AI QA Engineer – Healthcare, Remote

Harris Computer Corp. • United States

Remote
USD 120,000 - 160,000
Medical, dental, vision coverage
3 weeks vacation + 5 personal days
Employee stock ownership
+3
Remote Senior AI QA Engineer — Healthcare
Remote Senior AI QA Engineer — Healthcare

Harris Computer Corp. • Tennessee

Remote
USD 120,000 - 160,000
Career growth
Medical, dental, vision benefits
Vacation time (3 weeks) and personal 5
+2
Staff AI Engineer - QA
Staff AI Engineer - QA

Harriscomputer • United States

Remote
USD 120,000 - 160,000
medical benefits
vacation & personal days
employee stock ownership
+2