HITL Quality Analyst

Ninja Partners; United States; Remote; USD 60,000 - 90,000; Full time

Benefits offered by this job

Full-time

Competitive pay

Gov't benefits

Retirement match

Life Insurance

HMO day 1

PTO + birthday leave

Bonus plans

Training programs

Referral program

Nice office

Free lunch

Job summary

SupportNinja is seeking a HITL Quality Analyst to evaluate AI outputs against established standards, identify and track defects, and drive continuous improvements across teams in a remote setup. You will monitor quality trends, maintain defect logs, and prepare performance summaries for stakeholders.

The role emphasizes data-driven decision-making, collaboration, and adherence to processes to ensure high-quality AI support operations.

Qualifications

• Minimum of two (2) years of experience in quality assurance, quality evaluation, compliance, auditing, or related field.
• Experience evaluating work against defined quality standards, scorecards, policies, or procedures.
• Experience identifying, categorizing, documenting, and tracking defects or quality issues.
• Calibration experience to ensure scoring consistency preferred.
• Strong analytical and critical-thinking skills with pattern identification.
• Excellent written and verbal communication to document findings clearly.
• Ability to manage multiple priorities in a fast-paced environment.
• Commitment to data-driven decision making.

Responsibilities

• Review and evaluate AI outputs using established standards and defect classification.
• Identify, classify, document, and track defects for measurement and trend analysis.
• Maintain accurate defect logs and resolution tracking through completion.
• Analyze quality metrics to identify recurring issues, risks, and opportunities.
• Coordinate with Knowledge Management, AI Tuning, Governance, and Operations.
• Validate effectiveness of corrective actions via follow-up evaluations.
• Prepare reports, dashboards, and performance summaries for stakeholders.
• Participate in calibration sessions to ensure consistent scoring across reviewers.
• Identify opportunities to improve quality processes, reporting, and evaluation.

Skills

Quality assurance

Quality evaluation

Analytical thinking

Written communication

Verbal communication

Time management

Collaboration

Data-driven decisions

Job description

If you are unable to complete this application due to a disability, contact this employer to ask for an accommodation or an alternative application process.

HITL Quality Analyst

Service Delivery Full Time Salary Philippines - Remote, PH

Work Setup: Remote/WAH

Equipment: BYOD (Bring-Your-Own-Device)

Type of Contract: Full-Time

The HITL Quality Analyst is responsible for evaluating AI-generated outputs against established quality standards to ensure accuracy, consistency, and compliance with organizational requirements. This role identifies, classifies, and tracks defects, analyzes quality trends, and collaborates with cross-functional stakeholders to drive continuous improvements that enhance AI performance and operational effectiveness.

What does a day in the life of a HITL Quality Analyst look like?

• Review and evaluate AI-generated outputs using established quality standards, evaluation criteria, and defect classification methodologies
• Identify, classify, document, and track defects to support consistent quality measurement, trend analysis, and continuous improvement initiatives
• Maintain accurate quality records, defect logs, and resolution tracking to ensure findings are documented and monitored through completion
• Analyze quality metrics, detect trends, and performance data to identify recurring issues, operational risks, and opportunities for improvement
• Coordinate with Knowledge Management, AI Tuning, Governance, Operations, and other stakeholders to communicate findings and support timely issue resolution
• Validate the effectiveness of corrective actions by monitoring quality improvements and confirming defect resolution through follow-up evaluations
• Support the development and refinement of quality standards, evaluation methodologies, sampling strategies, and defect taxonomies
• Prepare reports, dashboards, and performance summaries that provide actionable insights to operational and business stakeholders
• Participate in calibration sessions to promote consistent evaluation practices and maintain scoring accuracy across reviewers
• Identify opportunities to improve quality processes, reporting, evaluation consistency, and operational performance through continuous improvement initiatives
• Adhere to company policies and procedures
• Meet or exceed performance targets for related KPIs
• Continuously improve knowledge of products, services, and processes by participating in training programs and continuous learning modules
• Collaborate with other departments as needed
• Maintain a positive, empathetic, and professional attitude toward customers and colleagues at all times
• Perform other duties as assigned

What are the required qualifications for aHITL Quality Analyst?

• Minimum of two (2) years of experience in quality assurance, quality evaluation, compliance, auditing, transaction monitoring, operational quality, or a related field
• Experience evaluating work against defined quality standards, scorecards, policies, or operational procedures
• Experience identifying, categorizing, documenting, and tracking defects or quality issues to support performance improvement initiatives
• Experience conducting calibration activities or applying standardized evaluation methodologies to ensure scoring consistency is preferred
• Experience analyzing quality metrics, trends, and operational data to identify improvement opportunities and recurring issues
• Strong analytical and critical-thinking skills with the ability to identify patterns, evaluate root causes, and recommend improvements
• Strong written and verbal communication skills with the ability to clearly documen findings and communicate recommendations to cross-functional stakeholders
• Strong organizational and time management skills with the ability to manage multiple priorities while maintaining evaluation accuracy and consistency
• Ability to work independently and collaboratively in a fast-paced environment while maintaining objectivity and attention to detail
• Commitment to quality, operational excellence, continuous improvement, and data-driven decision-making

Ninja Perks and Benefits

• Full-time employees
  • Competitive compensation
  • Adherence to government-mandated benefits
  • Retirement Savings Program with Company Matching
  • Life Insurance
  • HMO on day 1
  • Paid time off, birthday leave
  • Bonus and incentive plans
  • Opportunities for skills training and personal and professional development
  • Employee Referral Program
  • Beautiful office space
  • Free lunch provided daily

Disclaimer:

The duties and responsibilities listed above describe the post as it is in general terms and are not definitive. The post holder is expected to accept any reasonable alterations that may from time to time be necessary.

SupportNinja is proud to be an Equal Employment Opportunity employer and we do not discriminate based upon race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, veteran status, disability, or other applicable legally protected characteristics under federal, state, or local law.

#supportninjajobs