Remote Voice AI Quality & Evaluation Analyst

KaiCalls; Northern (KY); Hybrid; USD 70,000 - 100,000; Full time

Job summary

KaiCalls is seeking a meticulous QA reviewer to sample and review real calls under privacy rules and to maintain rubrics for accuracy, intent capture, latency, tone, transfer, consent and outcome.

You will distinguish genuine business intent from demos, report failure clusters, and partner with engineering on remediation. This remote role supports evaluating calls across verticals and languages with emphasis on evidence-backed writing and handling sensitive data.

Qualifications

• Exceptional listening and analytical judgment.
• Experience in QA, conversation review, trust and safety, or applied AI evaluation.
• Evidence-backed writing and careful handling of sensitive data.
• Comfort working with imperfect data.

Responsibilities

• Sample and review real calls under privacy rules.
• Maintain rubrics for accuracy, intent capture, latency, tone, transfer, consent, and outcome.
• Distinguish genuine business intent from demos and test traffic.
• Report failure clusters and partner with engineering on remediation.
• Report evaluation findings to drive measurable product improvements.

Skills

Active listening

Analytical judgment

QA/review experience

Data handling

Job description

KaiCalls is seeking a meticulous QA reviewer to sample and review real calls under privacy rules and to maintain rubrics for accuracy, intent capture, latency, tone, transfer, consent and outcome.

You will distinguish genuine business intent from demos, report failure clusters, and partner with engineering on remediation. This remote role supports evaluating calls across verticals and languages with emphasis on evidence-backed writing and handling sensitive data.