QA Auditor (Sampling, Quality Gates, Release Readiness)

Codvo.ai; Pune District; On-site; INR 800,000 - 1,500,000; Full time

Job summary

A leading data solutions provider is seeking a QA Auditor to protect dataset quality by conducting sampling-based audits and ensuring quality gates are met. The ideal candidate will have 3–7+ years of data QA experience, especially with auditing labeled datasets, with a strong command of sampling techniques and meticulous attention to detail. Responsibilities include executing QA audits, maintaining dashboards, and supporting release readiness. This is a full-time role based in Pune, India.

Qualifications

• 3–7+ years in data QA / annotation QA / operations quality; experience auditing labeled datasets (video preferred).
• Strong command of sampling-based QA: stratified sampling, acceptance thresholds, and audit scoring.
• High attention to detail; comfortable reading and enforcing detailed SOPs/guidelines.
• Strong documentation skills; can write clear defect descriptions and rework instructions.
• Comfortable with dashboards/spreadsheets; basic metric literacy (defect density, rework, throughput).

Responsibilities

• Execute sampling plans for ongoing QA and release audits.
• Audit annotations against guidelines and score defects.
• Run and validate quality gates prior to release.
• Generate rework tickets with precise instructions.
• Partner with Annotation Lead for calibration actions.
• Maintain QA dashboards on defect rates and trends.
• Support release readiness for the Evidence Pack.
• Escalate systematic issues to relevant reviews.
• Produce QA audit reports and gate status recommendations.

Job description

Job Title: QA Auditor (Sampling, Quality Gates, Release Readiness)

JD: QA Auditor — Surgical Video Annotation (Sampling, Quality Gates, Release Readiness)

Location: India - Pune | Type: Full-time

Reports to: Quality Lead / Program Lead

About Us

At Codvo, we are committed to building scalable, future-ready data platforms that power business impact. We believe in a culture of innovation, collaboration, and growth, where engineers can experiment, learn, and thrive. Join us to be part of a team that solves complex data challenges with creativity and cutting-edge technology.

Role goal

Protect dataset quality at scale by running sampling-based audits, enforcing quality gates, and ensuring every dataset release is audit-ready and meets acceptance thresholds.

Responsibilities

• Execute sampling plans (stratified by label type, difficulty, site/surgeon/device, model confidence) for ongoing QA and release audits.
• Audit annotations against guidelines + ontology; score defects using standardized rubrics (severity, type, root cause).
• Run and validate quality gates prior to release:
  • completeness/coverage checks
  • schema/constraint checks (illegal combinations, temporal consistency, boundary rules)
  • defect density and rework thresholds
• Generate rework tickets with precise instructions; track closure and verify fixes.
• Partner with Annotation Lead to drive calibration actions (training refreshers, guideline clarifications, examples library).
• Maintain QA dashboards: defect rate, rework %, gate pass rate, audit cycle time, top error patterns by annotator/team/label type.
• Support release readiness: ensure required artifacts are complete for the Evidence Pack (QA results, sampling logs, sign-offs).
• Escalate systematic issues to Adjudication/Clinical Review (definition gaps, ambiguous classes, recurring edge cases).
• Deliverables (what you produce weekly):
  • QA audit reports (sample size, findings, severity distribution, root-cause categories)
  • Gate status + release checklist sign-off recommendations
  • Rework queue and closure verification
  • Trend insights and corrective action plan (top 3 issues + fixes)

Requirements

• 3–7+ years in data QA / annotation QA / operations quality; experience auditing labeled datasets (video preferred).
• Strong command of sampling-based QA: stratified sampling, acceptance thresholds, and audit scoring.
• High attention to detail; comfortable reading and enforcing detailed SOPs/guidelines.
• Strong documentation skills; can write clear defect descriptions and rework instructions.
• Comfortable with dashboards/spreadsheets; basic metric literacy (defect density, rework, throughput).

Note- Please apply via our official careers portal only, as applications sent directly to executives may not be considered.