Data Annotation QA Specialist

Confidential; Surabaya ꦱꦸꦫꦧꦪ; On-site; IDR 60,000,000 - 120,000,000; Full time

Job summary

Confidential is seeking a Robotics Annotation QA Auditor to independently assess labeled video data for a leading robotics AI program. You will score annotations against a quality scorecard and document the rationale with frame-level references.

You will review videos of robots interacting with objects, verify label accuracy, segment boundaries, and language quality. The role emphasizes judgment, adherence to evolving guidelines, and producing clear, evidence-based feedback while maintaining

Qualifications

• At least 2 years in data annotation or AI training data QA.
• Experience reviewing others' work as QA auditor or quality lead.
• Proficient English writing with crisp grammar.
• Ability to work from long, technical guidelines and adapt to updates.
• Strong decision-making with evidence-based explanations.

Responsibilities

• Review robotics videos frame by frame and evaluate segments against a quality scorecard.
• Check label accuracy for objects, locations, actions, and hand usage.
• Verify segment boundaries and ensure full actions are captured.
• Assess language quality and precision of labels and prepositions.
• Identify granularity level and document edge cases with timestamps.
• Log guideline gaps and participate in calibration and QC processes.
• Contribute findings to client reports with actionable recommendations.

Skills

QA reviewer

Video annotation

English fluency

Spatial reasoning

Guidelines adaptation

Confidentiality

Tools

Video review tools

Rubric development

Job description

About the Role

We support a leading robotics AI research program that teaches robots to understand and carry out physical tasks by learning from labeled video. The quality of that labeled data directly shapes what the robots learn.

As a Robotics Annotation Quality Auditor, you won't be creating annotations. You'll independently assess annotations other people have produced. You'll score each one against the client's quality scorecard and written guidelines, then document exactly why you scored it that way.

You'll review videos of robots and people handling objects, frame by frame, across three kinds of work: natural-language task descriptions, fine-grained action segmentation, and object point tracking. For each segment you'll check four things:

• Does the label match what actually happens on screen?
• Do the start and end points fall in the right place?
• Are the objects, locations, and hands correctly identified?
• Is the instruction clear enough for a robot to follow?

The hardest part of the job is judgment. Guidelines for frontier research change often, and sometimes they allow more than one reasonable reading. You'll need to tell the difference between an annotator who got it wrong and one who made a defensible call. You'll need to apply that distinction consistently, and flag the cases where the guidelines themselves need fixing.

What You'll Do

• Review robotics video episodes frame by frame and evaluate each assigned segment against a structured quality scorecard.
• Check label accuracy: the objects, locations, and actions described, and which hand or gripper was used.
• Check segment boundaries: whether each segment starts and ends at the right frame and captures the full action.
• Check language quality: whether each label is a clear, concise, direct command with correct grammar and prepositions.
• Check granularity: whether actions are broken down to the level the task type requires.
• Evaluate point-tracking work for keypoint placement, drift, click frequency, and handling of objects entering, leaving, or being hidden from view.
• Write a specific, evidence-based comment for every deduction, with timestamps or frame references, so another reviewer can verify your call.
• Log guideline gaps, contradictions, and edge cases as you find them.
• Take part in calibration sessions and double-reviews, and have your own audits checked by a QC lead.
• Contribute findings to client report-outs, including recurring error patterns, documentation issues, and practical recommendations.

What We're Looking For

Required Qualifications

• At least 2 years in data annotation or AI training data, including at least 1 year reviewing or auditing other people's work as a QA reviewer, auditor, or quality lead.
• Experience with video annotation, such as temporal segmentation, action labeling, or object tracking.
• Professional written fluency in English, with a sharp eye for grammar, word choice, and concise phrasing.
• Strong spatial reasoning, including describing positions (left and right, rows, columns, shelves) precisely and consistently.
• Ability to work from long, technical guidelines that change over time, applying only the current version.
• Willingness to form your own judgment rather than accept the label in front of you.
• Clear, neutral written feedback that explains the problem and points to the evidence.
• Comfort with frame-level video review tools (training will be provided on the platform).

Nice to Have

• Annotation experience in robotics, computer vision, autonomous vehicles, or physical AI.
• Keypoint or point-tracking experience.
• Experience writing or revising rubrics, scorecards, or annotation guidelines.
• Experience contributing to client-facing quality reports.
• A background in linguistics, technical writing, or editing.

How You'll Work

• Accuracy over speed: Each episode needs focused, uninterrupted review, often 40 minutes or more. We'd rather have one well-evidenced audit than three rushed ones.
• Your work gets checked too: A QC lead reviews audits, and calibration means your calls will be compared with other reviewers'. We're looking for people who find that useful rather than uncomfortable.
• You'll know when to make a call: Most judgment calls get resolved within the team. Questions that affect the scoring standard itself get escalated. Knowing the difference is part of the job.
• Strict confidentiality: Client guidelines, footage, and materials stay inside secure client systems. You won't download, copy, or reproduce them anywhere else, including in personal notes or checklists.