ML Annotation QA Engineer — Data Quality & Root Cause

Gather AI; United States; À distance; USD 120 000 - 160 000; Plein temps

Résumé du poste

Gather AI is seeking an ML Annotation QA Engineer to own the quality of annotated data across our computer vision and ML programs. You will analyze data, define verdict rules, build trackers, and work closely with ML, QA, and Engineering to improve model performance in the field.

Responsibilities include ruling on annotations, maintaining a verdict taxonomy, and surfacing insights that drive tool and process improvements, with a focus on reliability and clear documentation.

Qualifications

• BS in Computer Science/Engineering or equivalent experience.
• 2–5 years of experience in ML QA, annotation quality, data quality, or analytics at an AI/ML company.
• Hands-on experience with annotated ML datasets, including assessing label quality.
• Demonstrated ability to find, diagnose, and report data anomalies to a technical audience.
• Able to derive defensible answers from unfamiliar data without pre-prepared data.
• Understanding of data privacy and confidentiality requirements with customer data.

Responsabilités

• Own the judgment-heavy quality analysis on annotated data that cannot be reliably outsourced.
• Own, version, and refine the verdict taxonomy, decision rules, and quality guidelines for the categories you cover.
• Build and maintain performance trackers over annotated data (error rates by facility, site, equipment, and data format).
• Detect anomalies against the baseline and flag them promptly.
• Run root cause analysis on flagged anomalies, distinguishing annotation error from model or system error.
• Report findings to engineering and ML with reproducible evidence and stated confidence.
• Identify systematic failure patterns and maintain a documented pattern library.
• Query and analyse annotation data with Python and SQL to test hypotheses.
• Provide concrete SOP changes when the root cause is labeling and not system behavior.
• Suggest annotation tool improvements and validate fixes.
• Stand up quality analysis for new annotation programs as they come online.
• Track work across Jira and contribute to pre-release validation.

Connaissances

Analytical thinking

Attention to detail

Written communication

Collaboration

Problem solving

Formation

BS in Computer Science/Engineering

BS in Electrical Engineering

Outils

Python

SQL

Jira

Description du poste

Gather AI is seeking an ML Annotation QA Engineer to own the quality of annotated data across our computer vision and ML programs. You will analyze data, define verdict rules, build trackers, and work closely with ML, QA, and Engineering to improve model performance in the field.

Responsibilities include ruling on annotations, maintaining a verdict taxonomy, and surfacing insights that drive tool and process improvements, with a focus on reliability and clear documentation.