ML Annotation Quality Engineer

Caraa; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Gather AI is seeking an ML Annotation QA Engineer to own the quality of annotated data across computer vision and ML programs. You will judge data quality, refine verdict taxonomies, and build trackers to surface degradation patterns in the field.

Collaboration with ML engineers and QA is essential as you diagnose annotation drift, tool issues, and model performance in production. Your role involves analyzing data with Python/SQL, defending calls to external vendors, and creating SOPs to improve

Qualifications

• BS in Computer Science, Engineering, or equivalent required.
• 2–5 years of ML QA, annotation quality, data quality, or analytics in AI/ML.
• Hands-on with annotated ML datasets and assessing label quality.
• Able to diagnose data anomalies and report to a technical audience.

Responsibilities

• Own the judgment-heavy quality analysis on annotated data that cannot be outsourced.
• Version and refine verdict taxonomy, rules, and quality guidelines.
• Build performance trackers for annotated data across facilities and data formats.
• Detect anomalies against a baseline and flag them promptly.
• Perform root cause analysis to distinguish annotation vs model vs field degradation.
• Provide reproducible evidence and confidence in findings to ML/engineering.
• Document failure patterns and maintain a knowledge library.
• Query data with Python and SQL to test hypotheses.
• Suggest SOP changes when labeling is the root cause.
• Define tool improvements and validate fixes.
• Stand up QA and reporting for new annotation programs.
• Track work in Jira and contribute to pre-release validation.

Education

BS in Computer Science/Engineering or equivalent

Tools

Jira

Job description

Gather AI is seeking an ML Annotation QA Engineer to own the quality of annotated data across computer vision and ML programs. You will judge data quality, refine verdict taxonomies, and build trackers to surface degradation patterns in the field.

Collaboration with ML engineers and QA is essential as you diagnose annotation drift, tool issues, and model performance in production. Your role involves analyzing data with Python/SQL, defending calls to external vendors, and creating SOPs to improve