Remote Annotation Data Engineer – ML Quality & Data Ops

The Judge Group; Dallas (TX); Remote; USD 90.000 - 100.000; Vollzeit

Benefits dieser Stelle

Medical, dental, and vision

Retirement plan with company contrib.

Paid time off and holidays

Career development

Zusammenfassung

The Judge Group is seeking an Annotation Data Engineer to support large-scale annotation programs. You will define quality standards, develop metrics, and oversee audits of captions and foundation model data across remote teams.

This full-time role allows CST or EST-based coordination with cross-functional AI and engineering teams. You will evaluate generated captions and outputs, define sampling plans and defect taxonomies, and ensure traceable quality records.

Qualifikationen

• Demonstrated experience in annotation quality, ML data operations, quality engineering, or data quality.
• Experience developing quality metrics and performance measurement systems.
• Experience working with large-scale annotation programs.

Aufgaben

• Define and maintain annotation quality standards, guidelines, defect categories, and acceptance criteria.
• Develop metrics and reporting for accuracy, consistency, completeness, rework, throughput, and reviewer agreement.
• Design sampling, audit, escalation, and root-cause analysis processes for large-scale annotation programs.

Kenntnisse

Annotation quality

Quality metrics

Large-scale annotation

Caption evaluation

Annotation instructions

Quality trend analysis

SQL

Python

Tools

SQL

Python

Reporting tools

Jobbeschreibung

The Judge Group is seeking an Annotation Data Engineer to support large-scale annotation programs. You will define quality standards, develop metrics, and oversee audits of captions and foundation model data across remote teams.

This full-time role allows CST or EST-based coordination with cross-functional AI and engineering teams. You will evaluate generated captions and outputs, define sampling plans and defect taxonomies, and ensure traceable quality records.