Annotation Quality Engineer for AI/ML Data Ops

Veriipro; Dallas (TX); On-site; USD 90,000 - 130,000; Full time

Job summary

Veriipro is seeking an Annotation Data Engineer to uphold annotation quality, drive ML data operations, and validate datasets for AI/ML initiatives. The role focuses on establishing quality standards, metrics, and audits across large annotation programs.

You will analyze quality trends, coordinate corrective actions with cross-functional teams, and ensure dataset releases meet training requirements using SQL and Python for reporting.

Qualifications

• 2–5 years of relevant experience in annotation quality, ML data operations, or data quality management.
• Experience supporting physical AI, robotics, computer vision, or ML programs.
• Experience developing quality metrics and KPIs.
• Experience with large-scale annotation programs and QA processes.
• Experience evaluating captions, reasoning outputs, or foundation model training datasets.
• Strong knowledge of annotation guidelines, sampling plans, acceptance criteria, and audit methodologies.
• Ability to perform quality trend analysis and drive corrective actions.
• Proficient in SQL and Python for data analysis and reporting.
• Experience creating quality dashboards and performance tracking.

Responsibilities

• Define, implement, and maintain annotation quality standards, guidelines, defect taxonomies, and acceptance criteria.
• Develop and maintain quality metrics and reporting frameworks to measure accuracy, consistency, completeness, rework rates, throughput, and reviewer agreement.
• Design and execute sampling strategies, annotation audits, escalation workflows, and root-cause analysis processes for large-scale annotation programs.
• Evaluate generated captions, reasoning outputs, and foundation model training datasets against established quality benchmarks.
• Analyze annotation quality trends, identify recurring defects, and coordinate corrective and preventive actions with cross-functional teams.
• Validate dataset releases to ensure compliance with defined quality requirements and readiness for model training.
• Maintain traceable quality records, audit documentation, issue logs, and continuous improvement plans.
• Conduct calibration sessions and support reviewer alignment to improve annotation consistency and inter-annotator agreement.
• Collaborate with annotation operations, data engineering, quality engineering, and AI/ML teams to optimize annotation workflows.
• Leverage SQL, Python, and reporting tools to analyze quality data, automate measurements, and generate actionable insights.

Skills

Annotation quality

Data quality engineering

Quality metrics

SQL

Python

Cross-functional communication

Root-cause analysis

Tools

SQL tooling

Python scripting

Job description

Veriipro is seeking an Annotation Data Engineer to uphold annotation quality, drive ML data operations, and validate datasets for AI/ML initiatives. The role focuses on establishing quality standards, metrics, and audits across large annotation programs.

You will analyze quality trends, coordinate corrective actions with cross-functional teams, and ensure dataset releases meet training requirements using SQL and Python for reporting.