Data Entry AI Evaluation Task Designer

OpenTrain AI; Northern (KY); Hybrid; USD 28,000 - 48,000; Part time

Job summary

OpenTrain AI is seeking a Data Entry AI Evaluation Task Designer to create expert-level evaluation materials for an advanced AI benchmark project. You will translate realistic data entry and validation challenges into structured tasks for AI agents.

The role suits someone experienced in data entry, QA, or data validation within regulated environments. It is a part-time, contractor position with remote work and flexible hours, paying $20-$35 per hour.

Qualifications

• Experience in data entry, quality assurance, or data validation in a regulated or audit-sensitive domain.
• Ability to document accuracy standards, error rates, and validation outcomes.
• Proficient written English.

Responsibilities

• Design tasks that simulate real-world data entry and validation.
• Construct and curate datasets using CSVs, PDFs, spreadsheets, and technical documents.
• Introduce malformed records, missing data, inconsistent formats, and formatting irregularities.
• Define the correct final state for each task, including error cases and reconciliation requirements.
• Write detailed grading rubrics with 35 or more criteria.
• Assess accuracy and completeness against clearly defined standards.
• Ensure materials reflect compliance-sensitive environments.
• Refine evaluation materials through careful asynchronous communication.

Skills

Data validation

Regulated/audit-sensitive domain

Attention to detail

Written English

Remote communication

Job description

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. It helps contributors discover projects, build a professional AI training profile, and apply in minutes. Creating an OpenTrain account is free.

• Build a portfolio around practical AI training and evaluation work.
• Find projects that match your skills and professional experience.
• Work remotely as part of a fast-growing field shaping how AI systems are built.

About AI Training Work

AI training is the human side of building modern artificial intelligence. People create examples, evaluate model outputs, and define quality standards so AI systems can learn to perform useful work accurately and reliably.

In this role, your expertise will help evaluate AI agents handling data entry and validation. The tasks you design will test whether systems can identify errors, reconcile information, and reach the correct final state.

• Contribute to cutting-edge AI evaluation work.
• Apply practical data-quality knowledge to realistic scenarios.
• Help define how AI agent performance is measured.

The Role

OpenTrain is recruiting a Data Entry AI Evaluation Task Designer to create expert-level materials for an advanced AI benchmark project. You will turn realistic data entry and validation challenges into structured evaluation tasks for AI agents.

The role combines practical experience in data entry, quality assurance, or data validation with the design of demanding datasets and grading standards for regulated or audit-sensitive work. The listing classifies this opportunity as entry level, while the required skills call for substantial professional experience in a relevant domain.

• Contractor position with part-time hours.
• 20+ hours per week.
• Pay range: $20-$35 USD per hour.
• Remote opportunity available in the countries specified in the listing.

What You’ll Do

You will create complete evaluation materials that represent the complexity of real-world data entry and validation. This includes preparing source files, documenting intentional issues, defining correct outcomes, and establishing objective standards for judging AI agent results.

The work requires precise written and verbal communication in an asynchronous environment. You will refine task materials so that errors, reconciliation requirements, and expected final states are clear and measurable.

• Design tasks that simulate real-world data entry and validation challenges.
• Construct and curate datasets using CSVs, PDFs, spreadsheets, and technical documents.
• Introduce and document malformed records, missing data, inconsistent formats, silent truncations, and formatting irregularities.
• Define the correct final state for each task, including error cases and reconciliation requirements.
• Write detailed grading rubrics with 35 or more criteria.
• Assess accuracy and completeness against clearly defined standards.
• Ensure materials reflect compliance-sensitive environments.
• Refine evaluation materials through careful asynchronous communication.

Requirements

You must have experience in data entry, quality assurance, or data validation within a regulated or audit-sensitive domain. Relevant examples include healthcare claims, finance back-office operations, or legal operations.

You should be comfortable analyzing inconsistencies across different file types and documenting accuracy standards, error rates, and validation outcomes. Strong written English, exceptional attention to detail, and a disciplined process-oriented approach are essential.

• Experience in data entry, quality assurance, or data validation in a regulated or audit-sensitive domain.
• Ability to identify malformed records, missing data, silent truncations, inconsistent formats, and reconciliation issues.
• Experience documenting accuracy standards, error rates, and validation outcomes.
• Ability to design detailed evaluation rubrics with measurable criteria for AI agent outputs.
• Proficient written English.
• Careful attention to detail and a disciplined independent work style.
• Ability to communicate proactively in a remote, distributed environment.
• English language proficiency.

Helpful Background

Experience with high-stakes compliance, data integrity, or audit requirements is valuable. Prior AI training experience is not required; practical domain knowledge and the ability to define accurate outcomes are central to the work.

• High-stakes compliance experience.
• Data integrity experience.
• Audit experience.
• Practical knowledge of regulated operational processes.