Data Analyst (AI Evaluation)

Appsierra Group; United States; Remote; USD 58,000 - 115,000; Full time

Benefits offered by this job

Remote work

Contractor engagement

Job summary

Appsierra Group is seeking an experienced Data Analyst for AI Evaluation to review datasets and outputs for accuracy and consistency. You will apply structured rubrics to large data volumes, identify errors, and document decisions with clear rationale.

The role emphasizes high attention to detail, the ability to work with ambiguous scenarios, and strong written and verbal communication. Remote, U.S.-based contractor engagement with flexible hours.

Qualifications

• 3–5+ years of experience in data review, data quality, QA, analytics, or related field.
• Proven experience reviewing, validating, and critiquing data or deliverables against defined criteria.
• Ability to follow detailed written instructions and apply complex evaluation rubrics consistently.
• Strong judgment and comfort working with ambiguity.
• Excellent error-spotting and attention-to-detail skills in high-volume or repetitive work.
• Clear written and verbal communication, with ability to document findings.

Responsibilities

• Review datasets and task outputs for accuracy, completeness, consistency, and adherence to guidelines.
• Identify and flag inconsistencies, errors, and outputs that deviate from defined instructions.
• Apply structured rubrics and evaluation criteria consistently across large datasets.
• Make well-reasoned decisions in ambiguous situations where guidelines may not fully address a scenario.
• Document the rationale behind evaluation decisions clearly and consistently.
• Identify recurring patterns or systemic data-quality issues and elevate them for further analysis.
• Provide clear, actionable written and verbal feedback, including recommended resolution paths.
• Maintain high levels of accuracy and attention to detail across repetitive and high-volume data review work.

Skills

Data quality

Data review

QA processes

Analytics

Attention to detail

Tools

SQL

Spreadsheet

Job description

Data Analyst (AI Evaluation)

Contract · Remote · U.S. Only

Annual Compensation Equivalent: $57,600–$115,200/year
Hourly Rate: $30–$60/hr

The annual compensation figure is a full-time equivalent based on 40 hours per week across 48 workable weeks. It is not a guaranteed salary or earnings level.

About the Role

We are looking for experienced Data Analysts to review datasets and task outputs for accuracy, completeness, consistency, and adherence to defined evaluation criteria.

You’ll apply structured rubrics across large volumes of data, identify errors and inconsistencies, handle ambiguous cases using sound judgment, and provide clear documentation explaining your decisions. No prior AI experience is required.

What You’ll Do

• Review datasets and task outputs for accuracy, completeness, consistency, and adherence to guidelines.

• Identify and flag inconsistencies, errors, and outputs that deviate from defined instructions.

• Apply structured rubrics and evaluation criteria consistently across large datasets.

• Make well-reasoned decisions in ambiguous situations where guidelines may not fully address a scenario.

• Document the rationale behind evaluation decisions clearly and consistently.

• Identify recurring patterns or systemic data-quality issues and elevate them for further analysis.

• Provide clear, actionable written and verbal feedback, including recommended resolution paths.

• Maintain high levels of accuracy and attention to detail across repetitive and high-volume data review work.

What We’re Looking For

• 3–5+ years of experience in data review, data quality, quality assurance, analytics, or a related field.

• Proven experience reviewing, validating, and critiquing data or deliverables against defined criteria.

• Ability to follow detailed written instructions and apply complex evaluation rubrics consistently.

• Strong judgment and comfort working with ambiguity.

• Excellent error-spotting and attention-to-detail skills, particularly across high-volume or repetitive work.

• Strong written and verbal communication skills.

• Ability to clearly document findings and explain evaluation decisions.

• Baseline spreadsheet proficiency, including sorting, filtering, and simple formulas.

Preferred Experience

• Experience with SQL or hands-on data querying.

• Data annotation, data labeling, or rubric-based evaluation.

• Quality assurance or data validation workflows.

• Experience identifying recurring data-quality patterns and escalating systemic issues.

Engagement & Compensation

• Contractor engagement.

• Remote work, U.S. based.

• Hourly rate: $30–$60/hr.

• Annual compensation equivalent: $57,600–$115,200 based on 40 hours per week across 48 workable weeks.

• Actual earnings depend on the amount and scope of work assigned.

• No prior AI experience is required.