AI Analytics Workflow Evaluator

OpenTrain AI

Snowflake, Northern (AZ, KY)

Hybrid

USD 69,000 - 83,000

Part time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

Remote work worldwide

Part-time contractor

20+ hours per week

English working language

Job summary

OpenTrain AI is recruiting contractors for AI training evaluation roles that focus on data-analysis workflows and validating AI-generated results in real-world scenarios.

As an AI Analytics Workflow Evaluator, you will apply SQL and Snowflake to verify outputs, manage evaluation environments, and document findings for clear scoring. This entry-level role supports a remote, flexible schedule while demanding strong analytics judgment and communication.

Qualifications

Advanced SQL skills for validating AI-generated analytical results.
Deep Snowflake knowledge including warehouses, roles and access controls.
Ability to reconcile metrics with raw data and explain evaluation decisions.
Experience with KPI definitions and leadership reporting is helpful.

Responsibilities

Run structured evaluation scenarios and compare AI outputs with source data.
Maintain reproducible evaluation environments and document findings.
Investigate undocumented product behavior and test analytics connections.
Participate in calibration sessions to support consistent grading and scoring.

Skills

Advanced SQL

Snowflake knowledge

Data analysis judgement

KPI reporting

Communication skills

Rubric-based evaluation

Security integrations

Analytics connectors

Tools

Snowflake

Claude connectors

ChatGPT connectors

OAuth

Slack

Google Workspace

Job description

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. OpenTrain AI is recruiting contractors for projects that help improve the systems behind modern AI assistants.

About AI Training Work

AI training is the human side of building artificial intelligence. Contributors review examples, test model behavior, evaluate responses, and identify errors so AI systems become more accurate, reliable, and useful.

This role focuses on evaluating AI assistants in realistic data-analysis workflows. Your analytics judgment will help determine whether generated SQL, figures, joins, filters, and time windows are accurate and logically sound.

Remote work available worldwide
Part-time contractor opportunity with 20+ hours per week
Work directly with cutting-edge AI evaluation tasks
Use practical analytics expertise to improve AI systems

The Role

As an AI Analytics Workflow Evaluator, you will assess how AI assistants perform data-analysis tasks connected to cloud data warehouses. The work combines hands-on SQL analysis, Snowflake environment management, product-behavior investigation, and careful evaluation of AI-generated results.

This opportunity is listed at an entry level, while advanced SQL and Snowflake capability are important for successful performance. At least three years of hands-on experience as a data analyst or analytics engineer is preferred.

Payment: $50-$60 per hour
Payment type: hourly
Engagement: part-time contractor
Expected availability: 20+ hours per week
Working language: English

What You'll Do

You will execute structured evaluation scenarios and compare AI-generated analytical outputs with source data. You will also maintain reproducible evaluation environments and document findings so scoring remains clear and consistent.

The work may include investigating undocumented product behavior, testing analytics connections, and participating in calibration sessions with other evaluators.

Run scenarios involving anomaly investigation, KPI reporting, and data refreshes
Write and execute SQL queries to verify AI-generated results against source data
Audit figures, joins, filters, time windows, aggregations, and analytical logic
Maintain, reset, and manage seeded Snowflake datasets
Administer warehouse roles, permissions, grants, and access controls
Configure and document connectivity and authentication across analytics and SaaS platforms
Investigate undocumented product behavior and record findings clearly
Participate in calibration sessions to support consistent grading and scoring

Required Skills

This role requires strong practical analytics judgment. You should be able to reconcile reported metrics with raw data, recognize subtle aggregation or logic errors, and explain your evaluation decisions clearly.

Experience administering database access and working with authentication or security integrations is valuable. Familiarity with business and finance analytics, including KPI definitions and reporting for leadership or external stakeholders, is helpful.

Advanced SQL skills for validating AI-generated analytical results
Snowflake expertise covering warehouses, query history, roles, grants, and access controls
Ability to reconcile reported metrics with raw data
Ability to identify aggregation, filtering, joining, or logic errors
Strong written and verbal communication
Careful documentation and sound judgment in rubric-based evaluation and QA
Understanding of KPI definitions and business or finance reporting practices
Experience using Claude or ChatGPT connectors for analytics workflows

Preferred Background

A background as a data analyst or analytics engineer will help you contribute effectively, particularly if you have worked with cloud data warehouses and stakeholder-facing reporting.

Experience reviewing AI-generated SQL or analytical outputs, rubric-based evaluation, quality assurance, or data labeling is useful. Familiarity with OAuth, security integrations, Slack, Google Workspace, or Microsoft 365 is also beneficial.

At least three years of hands-on data analyst or analytics engineering experience preferred
Experience reviewing AI-generated SQL and analytical outputs
Background in rubric-based evaluation, quality assurance, or data labeling
Familiarity with OAuth and security integrations
Experience with Slack, Google Workspace, or Microsoft 365

Why Build an AI Training Career

AI training and data labeling are among the fastest-growing ways to work in tech. People with analytics, coding, language, or other specialized expertise help shape how state-of-the-art AI systems behave.

The work is remote and often flexible, making it possible to build experience around other commitments. OpenTrain helps you find relevant projects, apply efficiently, and develop a credible portfolio as your AI training career grows.