Data Scientist (Part-Time | Remote | $100 –$120/hr )

Call For Referral; San Francisco (CA); Remote; USD 137,760 - 165,312; Part time

Job summary

A leading AI research lab is seeking a part-time Data Scientist specializing in AI task evaluation and statistical analysis. You will conduct analysis on AI agent performance in finance, transforming data into actionable insights. The role offers $100–$120 per hour and allows for remote, flexible working hours. Ideal candidates should have a strong background in statistical analysis and experience with Python or R.

Qualifications

• Strong foundation in statistical analysis, hypothesis testing, and pattern recognition.
• Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis.
• Hands-on experience with exploratory data analysis (EDA) and feature interpretation.
• Understanding of AI/ML evaluation methodologies and LLM performance metrics.
• Skilled in using Excel, SQL, and data visualization tools (e.g., Tableau, Looker).

Responsibilities

• Identify recurring patterns in AI agent failures across task components.
• Determine whether issues stem from task design, rubric clarity, file complexity, or agent limitations.
• Examine performance variations across finance sub-domains, file structures, and evaluation criteria.
• Build dashboards and analytical reports highlighting edge cases and performance clusters.
• Recommend refinements to rubric design, evaluation metrics, and task structures.
• Present key insights to data labeling teams, ML engineers, and research collaborators.

Skills

Statistical analysis

Hypothesis testing

Pattern recognition

Python

R

Exploratory data analysis

Feature interpretation

AI/ML evaluation methodologies

LLM performance metrics

Excel

SQL

Data visualization tools

Job description

Data Scientist (Part-Time | Remote | $100 –$120/hr )

This range is provided by Call For Referral. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

$100.00/hr - $120.00/hr

Helping AI Startups Hire 0→1 Founding Engineers & Product Talent | Backend & Infra | US & Europe

Data Scientist (AI Task Evaluation & Statistical Analysis Specialist)

Hourly Contract | Part-Time Remote | $100 –$120 per hour

1. About the Role

Mercor is partnering with a leading AI research lab to hire experienced Data Scientists specializing in AI task evaluation and statistical analysis.

In this role, you will conduct comprehensive failure analysis on AI agent performance across finance-sector tasks — identifying systemic patterns, diagnosing performance bottlenecks, and improving model evaluation frameworks.

You’ll work closely with AI engineers and research analysts to transform raw evaluation data into actionable insights, strengthening the quality, fairness, and reliability of large-scale AI systems.

2. Key Responsibilities

• Statistical Failure Analysis: Identify recurring patterns in AI agent failures across task components (prompts, rubrics, file types, tags, etc.).
• Root Cause Analysis: Determine whether issues stem from task design, rubric clarity, file complexity, or agent limitations.
• Dimensional Analysis: Examine performance variations across finance sub-domains, file structures, and evaluation criteria.
• Visualization & Reporting: Build dashboards and analytical reports that highlight edge cases, performance clusters, and opportunities for improvement.
• Framework Enhancement: Recommend refinements to rubric design, evaluation metrics, and task structures based on empirical findings.
• Stakeholder Communication: Present key insights to data labeling teams, ML engineers, and research collaborators.

3. Required Qualifications

• Strong foundation in statistical analysis, hypothesis testing, and pattern recognition.
• Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis.
• Hands‑on experience with exploratory data analysis (EDA) and feature interpretation.
• Understanding of AI/ML evaluation methodologies and LLM performance metrics.
• Skilled in using Excel, SQL, and data visualization tools (e.g., Tableau, Looker).

4. Preferred Qualifications

• Experience with AI/ML model evaluation or quality assurance pipelines.
• Background in finance or interest in learning financial domain structures.
• Familiarity with benchmark datasets, failure mode analysis, and evaluation frameworks.
• 2–4 years of relevant professional experience in data science, analytics, or applied statistics.

5. More About the Opportunity

• Commitment: Part‑time, 20–25 hours/week
• Schedule: Fully remote and asynchronous — work on your own time
• Duration: 1–2 months, with strong potential for extension
• Start Date: Immediate

6. Compensation & Contract Terms

• Hourly Rate: $100–$120/hour (based on experience and region)
• Payments: Weekly via Stripe Connect for approved work

⚡ PS: Mercor reviews applications daily. Please complete your interview and onboarding steps to be considered for this opportunity. ⚡