Remote Data Scientist for AI Evaluation & Analytics (Part-Time)

Call For Referral; San Francisco (CA); Remote; USD 137,760 - 165,312; Part time

Job summary

A leading AI research lab is seeking a part-time Data Scientist specializing in AI task evaluation and statistical analysis. You will conduct analysis on AI agent performance in finance, transforming data into actionable insights. The role offers $100–$120 per hour and allows for remote, flexible working hours. Ideal candidates should have a strong background in statistical analysis and experience with Python or R.

Qualifications

• Strong foundation in statistical analysis, hypothesis testing, and pattern recognition.
• Proficiency in Python (pandas, scipy, matplotlib/seaborn) or R for data analysis.
• Hands-on experience with exploratory data analysis (EDA) and feature interpretation.
• Understanding of AI/ML evaluation methodologies and LLM performance metrics.
• Skilled in using Excel, SQL, and data visualization tools (e.g., Tableau, Looker).

Responsibilities

• Identify recurring patterns in AI agent failures across task components.
• Determine whether issues stem from task design, rubric clarity, file complexity, or agent limitations.
• Examine performance variations across finance sub-domains, file structures, and evaluation criteria.
• Build dashboards and analytical reports highlighting edge cases and performance clusters.
• Recommend refinements to rubric design, evaluation metrics, and task structures.
• Present key insights to data labeling teams, ML engineers, and research collaborators.

Skills

Statistical analysis

Hypothesis testing

Pattern recognition

Python

R

Exploratory data analysis

Feature interpretation

AI/ML evaluation methodologies

LLM performance metrics

Excel

SQL

Data visualization tools

Job description

A leading AI research lab is seeking a part-time Data Scientist specializing in AI task evaluation and statistical analysis. You will conduct analysis on AI agent performance in finance, transforming data into actionable insights. The role offers $100–$120 per hour and allows for remote, flexible working hours. Ideal candidates should have a strong background in statistical analysis and experience with Python or R.