Product Data Scientist — AI Quality & Impact (Remote)

BotStacks

United States

Remote

USD 110,000 - 150,000

Full time

5 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Remote work
Competitive compensation
Career growth

Job summary

BotStacks is seeking a product data scientist to help build evaluation datasets, run experiments, and translate results into product recommendations. This remote role focuses on measuring answer quality and outcomes for AI assistants that leverage a business’s website and knowledge.

You will apply Python and SQL to analyze usage and outcomes, design experiments with clear baselines, and deliver reproducible analyses and dashboards while working with engineering on instrumentation and

Qualifications

  • Strong Python and SQL skills.
  • Practical statistics and awareness of bias, leakage, and experimental limitations.
  • Experience evaluating AI or machine-learning systems, demonstrated through work, research, or projects.
  • Clear communication and reproducible analytical methods.

Responsibilities

  • Build evaluation datasets and tests for answer quality, grounding, retrieval, and failure modes.
  • Use Python and SQL to analyze activation, product usage, and customer outcomes.
  • Measure tradeoffs among quality, latency, and cost.
  • Design experiments with clear baselines and communicate uncertainty.
  • Produce reproducible analyses and useful dashboards.
  • Partner with engineering on instrumentation and privacy-conscious data handling.
  • Turn findings into specific product recommendations.

Skills

Python
SQL
Statistics
Experiment design
Data analysis

Job description

BotStacks is seeking a product data scientist to help build evaluation datasets, run experiments, and translate results into product recommendations. This remote role focuses on measuring answer quality and outcomes for AI assistants that leverage a business’s website and knowledge.

You will apply Python and SQL to analyze usage and outcomes, design experiments with clear baselines, and deliver reproducible analyses and dashboards while working with engineering on instrumentation and

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Scientist
Data Scientist

BotStacks • United States

Remote
USD 110,000 - 150,000
Remote work
Competitive compensation
Career growth
Founding Data Scientist: AI Experimentation & Impact
Founding Data Scientist: AI Experimentation & Impact

present of AI • San Francisco (CA)

On-site
USD 120,000 - 160,000
Senior Data Analyst: Remote, AI-Driven Impact & Experiments
Senior Data Analyst: Remote, AI-Driven Impact & Experiments

SE Ranking • United States

Remote
USD 110,000 - 150,000
Private medical insurance
Flexible work hours
Learning budget
+2
Remote AI Data Quality Analyst - Python/SQL & ML Validation
Remote AI Data Quality Analyst - Python/SQL & ML Validation

EdgeQ Technologies Private Limited • United States

Remote
USD 70,000 - 115,000
100% remote work
Health, dental, and vision insurance
Paid time off and holidays
+1
AI Analytics Lead: Experiments & Impact
AI Analytics Lead: Experiments & Impact

present of AI • San Francisco (CA)

On-site
USD 120,000 - 170,000
AI Chatbot QA Analyst - Remote 16-Week Project
AI Chatbot QA Analyst - Remote 16-Week Project

Braintrust • United States

Remote
USD 26,000 - 37,000
AI Data Quality Analyst
AI Data Quality Analyst

EdgeQ Technologies Private Limited • United States

Remote
USD 70,000 - 115,000
100% remote work
Health, dental, and vision insurance
Paid time off and holidays
+3
AI Data Quality Specialist - Remote, Flexible Hours
AI Data Quality Specialist - Remote, Flexible Hours

Prolific • United States

Remote
USD 28,000 - 40,000
PayPal account
Remote Data Science & AI Quality Analyst
Remote Data Science & AI Quality Analyst

prolificacademicltd • United States

Remote
USD 91,000 - 129,000
Remote work
AI Data Quality Specialist - Remote & Flexible Hours
AI Data Quality Specialist - Remote & Flexible Hours

Prolific Academic Ltd • United States

Remote
USD 28,000 - 40,000
Flexible hours
Work from home
Competitive pay rates