Product Data Scientist

Zof AI; San Francisco, Northern (CA, KY); Hybrid; USD 150,000 - 210,000; Full time

Job summary

Zof AI in San Francisco, CA is seeking a Product Data Scientist to measure how well our agent fleets identify and fix defects. The role owns product-level metrics like detection precision/recall and remediation success, and designs experiments to prove improvements with statistical rigor.

You will build scalable analysis pipelines in Python and SQL, translate findings into actionable guidance for engineering and leadership, and report outcomes with transparency.

Qualifications

• Fluent in Python and SQL; strong statistics background.
• Experience turning messy data into defensible conclusions.
• Ability to design experiments and measure impact.
• Comfort with causality, bias, and confounding in analysis.
• Clear written and verbal communication; reports that persuade.
• Experience in fast-moving, high-impact environments.
• Evidence of analyses that changed a decision.

Responsibilities

• Define and own metrics describing how well agent fleets find and fix defects.
• Measure detection precision and recall against real customer codebases.
• Quantify remediation success rates and validation evidence strength.
• Design experiments to show whether product or model changes help.
• Partner with engineering on eval design for consistent scoring over time.
• Build Python and SQL analysis pipelines that others can rerun and trust.
• Translate findings into recommendations for engineering, product, and leadership.
• Own reporting that grounds product claims in evidence.

Skills

Python

SQL

Statistics

Experiment design

Causal inference

Data analysis

Communication

Fast-paced environment

Tools

Data warehouses

dbt

Notebook-to-production

CI pipelines

Job description

Zof AI is seeking a Product Data Scientist to measure how well our agent fleets actually find and fix defects. This role owns the numbers behind the product: detection precision and recall, remediation success rates, experiment design, and the statistical rigor that separates a real improvement from noise. If you have worked as an Applied Scientist, Product Data Scientist, Quantitative Analyst, or Machine Learning Analyst, this is that discipline at Zof AI. The ideal candidate is fluent in Python and SQL, reasons about causes rather than correlations, and would rather report an honest number than a flattering one.

Engineering · Mid-level · Full-time · On-site · San Francisco, CA

Responsibilities

• Define and own the metrics that describe how well our agent fleets find and fix defects.
• Measure detection precision and recall against real customer codebases.
• Quantify remediation success rates and the strength of the validation evidence behind them.
• Design experiments that show whether a product or model change actually helped.
• Partner with engineering on eval design so agent runs are scored consistently over time.
• Build analysis pipelines in Python and SQL that other people can rerun and trust.
• Translate findings into clear recommendations for engineering, product, and leadership.
• Own the reporting that keeps our claims about the product grounded in evidence.

Requirements

• Strong working fluency with Python and SQL.
• Solid grounding in statistics, experiment design, and inference.
• Experience analyzing messy real-world data and drawing defensible conclusions.
• Ability to turn a fuzzy question into a metric someone can act on.
• Comfort reasoning about causality, bias, and confounding.
• Clear written and verbal communication.
• Comfort operating in a fast-moving environment.
• Evidence of analysis that changed a decision.

Nice to have

• Experience measuring the quality of machine learning or AI systems.
• Experience with causal inference methods or Bayesian analysis.
• Experience with data warehouses, dbt, or notebook-to-production workflows.
• Familiarity with software testing, CI pipelines, or developer tooling metrics.

Hands-on experience measuring the quality of AI or machine learning systems with data, plus daily use of AI tools in your own analysis work, is required