Applied Data Scientist

Zof AI; San Francisco, Northern (CA, KY); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Zof AI in San Francisco, CA is seeking an Applied Data Scientist to quantify how our agent fleets detect defects and drive remediation. You will own metrics like precision/recall, error rates, and experiment design to distinguish real improvements from noise.

You should be fluent in Python and SQL, capable of turning fuzzy questions into actionable metrics, and able to communicate findings clearly to engineering, product, and leadership in a fast-moving, on-site environment.

Qualifications

• Strong working fluency with Python and SQL.
• Solid grounding in statistics, experiment design, and inference.
• Experience analyzing messy real-world data and drawing defensible conclusions.
• Ability to turn a fuzzy question into a metric someone can act on.
• Comfort reasoning about causality, bias, and confounding.
• Clear written and verbal communication.
• Comfort operating in a fast-moving environment.
• Evidence of analysis that changed a decision.

Responsibilities

• Define and own the metrics that describe how well our agent fleets find and fix defects.
• Measure detection precision and recall against real customer codebases.
• Quantify remediation success rates and the strength of the validation evidence behind them.
• Design experiments that show whether a product or model change actually helped.
• Partner with engineering on eval design so agent runs are scored consistently over time.
• Build analysis pipelines in Python and SQL that other people can rerun and trust.
• Translate findings into clear recommendations for engineering, product, and leadership.
• Own the reporting that keeps our claims about the product grounded in evidence.

Skills

Python

SQL

Statistics

Experiment design

Data analysis

Causality

Communication

Team collaboration

Tools

dbt

SQL databases

Job description

Zof AI is seeking an Applied Data Scientist to measure how well our agent fleets actually find and fix defects. This role owns the numbers behind the product: detection precision and recall, remediation success rates, experiment design, and the statistical rigor that separates a real improvement from noise. If you have worked as an Applied Scientist, Product Data Scientist, Quantitative Analyst, or Machine Learning Analyst, this is that discipline at Zof AI. The ideal candidate is fluent in Python and SQL, reasons about causes rather than correlations, and would rather report an honest number than a flattering one.

Engineering · Mid-level · Full-time · On-site · San Francisco, CA

Responsibilities

• Define and own the metrics that describe how well our agent fleets find and fix defects.
• Measure detection precision and recall against real customer codebases.
• Quantify remediation success rates and the strength of the validation evidence behind them.
• Design experiments that show whether a product or model change actually helped.
• Partner with engineering on eval design so agent runs are scored consistently over time.
• Build analysis pipelines in Python and SQL that other people can rerun and trust.
• Translate findings into clear recommendations for engineering, product, and leadership.
• Own the reporting that keeps our claims about the product grounded in evidence.

Requirements

• Strong working fluency with Python and SQL.
• Solid grounding in statistics, experiment design, and inference.
• Experience analyzing messy real-world data and drawing defensible conclusions.
• Ability to turn a fuzzy question into a metric someone can act on.
• Comfort reasoning about causality, bias, and confounding.
• Clear written and verbal communication.
• Comfort operating in a fast-moving environment.
• Evidence of analysis that changed a decision.

Nice to have

• Experience measuring the quality of machine learning or AI systems.
• Experience with causal inference methods or Bayesian analysis.
• Experience with data warehouses, dbt, or notebook-to-production workflows.
• Familiarity with software testing, CI pipelines, or developer tooling metrics.

Hands-on experience measuring the quality of AI or machine learning systems with data, plus daily use of AI tools in your own analysis work, is required