Applied Data Scientist — Metrics & Experimentation

Zof AI; San Francisco, Northern (CA, KY); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Zof AI in San Francisco, CA is seeking an Applied Data Scientist to quantify how our agent fleets detect defects and drive remediation. You will own metrics like precision/recall, error rates, and experiment design to distinguish real improvements from noise.

You should be fluent in Python and SQL, capable of turning fuzzy questions into actionable metrics, and able to communicate findings clearly to engineering, product, and leadership in a fast-moving, on-site environment.

Qualifications

• Strong working fluency with Python and SQL.
• Solid grounding in statistics, experiment design, and inference.
• Experience analyzing messy real-world data and drawing defensible conclusions.
• Ability to turn a fuzzy question into a metric someone can act on.
• Comfort reasoning about causality, bias, and confounding.
• Clear written and verbal communication.
• Comfort operating in a fast-moving environment.
• Evidence of analysis that changed a decision.

Responsibilities

• Define and own the metrics that describe how well our agent fleets find and fix defects.
• Measure detection precision and recall against real customer codebases.
• Quantify remediation success rates and the strength of the validation evidence behind them.
• Design experiments that show whether a product or model change actually helped.
• Partner with engineering on eval design so agent runs are scored consistently over time.
• Build analysis pipelines in Python and SQL that other people can rerun and trust.
• Translate findings into clear recommendations for engineering, product, and leadership.
• Own the reporting that keeps our claims about the product grounded in evidence.

Skills

Python

SQL

Statistics

Experiment design

Data analysis

Causality

Communication

Team collaboration

Tools

dbt

SQL databases

Job description

Zof AI in San Francisco, CA is seeking an Applied Data Scientist to quantify how our agent fleets detect defects and drive remediation. You will own metrics like precision/recall, error rates, and experiment design to distinguish real improvements from noise.

You should be fluent in Python and SQL, capable of turning fuzzy questions into actionable metrics, and able to communicate findings clearly to engineering, product, and leadership in a fast-moving, on-site environment.