Senior Data Scientist: Experimentation & Causal Inference

ByLabs; San Francisco (CA); On-site; USD 180,000 - 250,000; Full time

Job summary

ByLabs in San Francisco, CA is seeking a Senior Data Scientist for Experimentation & Causal Inference. You will own the statistical layer ensuring trustworthy, interpretable experiments and build automated systems that answer: Can we trust this result?

You will design, validate, and improve statistical frameworks on top of the platform, collaborating with data scientists and engineers within the Big Data group to deliver robust, scalable experimentation capabilities.

Qualifications

• MS or PhD in Statistics, Biostatistics, Economics (Econometrics), CS or related quantitative field.
• 3+ years designing and analyzing online controlled experiments at a tech company with meaningful user scale.
• Core knowledge: hypothesis testing, power analysis, multiple testing correction, sequential testing.
• Hands-on with at least two: CUPED/variance reduction, SRM detection, or data quality monitoring.
• Proficient in Python for statistical analysis and SQL for data extraction and validation.
• Strong communication to translate complex stats for non-technical stakeholders.
• Experience with AI coding tools and using AI to automate repetitive analytical tasks.

Responsibilities

• Design and maintain automated anomaly detection for live experiments including SRM checks.
• Define alerting thresholds and circuit-breaking criteria to protect decisions.
• Define and validate guardrail metrics for experiment sensitivity and interpretability.
• Implement CUPED and related covariate adjustment methods to reduce variance.
• Develop techniques to remove noise from historical user behavior for faster detection of true effects.
• Shorten experiment duration and enable more experiments per quarter through improved methods.
• Design and run continuous A/A health checks for the data pipeline.
• Monitor baseline volatility and ensure instrumentation stays trustworthy over time.
• Partner with product, engineering, and growth on design, sizing, interpretation; advise on causal methods when needed.
• Translate methods into specs, validation scripts, and decision frameworks for self-serve usage.

Skills

Python

SQL

Experiment design

Statistical analysis

Communication

AI tools

Education

MS/PhD in Statistics or related quantitative field

Tools

SciPy

Statsmodels

Job description

ByLabs in San Francisco, CA is seeking a Senior Data Scientist for Experimentation & Causal Inference. You will own the statistical layer ensuring trustworthy, interpretable experiments and build automated systems that answer: Can we trust this result?

You will design, validate, and improve statistical frameworks on top of the platform, collaborating with data scientists and engineers within the Big Data group to deliver robust, scalable experimentation capabilities.