Experimentation Analytics Lead

Expert Technology Services; San Francisco (CA); On-site; USD 140,000 - 210,000; Full time

Job summary

Expert Technology Services in San Francisco is seeking a data/experimentation specialist to design, run, and analyze digital A/B tests and multivariate experiments. You will apply rigorous statistics to validate results and drive product improvements.

You will work with large datasets using SQL and analytics tools, leveraging platforms like Adobe Target, Optimizely, Statsig, or LaunchDarkly. Strong communication helps translate insights into business impact.

Qualifications

• Experience conducting A/B tests and multivariate experiments.
• Strong statistical knowledge for validating experiments.
• Proficiency with SQL and analytics tools.

Responsibilities

• Conduct and analyze digital A/B tests and multivariate experiments to drive product improvements.
• Utilize statistical methods to ensure experiment validity.
• Work with large-scale datasets using SQL and analytics tools.
• Leverage experimentation platforms such as Adobe Target, Optimizely, Statsig, or LaunchDarkly.
• Apply advanced experimentation techniques and interpret ecommerce metrics for insights.
• Present results as actionable business recommendations.

Skills

Statistical knowledge

Hypothesis testing

Power analysis

Confidence intervals

Experimental design

SQL

Python or R

Data analysis

Communication of statistics

Tools

Adobe Target

Optimizely

Statsig

LaunchDarkly

Job description

Job Summary (List Format):

• Conduct and analyze digital A/B tests and multivariate experiments to drive product improvements.


• Utilize strong statistical knowledge (hypothesis testing, power analysis, confidence intervals, variance, experimental design) to ensure experiment validity.


• Work extensively with large-scale behavioral or event-level datasets using SQL and analytics tools.


• Leverage experimentation platforms such as Adobe Target, Optimizely, Statsig, LaunchDarkly, or similar.


• Apply advanced experimentation techniques (CUPED, sequential testing, Bayesian methods, causal inference, heterogeneous treatment effects).


• Interpret ecommerce product metrics and digital customer journeys to provide actionable insights.


• Identify and address experiment quality issues, effectively communicating statistical concepts to non-technical stakeholders.


• Preferably use Python or R for data analysis and experiment implementation.


• Present experiment results and insights as clear business recommendations.


• Collaborate cross-functionally within a retail environment, working onsite in San Francisco.


• Maintain strong communication skills and contribute to team knowledge sharing.


• Ensure all candidate submissions include a LinkedIn profile; background checks required.