Senior Product Manager - Tech, Experimentation & Failure

Steven.com; Greater London; On-site; GBP 110,000 - 140,000; Full time

Job summary

Steven.com in London is seeking a Senior Product Manager for Experimentation & Failure to lead a high-velocity program. You will set hypotheses, isolate variables, and run rigorous experiments across shows, creators, and IP properties, building a culture that learns faster through deliberate failure.

You will collaborate with the VP of Engineering & Applied AI to apply AI tooling to experiment design, analysis, and reporting, ensuring scalable testing at velocity.

Qualifications

• Proven experience leading product or content experimentation programs in tech/media.
• Strong grasp of statistics, experimental design, and power.
• Ability to form clear hypotheses and isolate variables in tests.
• Collaboration with engineers and data scientists to implement tests.

Responsibilities

• Lead the Experimentation & Failure team, setting hypotheses and single-variable tests.
• Drive high-velocity experimentation across shows, creators, IPs, and content.
• Ensure experiments are properly powered, with defined success/failure criteria.
• Build a centralized repository of experiments and learnings for reuse.
• Partner with Engineering & Applied AI to apply tooling to design, analysis and reporting.

Skills

Experimentation strategy

Statistical power

Hypothesis testing

Data collaboration

Written & verbal communication

Job description

SENIOR PRODUCT MANAGER - TECH, EXPERIMENTATION & FAILURE

COMPANY: STEVEN.COM

REPORTING TO: CITO

LOCATION: LONDON

ABOUT STEVEN.COM

Steven.com is building the operating system for the creator economy, forecast to pass a trillion dollars by the early 2030's. This industry is currently held back by fragmented tools and a lack of professional infrastructure. Steven.com is the unlock. We are the end-to-end Operating System designed to scale what is irreplaceably human.

We have built proprietary technology and obsessed teams to identify and scale the highest-potential creators across our core pillars:

• Creator Media: Amplifying reach, influence, and trust
• Creator Community: Transforming audiences into connected tribes
• Creator Products: Providing creators with the infrastructure to build and back aligned products and ventures
• Powered by Creator Tech & Intelligence: A proprietary data and technology suite that fuels smarter decisions and drives innovation across the entire flywheel.

Our Experimentation & Failure team has one of the most strategically important and unusual mandates at Steven.com: increase the rate of failure. As Steven Bartlett puts it: "the path to the correct answer is out-failing your competition." This isn't a growth team or an optimisation function. It's the team that exists to make sure we learn faster than anyone else.

ROLE MISSION

Lead the Experimentation & Failure team, reporting to the CITO. You'll out-experiment and out-fail the competition - running high-velocity, rigorous experiments across every show, creator, piece of content, and commercial bet at Steven.com, while building a team and a culture that treats deliberate failure as the primary learning mechanism. This is deeply hands-on: you'll be setting hypotheses, isolating variables, checking statistical power, reading results, and moving to the next test - not directing from a distance.

KEY OUTCOMES

• Own and drive the experimentation agenda across every show, creator, and IP property in the FlightStory portfolio - from podcast topic selection and episode structure through to thumbnail design and social tile copy. No detail too small to test.
• Lead hypothesis formation for every experiment, with clear success, failure, and inconclusive criteria defined in advance - and enforce single-variable discipline across the board.
• Ensure every experiment is adequately powered before launch: sample sizes calculated, measurement windows defined, results interpretable by design.
• Systematically increase experimentation velocity and build the intake process that makes high-volume testing the default across FlightStory and Steven.com.
• Build institutional memory - a searchable, structured record of every experiment run, what was learned, and what was decided - as a compounding organisational asset.
• Partner with the VP of Engineering & Applied AI to apply AI tooling to experiment design, analysis, and reporting, and to ensure infrastructure supports testing at this velocity.

CORE COMPETENCIES

• Deep, first-principles command of experimentation mathematics: statistical significance, power, sample size calculation, p-values, confidence intervals, Type I/II errors, and the difference between statistical and practical significance.
• Genuine mastery of the scientific method applied to product and content - hypothesis formation, single-variable isolation, measurement design, and result interpretation.
• A track record of building experimentation culture, not just running tests - creating an environment where the whole team experiments and failure is rewarded.
• Comfortable querying data and working shoulder-to-shoulder with engineers and data scientists at implementation depth.
• Strong written and verbal communication - able to write a hypothesis an engineer respects and explain a result a producer will act on.
• Experience operating at pace, in high-volume testing environments where speed of learning is the competitive edge.

YOU'LL THRIVE HERE IF

• You think in hypotheses, not features.
• You isolate one variable at a time, and understand viscerally why changing five things at once makes a result meaningless.
• You give a null or inconclusive result the same intellectual respect as a win - you know how to extract the signal either way.
• You're obsessive about measurement: an experiment that can't be measured is just a change, not an experiment.
• You're deeply sceptical of your own results, and design experiments to prove yourself wrong rather than confirm what you already believe.
• You move fast, expect others to move fast, and don't wait for perfect conditions to run a test.
• You believe failure is feedback, feedback is knowledge, and knowledge is power - and you build systems to generate that knowledge at the highest possible rate.

IDEAL BACKGROUND

• Demonstrable experience leading (not just participating in) product or content experimentation programs at a technology, media, or creator-economy company - owning the methodology, volume, and culture.
• Strong advantage: experience with algorithmic platforms and how to design experiments against platform-specific metrics; podcasting, video, social, or creator-economy background; familiarity with YouTube/Spotify/social analytics (CTR, retention, watch time, audience behaviour).
• Nice to have: experience building an experimentation platform from scratch; familiarity with causal inference beyond standard A/B testing (holdouts, quasi-experiments, diff-in-diff); experience experimenting on AI/ML systems or prompt variations in production; a background in statistics, maths, CS, economics, or a natural science; exposure to early-stage environments where you had to build the experimentation infrastructure yourself.