Incrementality & Media Experimentation Manager

NBCUniversal; New York (NY); Hybrid; USD 100,000 - 160,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Vision Insurance

401(k)

Paid Leave

Job summary

NBCUniversal’s Peacock unit is expanding its Media Experimentation & Incrementality program. The Manager leads the design, execution, and quality of incrementality studies to identify media that drives subscriber growth, working within the Media Analytics team to set the learning agenda and ensure defensible results.

This is a technical role requiring mastery of causal inference methods (RCTs, geo and matched designs, diff‑in‑dif), SQL, and Python/R to independently validate vendor outputs and

Qualifications

• 4+ years in marketing/media experimentation with direct design/QA ownership and experience partnering with an incrementality testing measurement vendor (e.g., Haus, Measured, or similar).
• Proven command of causal inference in media—appropriate use of RCT, geo, diff‑in‑diff, and platform holdouts—and the ability to evaluate vendor assumptions/methods.
• Familiarity with power analysis, and failure modes (leakage, seasonality, parallel trends); history of stopping or redesigning under‑powered tests.
• Platform experience: Google/Meta/TikTok incrementality and at least one CTV/Device environment (e.g., Roku, Samsung, Vizio).
• Experience reconciling vendor experiment reads with MMM/MTA and informing planning/pacing decisions without surprises.
• Fluent in SQL and Excel.
• Knowledge of big data infrastructure and data science techniques.
• Effective communicator and collaborator with strong project management; ability to work in a highly matrixed environment and influence media buyers/channel leads.

Responsibilities

• Own statistical rigor across the program: review and approve study designs, evaluate vendor methods and assumptions, enforce power standards, and reject or redesign under‑powered or invalid proposals.
• Shape the incrementality testing roadmap with Media Analytics leadership: define hypotheses, decision criteria, and power thresholds, and own end‑to‑end execution of the roadmap.
• Run independent validations: monitor for leakage, seasonality confounds, and design defects, and pressure‑test vendor deliverables with targeted SQL, R, or Python analysis.
• Triangulate experiment results with MMM and MTA: drive calibration across methods, propose tiebreakers where results diverge, and contribute to a unified view of incrementality across the measurement portfolio.
• Manage the day‑to‑day relationship with our experimentation vendor: prioritize the study backlog, align on methodology, maintain testing schedules, and escalate blockers early.
• Produce clear, defensible readouts: quantify lift and uncertainty, write concise decision memos, and hand off guidance to the Optimization & Insights team for in‑market execution and pacing.
• Support the expansion of causal measurement into subscriber engagement, including holdout‑based approaches.
• Mentor a Media Analytics analyst and uplevel partner teams on test design, statistical power, and common pitfalls, contributing to a standardized measurement approach across NBCU's Media Group.

Skills

SQL

Excel

Statistical reasoning

Communication

Project management

Causal inference

Python

R

Tools

Python

R

Job description

NBCUniversal’s Peacock unit is expanding its Media Experimentation & Incrementality program. The Manager leads the design, execution, and quality of incrementality studies to identify media that drives subscriber growth, working within the Media Analytics team to set the learning agenda and ensure defensible results.

This is a technical role requiring mastery of causal inference methods (RCTs, geo and matched designs, diff‑in‑dif), SQL, and Python/R to independently validate vendor outputs and