Senior Manager, AI Testing & Validation

Blend; Columbia (MD); On-site; USD 140,000 - 240,000; Full time

Benefits offered by this job

401K

Paid time off

Medical benefits

Job summary

Blend360 is seeking a Senior Manager, Intelligence Measurement & Validation to own the end-to-end testing program for a large-scale data monetization initiative. You will bridge marketing measurement, data science, and engineering to translate complex tests into actionable, defendable insights for partners.

The role involves designing testing approaches, developing automation to scale testing, and delivering concise executive readouts.

Qualifications

• 5–7+ years of experience in marketing measurement, media analytics, campaign measurement, attribution, data science, or related analytical discipline.
• Strong experience designing and executing testing and measurement methodologies, particularly backtesting, match-rate/audience coverage analysis, lift, and holdout testing.
• Experience working within clean rooms for marketing analysis, measurement, and modeling activities.
• Solid understanding of marketing and media measurement and how campaign and audience data can be used to validate intelligence and demonstrate value.
• Hands-on experience working with data and executing analyses—not simply defining requirements or reviewing someone else’s work.
• Strong analytical judgment and the ability to distinguish between what the data demonstrates and what it does not.
• Experience working cross-functionally with Data Science and Data Engineering teams.
• Ability to take an ambiguous business question, create structure around it, determine the right testing approach, and drive it through to an answer.
• Strong communication skills, with the ability to turn complex analytical results into clear, partner-ready stories.
• Excellent documentation and project management skills, with a high degree of ownership and attention to detail.
• Get-it-done mentality and ability to manage multiple analyses and partner requests while maintaining rigor and quality.

Responsibilities

• Own the end-to-end testing and validation lifecycle, including methodology, test design, documentation, execution, analysis, and readouts.
• Develop and maintain testing methodologies for backtesting, match-rate and audience coverage testing, lift/holdout testing, and other validation approaches.
• Lead the development of automation for testing methodologies to scale testing frequency and improve consistency.
• Run tests and ad-hoc analyses within clean rooms, and manage partner clean rooms.
• Run backtests that validate program intelligence against partners’ historical campaign and audience data.
• Design and execute match tests to measure household match rates, audience coverage, and data quality.
• Develop custom partner analyses based on specific business questions.
• Partner with Data Science and Data Engineering to translate test designs into executable analyses.
• Analyze results to determine what data can support and provide defensible recommendations.
• Translate complex findings into concise partner-facing presentations and executive readouts.
• Build and maintain documentation of testing methodologies, assumptions, processes, and results.
• Identify opportunities to improve the testing framework and increase repeatability.

Skills

Backtesting

Match-rate testing

Audience coverage

Lift and holdout testing

Clean rooms

Automation for testing

Cross-functional collaboration

Data analysis

Executive storytelling

Documentation and PM

Ambiguity to answer

Job description

Blend360 is seeking a Senior Manager, Intelligence Measurement & Validation to own the end-to-end testing program for a large-scale data monetization initiative. You will bridge marketing measurement, data science, and engineering to translate complex tests into actionable, defendable insights for partners.

The role involves designing testing approaches, developing automation to scale testing, and delivering concise executive readouts.