Master's thesis: Guardrails for AI-generated marketing insights

Nepa

Stockholms kommun

On-site

SEK 550,000 - 850,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Nepa is seeking a Data Scientist to shape how AI outputs are evaluated and guarded in marketing analytics. You will review LLM evaluation literature and design automated tests, building a robust evaluation framework for real client questions.

You will implement guardrail strategies on Nepa's MMM MCP and help define best practices for QC of AI outputs used in client-facing settings. This role offers hands-on experimentation with anonymised data and collaboration across the data science team.

Qualifications

  • Final-year master’s student in computer science, data science, statistics, or related field.
  • Comfortable programming in Python.
  • Curious about LLMs and agentic systems; hands-on experience is a plus.
  • Familiarity with statistics, ideally Bayesian; knowledge of marketing not required.
  • Analytical, structured, and good at explaining technical things clearly.

Responsibilities

  • Review the literature on LLM evaluation, hallucination detection and guardrails.
  • Build a test set of realistic client questions with expert-validated reference answers.
  • Design and implement an automated evaluation framework.
  • Implement and compare guardrail strategies on Nepa's MMM MCP.
  • Recommend how Nepa should QC client-facing AI outputs going forward.

Skills

Python programming
Bayesian statistics
LLMs & agentic systems
Analytical thinking
Clear communication

Education

Master's degree in CS/DS

Job description

At Nepa, we help great brands grow by turning insight into action. Our strategic marketing intelligence and optimisation platform brings Brand Tracking, Marketing Mix Modelling and Campaign Evaluation into one AI-powered view. More than 300 blue-chip clients across 50+ markets rely on it, including Fjällräven, Kellogg’s and Wolt. We’re profitable, we’re growing, and we’re building for the AI era. Now we’re looking for a Data Scientist to help shape what comes next.

Background

Nepa helps brands understand what drives their growth. One of our core offerings is marketing mix modelling (MMM): Bayesian models that estimate how much each marketing channel contributes to sales and other business outcomes, and how those outcomes would respond to changes in spend.

We are now making our MMM results available to AI assistants via the Model Context Protocol (MCP). Clients will be able to ask questions in plain language, such as "Which channels gave the best return last quarter?" or "What happens if we move budget from TV to social?", and get answers based on their own models.

This opens up powerful new ways of working with insights, but it also introduces a new risk. Large language models are non-deterministic and can be persuasive even when they are wrong. An answer can quote every number correctly and still lead a client to the wrong conclusion. In a client-facing setting, where real budget decisions are made, that is not acceptable.

The problem

How do we make sure that what an AI assistant tells a client is both accurate and hard to misinterpret?

MMM results are particularly easy to misread. For example, an assistant might:

present a correlation as if it proved cause and effect

extrapolate a response curve far beyond the spend levels the model has actually observed

compare or add metrics that are calculated on different bases (e.g. short- vs long-term ROI)

state a single point estimate with confidence when the model's uncertainty is large

answer a question the data cannot actually support, instead of saying so.

What kinds of errors and misinterpretations arise when LLM agents communicate model results, and how often do they occur?

Can these be detected automatically and reliably, for example with rule-based checks, claim verification against the underlying data, or LLM-as-judge approaches? How well do the automatic checks agree with expert assessment?

Which guardrail strategies reduce these problems most effectively (e.g. tool and response design, interpretation context returned along with the data, a separate verification step), and what do they cost in latency, compute and usefulness? Being overly cautious and vague is a failure too.

The exact scope will be shaped together with you, based on your background and interests.

What you will do

Review the literature on LLM evaluation, hallucination detection and guardrails

Build a test set of realistic client questions with expert-validated reference answers

Design and implement an automated evaluation framework

Implement and compare guardrail strategies on Nepa's MMM MCP

Recommend how Nepa should QC client-facing AI outputs going forward

You will work with anonymised or synthetic data, so the results can be published.

Who we're looking for
  • Final-year master's student in computer science, data science, statistics, engineering physics or a related field
  • Comfortable programming in Python
  • Curious about LLMs and agentic systems; hands-on experience is a plus
  • Familiarity with statistics, ideally Bayesian; knowledge of marketing is not required
  • Analytical, structured, and good at explaining technical things clearly
What we offer

A real problem at the frontier of applied AI, whose results will directly shape a product in development

Supervision from Nepa's MMM data science team, along with close collaboration with the team building the MCP

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Guardrails for AI-Generated Marketing Insights (MMM) Thesis
Guardrails for AI-Generated Marketing Insights (MMM) Thesis

Nepa • Stockholms kommun

On-site
SEK 550,000 - 850,000
Data Scientist
Data Scientist

Nepa • Stockholms kommun

On-site
SEK 550,000 - 850,000
Data Scientist: End-to-End ML for Marketing AI
Data Scientist: End-to-End ML for Marketing AI

Nepa • Stockholms kommun

On-site
SEK 550,000 - 850,000
Applied AI Engineer
Applied AI Engineer

Nooga • Stockholms kommun

Hybrid
SEK 655,236 - 873,648
AI Ops Engineer (Marketing)
AI Ops Engineer (Marketing)

Lovable • Stockholms kommun

On-site
SEK 900,000 - 1,500,000
ML Engineering Lead (LLM Ops)
ML Engineering Lead (LLM Ops)

Neko Health AB • Stockholms kommun

On-site
SEK 800,000 - 1,100,000
Software Engineer
Software Engineer

Lemonado • Stockholms kommun

On-site
SEK 600,000 - 1,000,000
Equity
Applied AI Engineer
Applied AI Engineer

Redpine • Stockholms kommun

On-site
SEK 900,000 - 1,100,000
Research Engineering Lead
Research Engineering Lead

Lovable • Stockholms kommun

On-site
SEK 85,000 - 120,000
AI Solutions Technical Engineering Manager
AI Solutions Technical Engineering Manager

Fdj United • Stockholms kommun

On-site
SEK 1,000,000 - 1,800,000