Get a reply from this employer
Your resume and cover letter, tailored to exactly what they’re hiring for.
Nepa is seeking a Data Scientist to shape how AI outputs are evaluated and guarded in marketing analytics. You will review LLM evaluation literature and design automated tests, building a robust evaluation framework for real client questions.
You will implement guardrail strategies on Nepa's MMM MCP and help define best practices for QC of AI outputs used in client-facing settings. This role offers hands-on experimentation with anonymised data and collaboration across the data science team.
At Nepa, we help great brands grow by turning insight into action. Our strategic marketing intelligence and optimisation platform brings Brand Tracking, Marketing Mix Modelling and Campaign Evaluation into one AI-powered view. More than 300 blue-chip clients across 50+ markets rely on it, including Fjällräven, Kellogg’s and Wolt. We’re profitable, we’re growing, and we’re building for the AI era. Now we’re looking for a Data Scientist to help shape what comes next.
Nepa helps brands understand what drives their growth. One of our core offerings is marketing mix modelling (MMM): Bayesian models that estimate how much each marketing channel contributes to sales and other business outcomes, and how those outcomes would respond to changes in spend.
We are now making our MMM results available to AI assistants via the Model Context Protocol (MCP). Clients will be able to ask questions in plain language, such as "Which channels gave the best return last quarter?" or "What happens if we move budget from TV to social?", and get answers based on their own models.
This opens up powerful new ways of working with insights, but it also introduces a new risk. Large language models are non-deterministic and can be persuasive even when they are wrong. An answer can quote every number correctly and still lead a client to the wrong conclusion. In a client-facing setting, where real budget decisions are made, that is not acceptable.
How do we make sure that what an AI assistant tells a client is both accurate and hard to misinterpret?
MMM results are particularly easy to misread. For example, an assistant might:
present a correlation as if it proved cause and effect
extrapolate a response curve far beyond the spend levels the model has actually observed
compare or add metrics that are calculated on different bases (e.g. short- vs long-term ROI)
state a single point estimate with confidence when the model's uncertainty is large
answer a question the data cannot actually support, instead of saying so.
What kinds of errors and misinterpretations arise when LLM agents communicate model results, and how often do they occur?
Can these be detected automatically and reliably, for example with rule-based checks, claim verification against the underlying data, or LLM-as-judge approaches? How well do the automatic checks agree with expert assessment?
Which guardrail strategies reduce these problems most effectively (e.g. tool and response design, interpretation context returned along with the data, a separate verification step), and what do they cost in latency, compute and usefulness? Being overly cautious and vague is a failure too.
The exact scope will be shaped together with you, based on your background and interests.
Review the literature on LLM evaluation, hallucination detection and guardrails
Build a test set of realistic client questions with expert-validated reference answers
Design and implement an automated evaluation framework
Implement and compare guardrail strategies on Nepa's MMM MCP
Recommend how Nepa should QC client-facing AI outputs going forward
You will work with anonymised or synthetic data, so the results can be published.
A real problem at the frontier of applied AI, whose results will directly shape a product in development
Supervision from Nepa's MMM data science team, along with close collaboration with the team building the MCP