Data Scientist AI

Levy NL; Utrecht; On-site; EUR 70,000 - 100,000; Full time

Job summary

Levy NL in Utrecht is seeking a GenAI Data Scientist to enhance the quality of its in-house chatbot. This role focuses on context engineering, system evaluation, and diagnosing why the chatbot deviates from expected behavior.

You will benchmark the current solution against state of the art, and push innovation across retrieval, generation, reasoning loops and tool calling.

Qualifications

• Hands-on experience with LLM-based applications and retrieval-augmented generation (RAG).
• Solid grasp of prompts, context windows, retrieval and grounding.
• Ability to debug data flows and context ingestion issues with engineers.
• Experience designing evaluation approaches beyond classical ML metrics.
• Clear communication with engineers, product owners and stakeholders.

Responsibilities

• Analyse chatbot outputs and pinpoint the improvements with the biggest impact.
• Identify context gaps and define additional information the agents need.
• Define evaluation criteria for agent performance with product owner and engineers.
• Work with use-case teams and stakeholders to understand scenarios and translate feedback into improvements.
• Improve retrieval, data structuring, and onboarding of new context sources.
• Build evaluation frameworks to support fast, large-scale experimentation.
• Translate functional requirements into concrete technical improvements.

Skills

LLM-based apps

RAG retrieval

Prompts & context windows

APIs & batch syncing

Data flow debugging

Evaluation design

Stakeholder communication

Tools

APIs

Data pipelines

Job description

Our client, a large organisation in the Dutch financial sector, is looking for a GenAI Data Scientist to raise the quality of its in-house chatbot. Your job is to make sure the chatbot actually does what users need it to do.

This is not a model training or ML research role. The underlying models are off-the-shelf; the real challenge lies in context engineering, system evaluation, and diagnosing why the chatbot behaves the way it does. The central question you help answer: why isn't the chatbot doing what we expect, and what context or structure does it need to get there?

You'll also benchmark the current solution against the state of the art and push innovation across retrieval, generation, reasoning loops, tool calling, and related GenAI techniques.

What you’ll do

• Analyse chatbot outputs and pinpoint the improvements with the biggest impact
• Identify context gaps and define what additional information the agents need
• Define evaluation criteria for agent performance together with the product owner and engineers
• Work directly with use-case teams and stakeholders to understand real-world scenarios, feed those into evaluation, and turn feedback into improvements
• Improve retrieval, data structuring, and the onboarding of new context sources
• Build evaluation frameworks that support fast, large-scale experimentation
• Translate functional requirements into concrete technical improvements

What you bring

• Hands-on experience with LLM-based applications and retrieval-augmented generation (RAG)
• A solid grasp of prompts, context windows, retrieval, and grounding, plus the judgement to say whether an output is good or bad, and why
• Experience with API integrations, batch syncs, and both structured and unstructured data
• The ability to debug data flows and context ingestion issues alongside engineers
• Experience designing evaluation approaches that go beyond classical ML metrics
• Strong functional understanding: you know why a chatbot exists and which user behaviour matters
• Clear communication skills, both in conversations with engineers, product owners and stakeholders, and in written improvement proposals

Nice to have

• Experience with human-in-the-loop evaluation

Location: Utrecht
Type: Contract - To- Perm
Duration: 12 months (Possible to go internal)
Hours: 36
Must be located in the Netherlands

You’ll thrive here if...

You're genuinely curious about why AI systems behave the way they do and how to make them better. You enjoy debugging behaviour, not just code. You think in structures, connect the dots quickly, and have a strong instinct for what "good" looks like.