Data Scientist

Infinity Quest Limited; Utrecht; On-site; EUR 70,000 - 100,000; Full time

Job summary

Infinity Quest Limited, Utrecht, is seeking an AI Evaluation Engineer to analyze chatbot outputs, diagnose context gaps, and collaborate with product and engineering teams to define rigorous evaluation criteria. You will improve retrieval and data structures, design scalable experiment frameworks, and translate requirements into concrete technical improvements.

You will work closely with cross-functional teams to understand real-world scenarios, evaluate model performance, and drive continuous

Qualifications

• Strong analytical mindset and curiosity about AI behavior.
• Solid understanding of GenAI systems.
• Experience with LLM-based applications.
• Experience with retrieval-augmented systems.
• Strong knowledge of prompts, context windows, and grounding.
• Ability to evaluate output quality and explain results.

Responsibilities

• Analyse chatbot outputs and identify the most impactful improvements.
• Diagnose context gaps and determine information needs for agents.
• Partner with product managers and engineers to define evaluation criteria for agent performance.
• Work with use-case teams to translate feedback into improvements.
• Improve retrieval mechanisms, data structuring, and onboarding of contextual sources.
• Design evaluation frameworks for rapid, large-scale experimentation and continuous innovation.
• Translate functional requirements into actionable technical improvements.

Skills

Analytical mindset

GenAI systems

LLM-based applications

Retrieval-augmented systems

Prompts & grounding

Output evaluation

Tools

API integrations

Batch synchronization

Data structuring

Job description

• Analyse chatbot outputs and identify the improvements with the greatest impact.
• Diagnose context gaps and determine what additional information agents require.
• Partner with product managers and engineers to define evaluation criteria for agent performance.
• Work directly with use-case teams and stakeholders to understand real-world scenarios and translate feedback into improvements.
• Improve retrieval mechanisms, data structuring, and onboarding of contextual sources.
• Design evaluation frameworks that enable rapid, large-scale experimentation and continuous innovation.
• Translate functional requirements into actionable technical improvements.

Your Talent:

• You bring a strong analytical mindset and a passion for understanding why AI systems behave the way they do;
• Strong understanding of GenAI systems.
• Experience with LLM-based applications.
• Experience with retrieval-augmented systems.
• Strong understanding of prompts, context windows, retrieval, and grounding.
• Ability to evaluate output quality and explain why results are effective or ineffective.

Technical Integration Skills:

• Familiarity with API integrations, batch synchronisation, and structured and unstructured data.
• Comfortable collaborating closely with engineers.
• Ability to troubleshoot data flow and context ingestion issues.