AI/LLM Performance Scientist

Infinity Quest Limited; Utrecht; On-site; EUR 70,000 - 100,000; Full time

Job summary

Infinity Quest Limited, Utrecht, is seeking an AI Evaluation Engineer to analyze chatbot outputs, diagnose context gaps, and collaborate with product and engineering teams to define rigorous evaluation criteria. You will improve retrieval and data structures, design scalable experiment frameworks, and translate requirements into concrete technical improvements.

You will work closely with cross-functional teams to understand real-world scenarios, evaluate model performance, and drive continuous

Qualifications

• Strong analytical mindset and curiosity about AI behavior.
• Solid understanding of GenAI systems.
• Experience with LLM-based applications.
• Experience with retrieval-augmented systems.
• Strong knowledge of prompts, context windows, and grounding.
• Ability to evaluate output quality and explain results.

Responsibilities

• Analyse chatbot outputs and identify the most impactful improvements.
• Diagnose context gaps and determine information needs for agents.
• Partner with product managers and engineers to define evaluation criteria for agent performance.
• Work with use-case teams to translate feedback into improvements.
• Improve retrieval mechanisms, data structuring, and onboarding of contextual sources.
• Design evaluation frameworks for rapid, large-scale experimentation and continuous innovation.
• Translate functional requirements into actionable technical improvements.

Skills

Analytical mindset

GenAI systems

LLM-based applications

Retrieval-augmented systems

Prompts & grounding

Output evaluation

Tools

API integrations

Batch synchronization

Data structuring

Job description

Infinity Quest Limited, Utrecht, is seeking an AI Evaluation Engineer to analyze chatbot outputs, diagnose context gaps, and collaborate with product and engineering teams to define rigorous evaluation criteria. You will improve retrieval and data structures, design scalable experiment frameworks, and translate requirements into concrete technical improvements.

You will work closely with cross-functional teams to understand real-world scenarios, evaluate model performance, and drive continuous