RAG AI Engineer - Production GenAI Quality

Fitura; Utrecht; On-site; EUR 90,000 - 120,000; Full time

Job summary

Fitura in the Netherlands is seeking an AI Engineer - RAG Platform Optimization to own model quality across production RAG systems, focusing on retrieval, grounding, and prompt strategies.

You will run experiments in Databricks, analyze telemetry, and build CI/CD data pipelines on Azure to scale improvements for thousands of internal users.

The role combines IR, data science, and software engineering, with a strong emphasis on measurable results and production-grade implementation.

Qualifications

• Master's degree in a quantitative field
• 3+ years of professional experience in ML, AI, or data science
• Strong Python and Databricks experience
• Experience with production-grade RAG systems
• English proficiency in business context

Responsibilities

• Perform failure analysis on RAG outputs and run large-scale experiments.
• Build automated GenAI evaluation pipelines measuring precision/recall and hallucinations.
• Analyze telemetry data in Databricks to detect regressions.
• Translate results into scalable Python production code and CI/CD data pipelines on Azure.
• Collaborate with engineers and product managers to embed metrics into release pipelines.

Skills

Python

Databricks

LLM architecture

Azure cloud environments

GenAI evaluation design

CI/CD pipelines

Education

Master's degree (WO) or equivalent in Data Science, AI, CS, Math, Statistics

Tools

MLflow

TruLens

Ragas

LangSmith

Job description

Fitura in the Netherlands is seeking an AI Engineer - RAG Platform Optimization to own model quality across production RAG systems, focusing on retrieval, grounding, and prompt strategies.

You will run experiments in Databricks, analyze telemetry, and build CI/CD data pipelines on Azure to scale improvements for thousands of internal users.

The role combines IR, data science, and software engineering, with a strong emphasis on measurable results and production-grade implementation.