Dev Ops Engineer C

1UP Training (Netherlands); Utrecht; Hybrid; EUR 90,000 - 125,000; Full time

Job summary

1UP Training is seeking an AI Engineer to advance a Retrieval-Augmented Generation (RAG) platform. You will enhance data-driven experiments, improve retrieval and answer quality, and translate findings into production-grade improvements.

You’ll work at the intersection of information retrieval, data science and software engineering, collaborating with platform, data science and product teams in Utrecht. Fluency in English is required, and a master’s degree with 3+ years in ML or applied AI is

Qualifications

• Master’s degree or equivalent in a related field.
• 3+ years of experience in ML or applied AI.
• Track record improving production-grade RAG or retrieval AI.
• Experience designing evaluation methods and metrics.
• Ability to turn analysis into automated production solutions.
• Experience with large datasets and distributed systems.
• Excellent communication to technical and non-technical stakeholders.
• Strong written and spoken English.

Responsibilities

• Improve retrieval performance, answer quality and user experience through analysis and optimization.
• Investigate low-quality answers and identify grounding issues.
• Develop automated evaluation frameworks for retrieval and answer quality.
• Run large-scale experiments on chunking, embeddings and retrieval strategies.
• Analyze user interactions and Databricks data to find improvement opportunities.
• Build feedback loops to identify weak responses and drive improvements.
• Define and monitor quality metrics for measurable releases.
• Translate analysis into scalable engineering solutions.
• Collaborate with software engineers, data scientists and product managers.

Skills

RAG & LLMs

Python

Databricks

Evaluation metrics

Data pipelines

Azure & CI/CD

Embedding models

Prompt optimization

Experimentation

Education

Master's degree in Data Science / AI / CS

3+ years in ML or data science

Tools

Databricks

Job description

AI Engineer – RAG Platform

About the role
As an AI Engineer focused on a Retrieval-Augmented Generation (RAG) platform, you will improve the system’s quality, accuracy and effectiveness through data-driven experimentation and measurable improvements.
You will work at the intersection of information retrieval, data science and software engineering. Your focus is improving the performance of the RAG system itself by identifying where and why it underperforms, testing potential improvements and translating findings into production outcomes.
You will contribute throughout the improvement lifecycle, from failure analysis and hypothesis generation to evaluation, implementation and monitoring. You will use user feedback, retrieval metrics, evaluation datasets and production telemetry to improve answer quality and user trust.

What you’ll do

• Improve retrieval performance, answer quality and the overall user experience through analysis, experimentation and optimization.
• Investigate low-quality answers and identify underlying retrieval or grounding issues.
• Develop automated evaluation frameworks to measure retrieval quality, answer quality and user satisfaction, identify regressions and provide actionable insights.
• Run large-scale experiments to assess chunking, embedding and retrieval strategies.
• Analyze user interactions and system performance data in Databricks to identify opportunities for improvement.
• Build feedback loops that identify weak responses and feed insights into development.
• Define and monitor quality metrics so releases deliver measurable improvements.
• Translate analysis and experimentation into scalable, maintainable engineering solutions.
• Work closely with software engineers, platform engineers, data scientists and product managers.

What you’ll bring

• A master’s degree or equivalent academic level in Data Science, Artificial Intelligence, Computer Science, Mathematics, Statistics or a related field.
• At least three years of relevant experience in machine learning, applied AI or data science.
• Demonstrable experience improving production-grade RAG systems or advanced retrieval-based AI solutions.
• Experience designing evaluation methodologies and quality metrics.
• A strong track record of turning analysis into automated production solutions.
• Experience working with large datasets and complex distributed environments.
• The ability to communicate findings and recommendations to technical and non-technical stakeholders.
• Proficiency in written and spoken English.
• A curious, analytical approach and a focus on taking ownership of measurable outcomes.

Technical skills

• Retrieval-Augmented Generation (RAG) and large language models (LLMs)
• Python and Databricks
• Evaluation frameworks for generative AI systems
• Embedding models and prompt optimization
• Data pipelines and workflow automation
• CI/CD practices and Azure

Nice to have

• Experience fine-tuning language models.
• Experience with ranking and retrieval systems.
• Experience with AI observability and monitoring.
• Experience working with large-scale production AI platforms.
• Knowledge of synthetic evaluation datasets and benchmark creation.

Eisen

• Master's degree or equivalent academic level in Data Science, Artificial Intelligence, Computer Science, Mathematics, Statistics or a related field
• At least 3 years of relevant experience in machine learning, applied AI or data science
• Demonstrable experience improving production-grade RAG systems or advanced retrieval-based AI solutions
• Experience designing evaluation methodologies and quality metrics
• Strong track record of converting analyses into automated production solutions
• Experience working with large datasets and complex distributed environments
• Ability to communicate findings and recommendations to both technical and non-technical stakeholders
• Proficiency in English, both written and spoken

Gunningscriteria

• Experience with fine-tuning language models
• Experience with ranking and retrieval systems
• Experience with AI observability and monitoring
• Experience working with large-scale production AI platforms
• Knowledge of synthetic evaluation datasets and benchmark creation