Full Stack AI Engineer

Engaige; Utrecht; Hybrid; EUR 90,000 - 130,000; Full time

Benefits offered by this job

Stock options

Growth opportunities toward CTO

Job summary

Engaige Technologies B.V. is building an AI customer-service platform for e-commerce. As a Senior AI Engineer, you’ll own evaluation, retrieval, and policy-driven workflows ensuring safe, scalable production models across multiple providers.

You’ll work closely with the Head of Product and CEO to set technical standards and drive improvements across the team. Our stack includes Python, Weaviate, Langfuse, OpenTelemetry, and Google Cloud Platform.

Qualifications

• Experience shipping ML models to production.
• Strong understanding of offline/online evaluation and A/B testing.
• Proficiency in Python and backend production code.

Responsibilities

• Own evaluation methodology for agents and benchmarks with real production conversations.
• Design offline evaluations predicting online behavior and align benchmarks with production results.
• Build live guardrails, monitoring, and staged rollout across model providers.
• Improve retrieval and classification including RAG and embedding-based categorization.
• Develop policy graphs enabling step-by-step agent behavior.
• Extend AI manager to identify knowledge gaps and drive weekly improvements.
• Collaborate with the Head of Product and CEO to set engineering standards.

Skills

Python

Backend production code

Machine learning in production

Evaluation & AB testing

RAG / vector search

LLMs in production

Weaviate

Langfuse

Google Cloud Platform

Tools

Weaviate

Langfuse

OpenTelemetry

Google Cloud Platform

Job description

Engaige Technologies B.V. is a Dutch company building an AI customer-service platform for e-commerce. Millions of customer cases flow through our infrastructure every year, across more than 30 countries. Our agents answer customer questions and take actions on the merchant's behalf, such as looking up orders, creating returns, paybacks, changing subscriptions, etc. through 50+ integrations such as Shopify, Zendesk, Gorgias, Intercom, Klaviyo, PostNL and Sendcloud.

About the team

The engineering team consists of 4 engineers. You will work closely with the Head of Product and the CEO, and engineers own their work from initial thought through production.

About the role

As a Senior AI Engineer, you will be responsible for the quality and reliability of the platform and agents that handle these customer cases. This includes how we evaluate them, how we retrieve and classify information, how agents follow merchant policies, and how changes reach production safely.

A wrong promise or a wrong action toward a customer is a refund and a merchant at risk, so measurement, precision and safe rollout are part of the job. The core of this role is classical production ML depth: you have trained, evaluated and shipped models before LLMs became widely available, and you know how models fail once they meet real data.

Responsibilities

• Own the evaluation methodology for our agents, including our benchmark that replays real production conversations with simulated tools, and LLM judges that check for unsupported promises, customer language and correct outcomes
• Design offline evaluations that predict online behavior, and close the gap between benchmark scores and production results
• Build live guardrails, monitoring and staged rollout for model, prompt and policy changes across multiple model providers
• Improve retrieval and classification, including RAG with vector search and reranking, and embedding-based contact-reason classification
• Develop the policy graphs that let agents follow merchant policies step by step instead of relying on a single large prompt
• Extend our AI manager, which investigates conversations, identifies knowledge and policy gaps, and runs weekly improvement rounds
• Partner with the Head of Product and the CEO to set technical standard for engineering and evaluation across the team

You may be a good fit if you:

• Have trained and shipped machine learning models to production before 2022
• Have a working understanding of offline and online evaluation, A/B testing, data leakage, training/serving skew, and working with noisy or delayed labels
• Have hands-on experience with ranking, recommendation or classification systems
• Have built monitoring for model quality and drift, and rolled out model changes safely
• Are proficient in Python and comfortable owning backend production code
• Have high agency and low ego, prefer shipping over planning, and are comfortable working on problems that are not yet fully defined

Strong candidates may also have:

• Experience with LLMs in production, such as tool calling, structured output, RAG or LLM-as-judge evaluation
• Experience with search, retrieval or agent systems
• Experience with Weaviate, Langfuse, OpenTelemetry or Google Cloud Platform
• Experience with E-commerce or building SaaS tools.

Our stack

Claude, OpenAI and open-weight models (Kimi, GLM, DeepSeek, Qwen), Weaviate, Python (FastAPI), Native Postgres, Cloud Tasks, Cloud Run and Terraform

Location

This role is ideally based full-time in our office in Utrecht. Hybrid from Utrecht or fully remote is possible. Valencia is also an option. We hire within Europe only.

Compensation

We offer a very generous salary and stock options. For the right candidate, this role has a possible growth path toward CTO.