Medior AI Engineer - Production AI & MLOps (Azure)

Keystone Solutions; Brussel Hoofdstad; On-site; EUR 70,000 - 95,000; Full time

Job summary

Keystone Solutions is seeking a Medior AI Engineer to join our consultancy mission at a client site in Belgium. You will join the AI Center of Excellence, focusing on Run phase production operations with a hybrid Azure/open-source stack.

You will monitor performance, reliability, and security, manage ingestion and integrations, and drive cost efficiency and safe change control across DEV to PROD environments. Excellent Dutch/French and English proficiency required.

Qualifications

• Proven experience deploying AI services in production.
• Strong knowledge of Azure cloud and open-source stack.
• Experience with cost monitoring and optimization.

Responsibilities

• Operate AI services in production: monitor availability, latency, errors; handle incidents and requests; analyze root causes and fix or escalate.
• Maintain evaluation sets with business content owners; track quality, non-answer rate, and user feedback; detect regressions.
• Manage ingestion pipelines: incremental refresh, failed documents, index health, onboarding of new content sources.
• Monitor tool and integration calls to business systems; handle failures and timeouts; review logs and exceptions.
• Manage model/version upgrades, configuration changes, and security updates; validate against eval sets before release.
• Monitor token consumption and cost per service; alert on drift; propose optimizations.
• Maintain Terraform, Helm, and Azure DevOps artifacts across DEV/TST/ACC/PROD environments.
• Apply data protection rules: trace retention, erasure, access restrictions per service DPIA.
• Maintain runbooks and documentation; report service health to owners and AI CoE.
• Deliver small enhancements and fixes; larger features handled by Build team.

Skills

Azure DevOps

AKS

Azure OpenAI

LangChain

LLM

Python

RAG

Terraform

Vector databases

Job description

Keystone Solutions is seeking a Medior AI Engineer to join our consultancy mission at a client site in Belgium. You will join the AI Center of Excellence, focusing on Run phase production operations with a hybrid Azure/open-source stack.

You will monitor performance, reliability, and security, manage ingestion and integrations, and drive cost efficiency and safe change control across DEV to PROD environments. Excellent Dutch/French and English proficiency required.