Production AI Engineer: Run & Improve LLM Services

NTek Software Solutions; Brussel Hoofdstad; Hybrid; EUR 65,000 - 95,000; Full time

Job summary

CLIENT is building AI services for Brussels public transport, delivering AI agents and retrieval assistants across the organization. The Medior AI Engineer will own the Run phase, keeping services reliable, measured, secure and cost-efficient in production.

You will collaborate with the AI Platform Architect, Senior AI Engineers, business owners, and IT operations teams. The AI Platform runs hybrid using Azure with OpenAI models, Azure AI Search, and self-hosted components on AKS, with LangChain

Qualifications

• Experience operating AI services in production and monitoring performance.
• Familiarity with cost optimization and production reliability practices.

Responsibilities

• Operate AI services in production: monitor availability, latency, errors; handle incidents and service requests.
• Maintain answer quality with evaluation sets and user feedback; detect regressions.
• Manage ingestion pipelines: incremental refresh, failed documents, index health, new content onboarding.
• Oversee agent integrations: monitor calls, handle timeouts, review logs for agents acting on users’ behalf.
• Manage model/version changes, prompts, dependencies, and security updates with safe validation.
• Track run costs per service and propose optimizations.
• Maintain Terraform, Helm and Azure DevOps artefacts; keep DEV/TST/ACC/PROD environments aligned.
• Write runbooks and keep documentation; report health to AI CoE.

Skills

AI engineering

Production monitoring

Incident handling

Python

LangChain

Azure

OpenAI

Terraform

Azure DevOps

Tools

Azure

AKS

MLflow

Databricks

LangChain

OpenTelemetry

Terraform

Azure DevOps

Job description

CLIENT is building AI services for Brussels public transport, delivering AI agents and retrieval assistants across the organization. The Medior AI Engineer will own the Run phase, keeping services reliable, measured, secure and cost-efficient in production.

You will collaborate with the AI Platform Architect, Senior AI Engineers, business owners, and IT operations teams. The AI Platform runs hybrid using Azure with OpenAI models, Azure AI Search, and self-hosted components on AKS, with LangChain