AI Inference Engineer

Next Ventures; Amsterdam; Hybrid; EUR 120,000 - 180,000; Full time

Benefits offered by this job

Hybrid work model

Job summary

Next Ventures is seeking a Senior AI Inference Engineer for government clients in the Netherlands, with hybrid work in Amstelveen or Eindhoven. You will own AI tooling and infrastructure lifecycles, deciding model-to-hardware deployment strategies, capacity planning, and performance optimization within an autonomous Scrum team.

The role requires production GPU LLM serving experience, strong knowledge of KV caching, batching, quantisation, and ADR-driven architecture decisions.

Qualifications

• Extensive experience building enterprise ML platforms.
• Proven track record in production GPU LLM serving.
• Strong knowledge of caching, batching and quantisation.

Responsibilities

• Benchmark an 8x H200 node: tensor parallelism versus replicas; measure throughput and latency.
• Translate results into customer-facing capacity models: tokens/min and requests/min per model/GPU.
• Build catalogue benchmarks covering performance, quality evaluation and hardware requirements.
• Design input/output content filtering using self-hosted open-weight models.
• Assess model licences, NVFP4/FP8/INT8 quantisation, memory fit and serving configurations.

Skills

Python

LLM serving

Kubernetes

GPU benchmarking

Quantisation

KV cache

ADRs

Tensor parallelism

Tools

vLLM

NVLink

Kubernetes

Job description

SENIOR AI INFERENCE ENGINEER OPPORTUNITY - 12 MONTH INITIAL ENGAGEMENT - GOVERNMENT CUSTOMERS

AMSTELVEEN OR EINDHOVEN BASED

HYBRID WORKING

Currently supporting a number of Government customers across the Nethelands, now seeking to add a AI Inference Engineer for their team within Benelux & Nordics, extending an enterprise AI platform for mission‑critical workloads. Working with the PM, PO and AI engineers in an autonomous Scrum team, you\'ll own AI tooling and infrastructure lifecycles and decide which models run on which hardware, at what capacity and quality, balancing delivery and operations.

Key responsibilities:

• Benchmark an 8x H200 node: tensor parallelism versus replicas with llm-d cache‑aware routing; measure throughput, time to first token and concurrency per GPU.
• Translate results into customer‑facing capacity models: tokens/minute and requests/minute per model/GPU.
• Build catalogue benchmarks covering performance, fixed‑set quality evaluation and hardware requirements.
• Design input/output content filtering using self‑hosted open‑weight models.
• Assess model licences, NVFP4/FP8/INT8 quantisation, memory fit and serving configurations.

You\'ll bring enterprise ML platform experience, production GPU LLM serving with vLLM or similar, and knowledge of KV cache, batching, prefix caching, quantisation and NVLink versus Ethernet. Python, practical Kubernetes, reproducible performance testing and written architecture decisions (ADRs) are key.

Hybrid: 1–2 days/week in Amstelveen/Eindhoven; flexibility required.