Senior AI Inference Engineer

Empiric; Amstelveen; On-site; EUR 90,000 - 130,000; Full time

Job summary

Empiric seeks a Senior AI Inference Engineer to lead production LLM inference on GPU infrastructure and secure enterprise deployments. You will optimise GPU-based AI workloads, benchmark model performance, and help scale high-performance inference across multi-GPU setups.

You will design and implement efficient serving frameworks, quantisation strategies, and routing for enterprise environments. Strong collaboration with engineering teams is essential to deliver reliable, scalable AI solutions.

Qualifications

• Strong hands-on experience developing and operating enterprise AI/ML platforms.
• Production experience deploying and optimising LLM inference on GPUs, ideally with vLLM.
• Deep understanding of inference optimisation, KV cache, batching, quantisation and GPU memory management.
• Experience benchmarking LLM performance, including throughput, time-to-first-token and concurrency.
• Working knowledge of Kubernetes and containerised AI workloads.
• Experience documenting technical design decisions and performance findings.
• Comfortable presenting technical findings to engineering teams and stakeholders.

Responsibilities

• Design production LLM inference workloads and optimise for performance.
• Benchmark AI model performance on GPU infrastructure and multi-GPU configs.
• Improve throughput, latency, concurrency and GPU utilisation.
• Evaluate tensor partitioning, KV caching, batching, and intelligent routing.
• Develop benchmarking frameworks for model quality, capacity and hardware needs.
• Explore open-weight LLMs and quantisation techniques (FP8, INT8, NVFP4).
• Design input/output filtering and guardrails for secure deployments.
• Work with Kubernetes and Python to support scalable inference environments.

Skills

LLM inference

GPU optimization

Production platforms

Kubernetes

Python

Benchmarking

Performance tuning

Security for enterprise

Tools

vLLM

NVIDIA GPUs

KServe

llm-d

CUDA

Job description

Senior AI Inference Engineer | LLM Serving & GPU Infrastructure

Are you an experienced AI/ML Engineer specialising in LLM inference, GPU optimisation and production AI infrastructure?

We're supporting a major European technology organisation developing a Sovereign AI Platform, enabling enterprises to deploy and operate Large Language Models securely on dedicated GPU infrastructure without relying on public cloud hyperscalers.

This is a highly technical, hands-on opportunity focused on LLM serving, inference optimisation, GPU performance benchmarking and AI model deployment within secure, enterprise-scale environments.

You'll play a key role in determining how AI models are deployed, optimised and scaled across high-performance GPU infrastructure.

What You'll Be Doing:

• Designing and optimising production LLM inference workloads using vLLM or comparable serving frameworks.
• Benchmarking AI model performance across NVIDIA H200 GPU infrastructure, including multi-GPU configurations.
• Optimising inference throughput, latency, concurrency and GPU utilisation.
• Evaluating tensor parallelism, KV caching, batching, prefix caching and intelligent request routing.
• Developing performance benchmarking frameworks to assess model quality, capacity and hardware requirements.
• Evaluating open-weight LLMs, including quantisation techniques such as FP8, INT8 and NVFP4.
• Designing model input/output filtering and AI guardrails for secure enterprise deployments.
• Working with Kubernetes and Python to support scalable AI inference environments.

What We're Looking For:

• Strong hands-on experience developing and operating enterprise AI/ML platforms.
• Production experience deploying and optimising LLM inference on GPUs, ideally using vLLM.
• Deep understanding of inference optimisation, KV cache, batching, quantisation and GPU memory management.
• Experience benchmarking LLM performance, including throughput, time-to-first-token and concurrency.
• Working knowledge of Kubernetes and containerised AI workloads.
• Experience documenting technical design decisions and performance findings.
• Comfortable presenting technical findings to engineering teams and stakeholders.

Nice to Have:

• NVIDIA H100/H200 or comparable GPU infrastructure.
• KServe, llm-d or other distributed LLM serving frameworks.
• AMD ROCm and heterogeneous GPU environments.
• MLflow, model evaluation or benchmarking frameworks.
• AI safety, content filtering and guardrail models.
• Experience supporting sovereign, private cloud or regulated AI environments.

Why Consider This Opportunity?

You’ll be working at the intersection of AI Engineering, GPU Infrastructure and High-Performance Computing, helping build a production-grade AI platform designed for demanding enterprise workloads.

This is an opportunity to take real ownership of inference performance, model selection and GPU capacity planning, rather than simply integrating LLM APIs into applications.