Senior Machine Learning Engineer, LLM Inference Optimization

Jobgether SRL; Netherlands; On-site; EUR 120,000 - 180,000; Full time

Benefits offered by this job

Competitive compensation

Career growth

Ownership over technical work

International environment

Job summary

Jobgether SRL is seeking a Senior Machine Learning Engineer in the Netherlands to drive optimization of large language and vision-language model inference from artifacts to production. You will work across model internals, serving architectures, and benchmarking to improve latency and cost.

This hands-on role requires Python and PyTorch, experience with modern inference stacks, and the ability to design reproducible benchmarks and safe rollouts for high-throughput AI workloads.

Qualifications

• Strong Python and PyTorch software skills.
• Experience deploying or optimizing LLM/VLM inference systems.
• Experience with modern inference stacks.
• Understanding transformer bottlenecks including KV cache, attention, memory bandwidth.
• Ability to reason quantitatively about latency, throughput, and cost trade-offs.
• Experience diagnosing complex performance problems and translating findings into production improvements.
• Strong communication and collaboration skills.
• Experience with quantization/training optimization techniques is a plus.

Responsibilities

• Own optimization initiatives for specific model families and inference backends.
• Evaluate inference engines and recommend serving configurations.
• Diagnose and resolve production regressions.
• Optimize LLM/VLM endpoints for latency, throughput, memory use, GPU utilization, and cost per token.
• Deploy, benchmark, and extend inference engines (e.g., vLLM, Triton, TensorRT-LLM).
• Build model-compression workflows (quantization, distillation, low-bit serving).
• Implement advanced inference techniques (speculative decoding, KV-cache optimization).
• Develop reproducible benchmarkHarnesses covering latency and tokens/second per GPU.
• Collaborate with kernel/platform engineers to identify bottlenecks across stack.
• Investigate performance trade-offs and guide optimization with benchmarks.
• Produce design docs, performance reports, and rollout plans.

Skills

Python

PyTorch

LLM inference

VLM inference

Performance tuning

Benchmarking

System design

GPU/CUDA

Open-source

Collaboration

Tools

vLLM

TensorRT-LLM

Triton Inference Server

NVIDIA Dynamo

KServe

Ray Serve

SGLang

Job description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Machine Learning Engineer, LLM Inference Optimization based in Netherlands.

As a Senior Machine Learning Engineer, you will drive the optimization of large language and vision-language model inference from model artifacts through production deployment. You will work across model internals, inference engines, serving architectures, and benchmarking to improve latency, throughput, memory efficiency, GPU utilization, reliability, and cost per token. This is a hands‑on role focused on solving complex performance challenges and delivering measurable improvements to production systems. You will collaborate closely with kernel, platform, infrastructure, research, product, and customer-facing teams. Your work will involve evaluating serving configurations, diagnosing performance and quality regressions, and implementing advanced inference optimization techniques. You will also establish reproducible benchmarks and safe rollout practices for high-throughput AI workloads.

Accountabilities

• Own optimization initiatives for specific model families, customer endpoints, and inference serving backends.
• Evaluate inference engines and recommend practical serving configurations based on workload requirements.
• Diagnose and resolve model quality, performance, and reliability regressions during production rollouts.
• Optimize LLM and VLM endpoints for latency, throughput, memory efficiency, GPU utilization, model quality, and cost per token.
• Deploy, configure, benchmark, and extend modern inference engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, or equivalent technologies.
• Build and productionize model-compression workflows, including quantization, quantization‑aware training, distillation, low‑bit serving, and accuracy recovery.
• Implement or integrate advanced inference techniques such as speculative decoding, draft models, KV-cache optimization, prefix caching, chunked prefill, continuous batching, and disaggregated prefill/decode serving.
• Develop reproducible benchmark harnesses covering TTFT, TPOT, tokens per second per GPU, p95/p99 latency, GPU memory usage, reliability, and cost per token.
• Partner with GPU kernel and platform engineers to identify bottlenecks across model code, kernels, runtimes, schedulers, gateways, and cluster infrastructure.
• Investigate performance trade‑offs quantitatively and use benchmark results to guide optimization decisions.
• Produce clear design documentation, performance reports, rollout plans, and technical explanations for internal and customer-facing stakeholders.
• Contribute to safe, measurable, and reliable production rollouts of inference improvements.

Requirements

• Strong software engineering skills in Python and PyTorch.
• Hands‑on experience deploying, operating, or optimizing LLM, VLM, or high‑throughput transformer inference systems.
• Practical experience with at least one modern inference stack, such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or an equivalent internal system.
• Strong understanding of transformer inference bottlenecks, including KV cache, attention mechanisms, memory bandwidth, batching, parallelism, and long-context serving.
• Ability to reason quantitatively about latency, throughput, model quality, resource utilization, and cost trade-offs.
• Experience diagnosing complex performance problems and translating findings into production improvements.
• Strong communication skills and the ability to collaborate effectively with research, kernel, infrastructure, product, and customer teams.
• Experience with quantization‑aware training, post-training quantization, FP8, INT8, INT4, NVFP4, MXFP4, AWQ, GPTQ, SmoothQuant, or related optimization techniques is a plus.
• Familiarity with distillation, speculative decoding, EAGLE, Medusa, multi-token prediction, or other inference acceleration approaches is advantageous.
• Experience supporting agentic workloads involving tool calling, structured outputs, streaming APIs, high concurrency, or multi-step orchestration is a plus.
• Familiarity with CUDA or Triton is beneficial, even if the role is not primarily focused on kernel engineering.
• Contributions to open-source projects such as vLLM, SGLang, TensorRT-LLM, FlashInfer, LMCache, PyTorch, Triton, Ray, KServe, or related technologies are advantageous.
• Ability to work independently, take ownership, and operate effectively in a fast-moving technical environment.

Benefits

• Competitive compensation.
• Career growth and continuous learning opportunities.
• Flexibility and significant ownership over technical work.
• Collaborative and innovative working environment.
• Opportunity to work on impactful AI infrastructure and inference optimization projects.
• Exposure to advanced LLM and VLM serving technologies and large-scale AI workloads.
• International environment with experienced engineering and AI teams.