Senior LLM Inference Optimization Engineer

Fdetimes; Amsterdam; Híbrido; EUR 120.000 - 180.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Competitive compensation

Career growth and learning

Flexibility and ownership

Collaborative and innovative culture

Impactful AI projects

International environment and talented

Descripción de la vacante

Nebius is seeking a Senior Machine Learning Engineer to own model and endpoint optimization from artifacts to production deployment on the Applied AI team. You will tackle model internals, quantization, speculative decoding, KV-cache optimization, and multi-node GPU strategies to reduce latency, boost throughput, and lower token成本 while preserving quality.

You will collaborate with kernel and platform engineers to diagnose performance regressions, evaluate engine configurations, and validate

Formación

• Strong engineering skills in Python and PyTorch.
• Hands-on experience deploying or optimizing LLM/VLM or high-throughput transformer inference.
• Knowledge of modern inference stacks (vLLM, Triton, TensorRT-LLM, SGLang, NVIDIA Dynamo, Ray Serve or KServe).
• Understanding transformer inference bottlenecks: KV cache, memory bandwidth, batching, and long-context serving.
• Ability to quantify latency/throughput trade-offs and cost.
• Experience designing distributed inference system architectures (routing, scheduling, PDD, multi-node inference, or expert parallelism).
• Strong communication and collaboration across research, kernel, infrastructure, product, and customers.

Responsabilidades

• Own optimization projects for specific model families, endpoints, or serving backends.
• Compare inference engines and configure best-serving setups per workload.
• Diagnose model-quality and performance regressions during production rollouts.
• Improve LLM/VLM endpoint latency, throughput, memory efficiency, GPU utilization, and cost per token.
• Deploy, benchmark, and extend engines (e.g., vLLM, Triton, TensorRT-LLM, Triton Server, CUDA) in production.
• Develop model-compression workflows including quantization and distillation.
• Implement speculative decoding, KV-cache optimization, and prefill/decode strategies.
• Design and tune LLM request routers and scheduling to balance latency and reliability.
• Scale dense and Mixture-of-Experts inference across multiple GPU nodes with proper parallelism.

Conocimientos

Python

PyTorch

LLM/Transformer inference

Distributed inference architecture

Inference stacks (vLLM/Triton/Ray)

Herramientas

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

NVIDIA Dynamo

Ray Serve

KServe

Descripción del empleo

Nebius is seeking a Senior Machine Learning Engineer to own model and endpoint optimization from artifacts to production deployment on the Applied AI team. You will tackle model internals, quantization, speculative decoding, KV-cache optimization, and multi-node GPU strategies to reduce latency, boost throughput, and lower token成本 while preserving quality.

You will collaborate with kernel and platform engineers to diagnose performance regressions, evaluate engine configurations, and validate