Senior ML Engineer: LLM Inference & Optimization

Nebius Group; Brussel; Presencial; EUR 120.000 - 180.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Competitive compensation

Career growth opportunities

Flexibility and ownership

Collaborative culture

International environment and talented

Impactful AI projects

Descripción de la vacante

Nebius is seeking a Senior Machine Learning Engineer on the Applied AI team to optimize model endpoints from artifacts to production, spanning quantization, caching, and inference engines while preserving quality and reliability.

You will collaborate with kernel and platform engineers to diagnose performance regressions, evaluate distributed inference, and deploy scalable solutions across multi‑node GPU deployments, with a focus on latency, throughput, and cost efficiency.

Formación

• Strong engineering skills in Python and PyTorch.
• Hands‑on experience deploying or optimizing LLM, VLM, or high‑throughput transformer inference.
• Practical knowledge of inference stacks such as vLLM, TensorRT‑LLM, Triton, or KServe.
• Understanding of transformer inference bottlenecks involving KV cache, attention, memory bandwidth, and batching.
• Ability to quantify trade‑offs among latency, throughput, quality, utilization, and cost.
• Experience designing distributed inference system architecture with request routing or multi‑node inference.
• Strong communication and collaboration skills across teams.

Responsabilidades

• Own optimization projects for specific model families, customer endpoints, or serving backends.
• Compare inference engines and recommend serving configurations for varied workloads.
• Diagnose model-quality and performance regressions during production rollouts.
• Improve LLM/VLM endpoint latency, throughput, memory efficiency, GPU utilization, and cost per token.
• Deploy, configure, benchmark, and extend engines like vLLM, SGLang, TensorRT-LLM, Triton, NVIDIA Dynamo.
• Develop production model‑compression workflows including quantization and distillation.
• Implement speculative decoding, KV-cache optimization, and prefill/decode strategies.
• Design and improve request routers and scheduling policies balancing utilization and latency.
• Scale dense and mixture‑of‑experts inference across multiple GPU nodes.
• Build reproducible benchmarks measuring TTFT, tokens per second per GPU, and latency.
• Collaborate with kernel and platform teams to trace bottlenecks and improve throughput.
• Produce design documents, performance reports, and rollout plans.

Conocimientos

Python

PyTorch

LLM inference

VLM inference

Distributed systems

Performance optimization

Herramientas

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

NVIDIA Dynamo

Ray Serve

KServe

Descripción del empleo

Nebius is seeking a Senior Machine Learning Engineer on the Applied AI team to optimize model endpoints from artifacts to production, spanning quantization, caching, and inference engines while preserving quality and reliability.

You will collaborate with kernel and platform engineers to diagnose performance regressions, evaluate distributed inference, and deploy scalable solutions across multi‑node GPU deployments, with a focus on latency, throughput, and cost efficiency.