Staff ML Engineer: LLM Inference & Optimization

Nebius Group; Netherlands; Remote; EUR 140,000 - 190,000; Full time

Benefits offered by this job

Competitive compensation

Career growth

Flexibility and ownership

Collaborative culture

Job summary

Nebius Group is hiring for a role focused on production-grade inference optimization across LLM and VLM endpoints. You will drive latency, throughput, and cost improvements, working across model internals, inference engines, and deployment platforms.

You will build frameworks for evaluation and rollout, guide engineers and researchers, and deploy engines like vLLM, TensorRT-LLM, and Triton. Strong Python/PyTorch skills and experience with modern stacks are essential.

Qualifications

• Proven Python and PyTorch engineering skills.
• Experience deploying or optimizing high-throughput transformer inference systems (LLMs/VLMs).
• Knowledge of modern inference stacks (vLLM, TensorRT-LLM, Triton, KServe, etc.).
• understanding of transformer bottlenecks and memory/bandwidth tradeoffs.
• Ability to reason about latency, throughput, quality, and cost.
• Strong communication and cross-team collaboration.

Responsibilities

• Own an optimization pillar such as quantization or long-context serving.
• Build evaluation, benchmarking, and rollout frameworks across model families.
• Provide technical direction to MLEs, Applied Scientists, and engineers.
• Optimize LLM and VLM endpoints for latency, throughput, memory, and cost.
• Deploy and benchmark inference engines (vLLM, TensorRT-LLM, Triton, etc.).
• Develop model-compression workflows including quantization and distillation.
• Implement speculative decoding, KV-cache optimization, and continuous batching.
• Create reproducible benchmark harnesses for tokens/s, latency, and memory.
• Collaborate with kernel and platform engineers to diagnose bottlenecks.
• Produce design docs, performance reports, and customer-facing explanations.

Skills

Python

PyTorch

LLM inference

VLM inference

quantization

latency optimization

benchmarking

team collaboration

Tools

vLLM

TensorRT-LLM

Triton Inference Server

NVIDIA Dynamo

KServe

Job description

Nebius Group is hiring for a role focused on production-grade inference optimization across LLM and VLM endpoints. You will drive latency, throughput, and cost improvements, working across model internals, inference engines, and deployment platforms.

You will build frameworks for evaluation and rollout, guide engineers and researchers, and deploy engines like vLLM, TensorRT-LLM, and Triton. Strong Python/PyTorch skills and experience with modern stacks are essential.