Staff ML Engineer: AI Inference & Optimization

Nebius Group; Netherlands; On-site; EUR 120,000 - 180,000; Full time

Benefits offered by this job

Competitive compensation

Career growth

Flexibility

Collaborative culture

Impactful AI projects

International environment

Job summary

Nebius is building production-grade inference for open-source, customer, and internal models. This role owns the path from model artifact to fast, reliable endpoints at the intersection of model internals, inference engines, serving architecture, benchmarking, and production deployment.

You will optimize latency, throughput, memory usage, and cost per token; deploy engines such as vLLM, TensorRT-LLM, and Triton; and build reusable benchmarks and compression workflows with engineers and

Qualifications

• Strong Python and PyTorch programming capabilities for large-scale models.
• Hands-on experience deploying or optimizing LLM/VLM and high-throughput transformer inference systems.
• Experience with at least one modern inference stack (vLLM, SGLang, TensorRT-LLM, Triton, NVIDIA Dynamo, Ray Serve, or KServe).
• Deep understanding of transformer bottlenecks (KV cache, attention, memory bandwidth, batching, long-context serving).
• Ability to reason quantitatively about latency, throughput, quality, utilization, and cost.
• Strong communication and collaboration skills across teams.

Responsibilities

• Own platform-level optimization pillars (quantization, caching, engine selection, or long-context serving).
• Create reusable evaluation, benchmarking, and rollout frameworks for multiple model families.
• Provide technical direction to ML engineers, scientists, GPU kernel, and infra teams.
• Optimize LLM/VLM endpoints for latency, throughput, memory efficiency, GPU utilization, and cost per token.
• Deploy and benchmark inference engines such as vLLM, TensorRT-LLM, Triton, and related systems.
• Build production-ready model-compression workflows (quantization, distillation, low-bit serving).
• Develop reproducible benchmarks for TTFT, TPOT, tokens/sec per GPU, p95/p99 latency, and cost per token.

Skills

Python

PyTorch

LLM/VLM deployment

Inference optimization

Quantization techniques

Performance engineering

Technical communication

Tools

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

NVIDIA Dynamo

Ray Serve

KServe

Job description

Nebius is building production-grade inference for open-source, customer, and internal models. This role owns the path from model artifact to fast, reliable endpoints at the intersection of model internals, inference engines, serving architecture, benchmarking, and production deployment.

You will optimize latency, throughput, memory usage, and cost per token; deploy engines such as vLLM, TensorRT-LLM, and Triton; and build reusable benchmarks and compression workflows with engineers and