Senior ML Engineer: Inference & Latency Optimization

Nebius Group; Greater London; On-site; GBP 90,000 - 140,000; Full time

Benefits offered by this job

Competitive compensation

Career growth and learning

Flexibility and ownership

Collaborative culture

Impactful AI projects

International environment

Job summary

Nebius is seeking a Senior Machine Learning Engineer to own model and endpoint optimization from artifacts to production deployment on our Applied AI team. You will improve latency, throughput, memory efficiency, GPU utilization, and cost per token while maintaining model quality.

This hands-on role involves diagnosing complex serving problems, evaluating configurations, and delivering measurable production improvements in collaboration with kernel and platform engineers.

Qualifications

• Strong Python and PyTorch engineering skills with production exposure.
• Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformer inference systems.
• Practical knowledge of modern inference stacks including vLLM, SGLang, TensorRT-LLM, Triton, or similar.
• Understanding of transformer bottlenecks: KV cache, attention, memory bandwidth, batching, and long-context serving.
• Ability to reason quantitatively about latency, throughput, quality, utilization, and cost tradeoffs.
• Strong communication and collaboration across diverse teams.

Responsibilities

• Own optimization work for specific model families, customer endpoints, or serving backends.
• Run engine comparisons and recommend practical serving configurations for workloads.
• Debug model quality or performance regressions during production rollouts.
• Optimize LLM and VLM endpoints for latency, throughput, memory efficiency, GPU utilization, quality, and cost per token.
• Deploy, configure, benchmark, and extend inference engines such as vLLM, SGLang, TensorRT-LLM, Triton, Dynamo or similar systems.
• Build production-ready model-compression workflows including quantization and distillation.
• Implement or integrate speculative decoding, KV-cache optimization, prefix caching, and continuous batching.
• Create reproducible benchmark harnesses for latency, tokens per second per GPU, and cost per token.
• Collaborate with kernel and platform engineers to diagnose bottlenecks.

Skills

Python

PyTorch

LLM inference

Performance optimization

Systems thinking

Communication

Tools

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

NVIDIA Dynamo

Ray Serve

KServe

Job description

Nebius is seeking a Senior Machine Learning Engineer to own model and endpoint optimization from artifacts to production deployment on our Applied AI team. You will improve latency, throughput, memory efficiency, GPU utilization, and cost per token while maintaining model quality.

This hands-on role involves diagnosing complex serving problems, evaluating configurations, and delivering measurable production improvements in collaboration with kernel and platform engineers.