Senior ML Engineer — AI Inference & Performance

Nebius Group; London (KY); On-site; USD 160,000 - 260,000; Full time

Benefits offered by this job

Competitive compensation

Career growth

Flexibility

Collaborative culture

Impactful AI projects

International teams

Job summary

Nebius is building a fast, cost-efficient AI cloud platform. We seek a Senior Machine Learning Engineer to own model and endpoint optimization from artefacts to production deployment.

You will work across model internals, inference engines, and serving architectures, aiming to improve latency, throughput, memory efficiency, GPU utilization and cost per token while preserving model quality. This hands-on role involves benchmarking, reproducible results, and close collaboration with kernel and

Qualifications

• Strong Python and PyTorch engineering skills.
• Hands-on experience deploying or optimizing LLM/VLM or high-throughput transformer inference systems.
• Practical knowledge of inference stacks such as vLLM, SGLang, TensorRT-LLM, Triton, NVIDIA Dynamo, Ray Serve, or KServe.
• Understanding of transformer bottlenecks (KV cache, attention, memory bandwidth) and latency tradeoffs.
• Ability to reason quantitatively about latency, throughput, quality, and cost.

Responsibilities

• Own optimization work for specific model families, customer endpoints, or serving backends.
• Run engine comparisons and recommend practical serving configurations for workloads.
• Debug model quality or performance regressions during production rollouts.
• Optimize LLM and VLM endpoints for latency, throughput and cost per token.
• Deploy, configure, benchmark, and extend inference engines (e.g., vLLM, TensorRT-LLM, Triton).
• Build and productionize model compression workflows (quantization, distillation, low-bit serving).
• Implement speculative decoding, KV-cache optimization, and continuous batching.
• Create reproducible benchmarks for latency, tokens per second per GPU, and memory.
• Collaborate with kernel and platform engineers to diagnose bottlenecks.
• Write design docs, performance reports, and rollout plans.

Skills

Python

PyTorch

LLM inference

VLM inference

Transformer optimization

Tools

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

NVIDIA Dynamo

KServe

Ray Serve

Job description

Nebius is building a fast, cost-efficient AI cloud platform. We seek a Senior Machine Learning Engineer to own model and endpoint optimization from artefacts to production deployment.

You will work across model internals, inference engines, and serving architectures, aiming to improve latency, throughput, memory efficiency, GPU utilization and cost per token while preserving model quality. This hands-on role involves benchmarking, reproducible results, and close collaboration with kernel and