Senior ML Engineer – LLM Inference Optimization (Remote)

FDE Times; Abu Dhabi; On-site; AED 717,000 - 963,000; Full time

Benefits offered by this job

Health insurance

401(k) plan

Parental leave

Remote work reimbursement

Disability & life insurance

Job summary

Nebius is seeking a Senior Machine Learning Engineer on the Applied AI team to own model and endpoint optimization from artifacts to production deployment. You will tackle model internals, quantization, speculative decoding, KV-cache optimization, and benchmarking to improve latency, throughput, memory efficiency, GPU utilization, and cost per token while preserving quality and reliability.

You will collaborate with kernel and platform engineers to diagnose bottlenecks, evaluate engine-level

Qualifications

• Proficient in Python and PyTorch.
• Experience deploying or optimizing LLM/VLM inference.
• Knowledge of modern inference stacks (vLLM, SGLang, TensorRT-LLM, Triton, NVIDIA Dynamo, Ray Serve or KServe).
• Understand transformer bottlenecks like KV cache, attention, memory bandwidth, batching, and long-context serving.
• Ability to balance latency, throughput, quality, utilization and cost.
• Experience with distributed inference architecture (request routing, distributed scheduling, PDD, multi-node inference, expert parallelism).
• Strong communication and collaboration across teams.

Responsibilities

• Own optimization projects for specific model families, endpoints, or serving backends.
• Compare inference engines and configure serving settings per workload.
• Diagnose model-quality and performance regressions during production rollouts.
• Improve LLM/VLM endpoint latency, throughput, memory use, and GPU utilization.
• Deploy and benchmark inference engines like vLLM, SGLang, TensorRT-LLM, Triton, NVIDIA Dynamo.
• Develop production model-compression workflows (quantization, QAT, distillation, low-bit serving, accuracy recovery).
• Implement speculative decoding, KV-cache optimization, prefix caching, prefill strategies, and disaggregated prefill/decode serving.
• Design and improve LLM request routers and scheduling to balance utilization while meeting latency targets.
• Scale dense and mixture-of-experts inference across multiple GPU nodes; tune parallelism and topology.
• Build reproducible benchmarks measuring TTFT, TPOT, tokens per second per GPU, p95/p99 latency, memory and cost.
• Collaborate with GPU kernel and platform engineers to trace bottlenecks across model code and cluster layers.
• Produce design documents, performance reports, rollout plans, and customer-facing explanations.

Skills

Python

PyTorch

LLM deployment

VLM inference

Distributed inference

Performance optimization

Tools

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

NVIDIA Dynamo

Ray Serve

KServe

Job description

Nebius is seeking a Senior Machine Learning Engineer on the Applied AI team to own model and endpoint optimization from artifacts to production deployment. You will tackle model internals, quantization, speculative decoding, KV-cache optimization, and benchmarking to improve latency, throughput, memory efficiency, GPU utilization, and cost per token while preserving quality and reliability.

You will collaborate with kernel and platform engineers to diagnose bottlenecks, evaluate engine-level