Senior LLM Inference & Optimization Engineer

FDE Times; United Arab Emirates; On-site; AED 717,000 - 963,000; Full time

Benefits offered by this job

Health insurance

401(k) plan

Parental leave

Remote work reimbursement

Disability & life insurance

Job summary

Nebius is seeking a Senior Machine Learning Engineer for the Applied AI team to optimize model artifacts and production deployment, improving latency, throughput, memory use, GPU utilization, and cost per token while maintaining model quality.

You will collaborate with kernel and platform engineers to diagnose serving problems, compare configurations, and validate improvements through reproducible benchmarks and safe, real-world rollouts.

Qualifications

• Must have strong Python and PyTorch engineering skills.
• Hands-on experience deploying or optimizing LLM/VLM or high-throughput transformer inference.
• Knowledge of modern inference stacks (vLLM, Triton, TensorRT-LLM, etc.).
• Ability to quantify trade-offs among latency, throughput, quality and cost.
• Experience designing or optimizing distributed inference system architectures.
• Excellent communication and collaboration across teams.

Responsibilities

• Own optimization projects for model families and serving backends.
• Compare inference engines and recommend configurations per workload.
• Diagnose model-quality and performance regressions in production.
• Improve endpoint latency, throughput, memory use, and cost per token.
• Deploy, configure, benchmark, and extend inference engines (e.g., Triton, vLLM, SGLang).
• Develop quantization and compression workflows (quantization-aware training, distillation).
• Collaborate with kernel and platform engineers to trace bottlenecks across stack.

Skills

Python

PyTorch

Distributed inference

Performance optimization

Latency analysis

Communication skills

Tools

vLLM

SGLang

TensorRT-LLM

Triton Inference Server

NVIDIA Dynamo

Ray Serve

KServe

Job description

Nebius is seeking a Senior Machine Learning Engineer for the Applied AI team to optimize model artifacts and production deployment, improving latency, throughput, memory use, GPU utilization, and cost per token while maintaining model quality.

You will collaborate with kernel and platform engineers to diagnose serving problems, compare configurations, and validate improvements through reproducible benchmarks and safe, real-world rollouts.