Senior AI Inference Scientist: LLM & VLM Optimization

Nebius Group; Palo Alto (CA); On-site; USD 195,000 - 262,000; Full time

Benefits offered by this job

Health insurance

401(k) plan with company match

Parental leave

Remote work reimbursement

Disability & life insurance

Job summary

Nebius is seeking a highly skilled AI research scientist to advance frontier LLM and VLM inference and drive production-ready innovations at scale. You will own focused research, write strong code, and produce prototypes, publications, and artifacts for engineers to build upon.

Responsibilities span quantization, distillation, speculative decoding, KV-cache optimization, and model/runtime co-optimization, working with MLEs and platform engineers to deliver impactful results.

Qualifications

• PhD in computer science or closely related field with strong ML systems focus.
• Strong Python and PyTorch implementation skills for experiments and prototypes.
• Deep knowledge of LLMs/VLMs, transformer inference, decoding, and model compression.
• Experience with experimental design, ablations, metrics, and rigorous analysis.
• Excellent written and verbal communication.

Responsibilities

• Lead research projects on efficient LLM/VLM inference from hypotheses to production handoff.
• Develop and evaluate methods: quantization, QAT, distillation, speculative decoding, KV-cache reuse.
• Build prototypes using PyTorch, Triton, CUDA-adjacent tooling; collaborate to production components.
• Research LLM routing, scheduling, cache-aware load balancing, and disaggregation strategies.
• Investigate multi-node inference for dense and MoE models; study expert placement and trade-offs.
• Provide rigorous evaluations on quality, latency, throughput, memory, and cost per token.

Skills

Python

PyTorch

Experimental design

ML systems

Transformer inference

Quantization

Distillation

Production serving

Communication

RLHF

Education

PhD in computer science / ML / systems

Strong publication record or artifacts in ML/efficient inference

Tools

PyTorch internals

TensorRT-LLM

CUDA

Triton

Job description

Nebius is seeking a highly skilled AI research scientist to advance frontier LLM and VLM inference and drive production-ready innovations at scale. You will own focused research, write strong code, and produce prototypes, publications, and artifacts for engineers to build upon.

Responsibilities span quantization, distillation, speculative decoding, KV-cache optimization, and model/runtime co-optimization, working with MLEs and platform engineers to deliver impactful results.