Senior LLM/VLM Inference Scientist - Production-Ready

Fdetimes; Palo Alto (CA); Hybrid; USD 195,000 - 262,000; Full time

Benefits offered by this job

Health insurance

401(k) plan

Parental leave

Remote work stipend

Disability & life insurance

Job summary

Nebius is seeking a world-class research scientist to advance frontier LLM and VLM inference. You will own focused research and optimization projects, write strong code, and produce prototypes, publications, and artifacts that engineers can build on.

Your work spans quantization, distillation, speculative decoding, KV-cache optimization, and model/runtime co-optimization, evaluating trade-offs in quality, latency, throughput, memory, and cost per token.

Qualifications

• PhD in computer science, machine learning, ML systems, or closely related field.
• Strong publication record or equivalent ML research artifacts in efficient inference, model compression, or related areas.
• Strong Python and PyTorch implementation skills for experiments and prototypes.
• Deep knowledge of LLMs, VLMs, transformer inference, decoding algorithms, and production-serving trade-offs.
• Excellent experimental design with rigorous evaluation, baselines, metrics, and failure analysis.
• Excellent written and verbal communication.

Responsibilities

• Lead research projects in efficient LLM/VLM inference from hypotheses to prototyping and production handoff.
• Develop and evaluate methods spanning quantization, QAT, distillation, speculative decoding, KV-cache techniques, and model/runtime co-optimization.
• Build prototypes using PyTorch, Triton, CUDA-adjacent tooling; collaborate with MLEs and platform engineers to productionize.
• Research LLM request routing and scheduling strategies, including cache-aware load balancing and PDD; assess latency, throughput, and cost.
• Investigate multi-node inference, WideEP, expert placement, and load balance trade-offs to guide architecture decisions.
• Produce rigorous evaluations on quality, latency, throughput, stability, memory, tail latency, and cost per token; share results via blogs and papers.
• Collaborate with cross-functional teams to align research priorities with measurable production impact.
• Mentor engineers and scientists on experimental design and scientific rigor.

Skills

Python

PyTorch

LLM inference

VLM inference

Experiment design

Scientific communication

Education

PhD in CS/ML/ML systems

Tools

Triton

CUDA

PyTorch internals

Job description

Nebius is seeking a world-class research scientist to advance frontier LLM and VLM inference. You will own focused research and optimization projects, write strong code, and produce prototypes, publications, and artifacts that engineers can build on.

Your work spans quantization, distillation, speculative decoding, KV-cache optimization, and model/runtime co-optimization, evaluating trade-offs in quality, latency, throughput, memory, and cost per token.