Senior ML Solutions Architect, LLM Inference & Tuning

Nebius Group; United States; Remote; USD 208,000 - 261,000; Full time

Benefits offered by this job

Health Insurance

401(k) Plan

Parental Leave

Remote Work Reimbursement

Disability & Life Insurance

Job summary

Nebius is seeking a Principal ML Solutions Architect for Token Factory, our serverless platform for running open‑source LLMs in production. You will own complex engagements, design efficient inference workflows, and guide platform direction across teams from backend to research.

You will mentor engineers, shape roadmap priorities, and represent Nebius at events. Remote work from the United States is welcome, with a focus on delivering scalable, cost‑effective AI solutions.

Qualifications

• 8+ years of experience in ML/AI systems, with at least 4 years focused on LLMs and generative AI.
• Demonstrated technical leadership: owning ambiguous, high‑impact problems end to end and influencing decisions across teams and customers.
• Expert knowledge of the LLM ecosystem: model architectures, fine‑tuning approaches, and inference internals.
• Deep, hands‑on command of inference optimization: quantization, KV‑cache management, batching, routing, etc.
• Hands‑on experience with running LLMs in production at scale, deploying, operating, and debugging inference workloads.
• LLM fine‑tuning, including SFT/LoRA and data preparation/curation; RL‑based fine‑tuning experience.
• LLM evaluation: building benchmarks and eval pipelines, including LLM‑as‑a‑judge setups.
• Proficiency with inference frameworks and libraries (vLLM, SGLang, TensorRT‑LLM).

Responsibilities

• Own the most complex, high‑stakes customer engagements from architecture through production across multiple modalities.
• Optimize LLM inference at framework and hardware level; codify best practices into playbooks.
• Lead supervised and reinforcement fine‑tuning to maximize model quality.
• Design and implement production‑ready LLM solutions using Token Factory’s inference services.
• Provide deep technical expertise in prompt engineering, RAG architectures, model selection, and cost/performance trade‑offs at scale.
• Partner with product, engineering and research to surface customer needs and influence the roadmap.
• Guide customers from PoC to production with focus on performance, reliability, and cost efficiency.
• Mentor Senior and mid‑level Solutions Architects; raise the technical bar of the team.
• Represent Token Factory externally through talks, blog posts, and conferences.

Skills

ML/AI systems

Technical leadership

LLM ecosystem

Inference optimization

Python

Prompt engineering

RAG architectures

Communication

Tools

vLLM

TensorRT-LLM

SGLang

Transformers

OpenAI SDK

Anthropic SDK

Kubernetes

Docker

Git

Job description

Nebius is seeking a Principal ML Solutions Architect for Token Factory, our serverless platform for running open‑source LLMs in production. You will own complex engagements, design efficient inference workflows, and guide platform direction across teams from backend to research.

You will mentor engineers, shape roadmap priorities, and represent Nebius at events. Remote work from the United States is welcome, with a focus on delivering scalable, cost‑effective AI solutions.