Principal ML Researcher

Toloka

España

Remote

EUR 120,000 - 180,000

Full time

14 days+

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Full remote or hybrid model
Competitive compensation + ESOP
PTO and benefits
IT setup and home office

Job summary

Toloka is seeking a Principal ML Researcher to define the technical vision for its core ML and post-training stack. You will bridge frontier AI research with platform engineering, mentor researchers, and shape scalable evaluation and deployment pipelines. The role supports a global, remote-first team and offers competitive compensation and ESOP.

Qualifications

  • 6+ years in ML engineering or applied research with leadership in LLM post-training/fine-tuning at scale.
  • Deep theoretical and practical mastery of LLM alignment: SFT, LoRA/PEFT, RLHF/RLAIF, reward modeling.
  • Mastery of LLM evaluation & calibration, building evaluation pipelines from scratch.

Responsibilities

  • Define multi-quarter strategy and architecture for Toloka’s post-training stack.
  • Architect greenfield post-training and RL pipelines (GRPO, PPO/DPO, reward modeling, RLAIF).
  • Design automated evaluation ecosystems with human-aligned LLM-as-a-judge frameworks.
  • Lead model distillation, quantization, context compression, and efficient inference at scale.
  • Shape agentic platform capabilities and tool-use workflows for multi-step reasoning.

Skills

LLM alignment
RLHF/RLAIF
Python
PyTorch
Distributed training

Tools

Deepspeed
Megatron
FSDP
vLLM
TensorRT-LLM

Job description

About Toloka

At Toloka AI we create data that powers leading GenAI models and innovations. We work with frontier labs, big tech, renowned AI startups, enterprises and non-profit research organizations worldwide. We use a combination of Experts + Crowd + Tech Platform to teach AI models to reason and evaluate their efficacy and safety. We have experts in more than 50 different domains—from doctors and lawyers to physicists and engineers—and boast one of the most diverse global crowds, representing over 100 countries and speaking 40+ languages. We are a well-funded startup with an enviable portfolio of clients including Anthropic, Amazon, Microsoft, Poolside, Recraft, and Shopify.

Recently, we secured strategic investment led by Bezos Expeditions and Nebius Group with participation from Mikhail Parakhin, CTO of Shopify and board advisor to leading GenAI companies, who now serves as our Chairman of the Board. Our remote-first team is globally distributed around the world: USA, UK, the Netherlands, Serbia, and more.

About the Team

We are the ML team inside Toloka — we build the machine-learning products that power the platform itself, so every project running on Toloka is faster, cheaper, and more reliable.

A few examples of what we own:

  • Enterprise post-training- we are helping real companies by providing them small LLMs that beat frontier models in quality at a fraction of the cost.
  • Off-the-shelf post-training - we are conducting research on datasets we deliver to clients, showing that training on these datasets will improve their performance.
  • Fine-tuning and RL — adapting frontier and open-source models to Toloka's tasks to hit the right quality at the right cost.
  • Evaluation, benchmarking, cost modeling, and model selection across providers.
  • LLM QA — the core technology behind Toloka's automated quality-check mechanism. Every annotation flowing through Self-Service is reviewed by an LLM agent we design, train, and operate.

We own the full chain. The same team designs the ML solution, ships it to production, keeps it running 24/7, analyzes the results coming back from real projects, and feeds that signal into the next iteration. No hand-off between research, engineering, and operations — it's all us.

About the Position

As a Principal ML Researcher, you will define the overarching technical vision, research strategy, and architecture for Toloka’s core ML and post-training stack. In this high-impact role, you will bridge frontier AI research and large-scale platform engineering.

You will lead technical strategy across greenfield post-training paradigms (such as GRPO, process/outcome reward modeling, and RLAIF), architect resilient automated evaluation ecosystems, and set the standard for how foundation models are adapted and served on our platform. As a technical authority, you will mentor senior ML engineers, collaborate directly with executive leadership, and represent Toloka in the broader AI research community.

What you’ll do
  • Drive Technical Strategy: Define the multi-quarter strategy and architecture for Toloka’s post-training stack, ensuring our fine-tuning, RL, and evaluation capabilities stay ahead of industry trends.
  • Architect Greenfield Post-Training & RL Pipelines: Spearhead the transition beyond basic SFT into advanced RL frameworks (GRPO, PPO/DPO, reward modeling, Process Reward Models, and RLAIF) to power platform-wide LLM alignment and agent behavior.
  • Pioneer Next-Gen Evaluation Harnesses: Design and calibrate bulletproof automated evaluation ecosystems, including human-aligned LLM-as-a-judge frameworks, dynamic benchmarking suites, and regression control setups.
  • Advance Distillation & Inference Optimization: Lead research initiatives on model distillation, quantization, context compression (e.g., gisting), and speculative decoding to optimize latency-cost trade-offs at platform scale.
  • Shape Agentic Platform Capabilities: Architect autonomous guiding agents and tool-use workflows, establishing foundational frameworks for multi-step reasoning, self-correction, and evaluation-driven feedback loops.
  • Technical Leadership & Mentorship: Elevate the engineering and research bar across the team through architectural design reviews, hands‑on mentoring of Senior ML Researchers, and establishing best practices for reproducible ML R&D.
  • Cross-Functional & Community Influence: Partner with Product and Engineering directors to translate complex client challenges into scalable platform architecture; author high-impact research write-ups, blog posts, and external tech talks.
What we're looking for
  • 6+ years in ML engineering or applied research, with 3+ years of proven track record leading LLM post‑training, fine‑tuning, or alignment initiatives at scale.
  • Deep theoretical and practical mastery of LLM alignment: SFT, LoRA/PEFT, RLHF/RLAIF (GRPO, DPO, PPO), reward modeling, and reasoning‑oriented post‑training.
  • Mastery of LLM Evaluation & Calibration: Proven history of building evaluation pipelines from scratch, addressing LLM-as-a-judge biases, and tightly calibrating automated evals against human gold standards.
  • Expert Distributed Training & Systems Engineering: Strong proficiency in Python, PyTorch, distributed training frameworks (Deepspeed, Megatron, FSDP), and high‑performance inference engines (vLLM, TensorRT-LLM).
  • Architectural Vision & Product Leadership: Demonstrated ability to map ambiguous technical challenges into robust platform features, balancing state‑of‑the‑art research with production reliability and cost constraints.
  • Technical Authority: Proven experience acting as a technical leader, mentoring senior researchers/engineers, and influencing cross‑functional roadmaps.
  • Language: Fluent spoken and written English (C1), with clear communication skills to articulate complex technical concepts to both internal teams and external clients/community.
What we can offer
  • You will be part of an international, dynamic environment that drives innovation and sets new standards in the AI and technology sector.
  • Competitive compensation package including base salary, bonus, and ESOP.
  • Paid PTO and benefits will vary depending on location.
  • We offer a full remote or hybrid model (if you are based in NL or Serbia).
  • IT setup and home office allowances.
Equal Opportunity Employer:

Toloka is committed to providing equal opportunity and fostering an inclusive environment. We welcome applications from all qualified individuals and do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, gender identity, age, marital status, veteran status, disability, or any other characteristic protected by applicable law. Selection decisions are made based on qualifications, merit, and business need.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Program Director - Enterprise Fine-Tuning
Program Director - Enterprise Fine-Tuning

Toloka • Spain

Remote
EUR 100,000 - 180,000
International environment
Competitive compensation package
Paid PTO and benefits
+2
Program Director — Enterprise Fine-Tuning
Program Director — Enterprise Fine-Tuning

Toloka • Spain

Hybrid
EUR 120,000 - 150,000
Base salary
Bonus
ESOP
+3
Senior ML Engineer (LLM Fine-tuning) - Full remote
Senior ML Engineer (LLM Fine-tuning) - Full remote

Social Discovery Group • Spain

Remote
EUR 60,000 - 95,000
Remote opportunity
Vacation 28 days per year
Wellness days 7 per year
+2
Senior LLM Engineer Research · Barcelona
Senior LLM Engineer Research · Barcelona

Multiverse • Barcelona

Hybrid
EUR 90,000 - 130,000
Indefinite contract
Private health insurance
Hybrid opportunity
+2
Member of Technical Staff - Post-Training (Reinforcement Learning & Generalization)
Member of Technical Staff - Post-Training (Reinforcement Learning & Generalization)

Cohere • Spain

Remote
EUR 186,000 - 336,000
Lunch stipend
Health and dental benefits
RRSP matching / 401K / Pension
+6
Software Architect - AI Agentic Platform (m/f/x/d)
Software Architect - AI Agentic Platform (m/f/x/d)

Symanto Research • Comunidad Valenciana

Hybrid
EUR 110,000 - 190,000
Hybrid work model
Training budget
20 days of workation
+3
Ultralytics LLM Engineer
Ultralytics LLM Engineer

Ultralytics • Colmenar Viejo

Hybrid
EUR 90,000 - 130,000
Time Off:24 days paid vacation, your生日
Home setup allowance
MacBook Air as work device
+1
Senior Ml Engineer
Senior Ml Engineer

Digital Waffle • Spain

Remote
GBP 110,000 - 190,000
LLM Engineer
LLM Engineer

Ultralytics LLC • Madrid

Hybrid
EUR 142,000 - 189,000
Equity
Hybrid schedule
24 days vacation
+2
Senior Llm Engineer
Senior Llm Engineer

Multiverse Computing • Barcelona

On-site
EUR 150,000 - 210,000
Indefinite contract
Equal pay guaranteed
Variable bonus
+11