Research Intern: Speculative Decoding for Faster LLMs

Together AI; San Francisco (CA); On-site; USD 80,000 - 96,000; Full time

Benefits offered by this job

Housing stipend

Competitive compensation

Job summary

Together AI, the AI Native Cloud, is seeking a research intern to work on speculative decoding for frontier language models. You will train and evaluate draft models at scale, explore new architectures, and validate ideas with realistic workloads on dedicated GPU clusters.

As an intern, you will contribute to open questions in draft models, distillation strategies, and verification schemes while communicating progress to the team and documenting results for wider dissemination.

Qualifications

• Strong knowledge of ML and DL fundamentals.
• Hands-on experience training or fine-tuning large models with distributed training frameworks.
• Familiarity with Mixture-of-Experts and hybrid attention architectures.

Responsibilities

• Research and prototype new speculative decoding algorithms and draft model architectures.
• Train, fine-tune and distill draft models and design rigorous experiments.
• Implement optimized serving engine versions for real-world workloads.
• Communicate plans, progress and results to the broader team.
• Share findings via technical write-ups and blog posts and help productionize ideas.

Skills

Machine Learning fundamentals

Deep Learning

Distributed training

Education

Master's or PhD in CS/EE or related

Job description

Together AI, the AI Native Cloud, is seeking a research intern to work on speculative decoding for frontier language models. You will train and evaluate draft models at scale, explore new architectures, and validate ideas with realistic workloads on dedicated GPU clusters.

As an intern, you will contribute to open questions in draft models, distillation strategies, and verification schemes while communicating progress to the team and documenting results for wider dissemination.