Research Intern: Speculative Decoding for Frontier LLMs

Together Computer Inc; San Francisco (CA); On-site; USD 80,000 - 96,000; Full time

Benefits offered by this job

Housing stipends

Competitive compensation

Job summary

Together AI in San Francisco is seeking a research intern for the Speculative Decoding team to explore faster draft-model generation for frontier open models such as DeepSeek, Kimi and GLM. You will train and fine-tune draft models at scale on GPU clusters and test ideas against realistic serving workloads.

Applicants should be pursuing a Master's or Ph.D. in CS or EE, with strong ML/DL fundamentals and hands-on experience with distributed training frameworks (Megatron-LM, FSDP).

Qualifications

• Currently pursuing a Master's or Ph.D. in CS, EE, or a related field.
• Strong knowledge of ML/DL fundamentals.
• Hands-on experience training or fine-tuning large models with distributed frameworks.

Responsibilities

• Research and prototype new efficient speculative decoding algorithms and draft model architectures
• Train, fine-tune and distill draft models, and design rigorous experiments to validate hypotheses.
• Implement optimized version for serving engines to show efficacy in real world workloads
• Communicate the plans, progress, and results of projects to the broader team
• Share findings through technical write-ups and blog posts, and help bring successful ideas into production

Skills

Machine Learning

Deep Learning

Distributed training

Mixture-of-Experts

Hybrid attention

Education

Master's or Ph.D. in Computer Science or related field

Tools

Megatron-LM

FSDP

vLLM

SGLang

TensorRT-LLM

Job description

Together AI in San Francisco is seeking a research intern for the Speculative Decoding team to explore faster draft-model generation for frontier open models such as DeepSeek, Kimi and GLM. You will train and fine-tune draft models at scale on GPU clusters and test ideas against realistic serving workloads.

Applicants should be pursuing a Master's or Ph.D. in CS or EE, with strong ML/DL fundamentals and hands-on experience with distributed training frameworks (Megatron-LM, FSDP).