Speculative Decoding Research Intern

Together Computer Inc; San Francisco (CA); On-site; USD 80,000 - 96,000; Full time

Benefits offered by this job

Housing stipend

Job summary

Together AI in the United States is seeking a research intern to explore speculative decoding, including new drafter architectures and distillation techniques. You will train and fine-tune draft models at scale on GPU clusters and evaluate against realistic serving workloads with long-context and agentic traffic.

The internship runs 12 to 14 weeks, with compensation including housing stipends. Applicants are currently enrolled in a Master’s or PhD program in CS/EE and have experience with

Qualifications

• Pursuing a Master’s or PhD in CS/EE or related field.
• Experience training or fine-tuning large models with distributed frameworks.
• Familiar with Mixture-of-Experts and hybrid attention.
• Strong ML/DL fundamentals and research mindset.

Responsibilities

• Research and prototype new speculative decoding algorithms and draft model architectures.
• Train, fine-tune and distill draft models and design experiments to validate hypotheses.
• Implement optimized serving engine versions for real-world workloads.
• Communicate plans and results to the broader team.
• Publish findings through write-ups and blog posts.

Skills

Machine Learning

Deep Learning

Distributed training

Model architectures

Education

Master's or PhD in CS/EE or related

Tools

Megatron-LM

FSDP

vLLM

TensorRT-LLM

Job description

Together AI in the United States is seeking a research intern to explore speculative decoding, including new drafter architectures and distillation techniques. You will train and fine-tune draft models at scale on GPU clusters and evaluate against realistic serving workloads with long-context and agentic traffic.

The internship runs 12 to 14 weeks, with compensation including housing stipends. Applicants are currently enrolled in a Master’s or PhD program in CS/EE and have experience with