Speculative Decoding Research Intern: AI Systems & Training

Together AI; San Francisco (CA); On-site; USD 80,000 - 96,000; Full time

Benefits offered by this job

Housing stipend

Competitive compensation

Job summary

Together AI is seeking an intern for the Speculative Decoding team to explore efficient draft models and verification schemes. You will train and fine-tune draft models at scale on dedicated GPU clusters, testing ideas against realistic serving workloads, including long-context and agentic traffic.

Internship includes housing stipends and competitive compensation, with projects spanning model training, inference optimization, and production-readiness.

Qualifications

• Pursuing a Master’s or PhD in CS, EE, or related field.
• Strong knowledge of ML and DL fundamentals.
• Hands-on experience training or fine-tuning large models with distributed frameworks (Megatron-LM, FSDP).
• Familiarity with modern model architectures (Mixture-of-Experts, hybrid attention).

Responsibilities

• Research and prototype new speculative decoding algorithms and draft model architectures.
• Train, fine-tune and distill draft models; design rigorous experiments to validate hypotheses.
• Implement optimized versions for serving engines to show efficacy in real-world workloads.
• Communicate plans, progress, and results to the broader team.
• Share findings through write-ups and blog posts and help bring ideas into production.

Skills

ML fundamentals

DL fundamentals

Distributed training

Mixture-of-Experts

Hybrid attention

Education

Master's or Ph.D. in CS/EE

Tools

Megatron-LM

FSDP

vLLM

SGLang

TensorRT-LLM

Job description

Together AI is seeking an intern for the Speculative Decoding team to explore efficient draft models and verification schemes. You will train and fine-tune draft models at scale on dedicated GPU clusters, testing ideas against realistic serving workloads, including long-context and agentic traffic.

Internship includes housing stipends and competitive compensation, with projects spanning model training, inference optimization, and production-readiness.