Research Intern, Frontier Speculative Decoding (Winter 2027)

Job summary

Responsibilities

The Speculative Decoding team at Together AI builds the draft models and algorithms that let large language models generate faster without changing their outputs. Our speculators run in production on frontier open models such as DeepSeek, Kimi and GLM.

• Research and prototype new efficient speculative decoding algorithms and draft model architectures
• Train, fine-tune and distill draft models, and design rigorous experiments to validate hypotheses.
• Implement optimized version for serving engines to show efficacy in real world workloads
• Communicate the plans, progress, and results of projects to the broader team
• Share findings through technical write-ups and blog posts, and help bring successful ideas into production

What they're looking for

• Currently pursuing a Master's or Ph.D. degree in Computer Science, Electrical Engineering, or a related field
• Strong knowledge of Machine Learning and Deep Learning fundamentals
• Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP)
• Familiarity with modern model architectures, such as Mixture-of-Experts and hybrid attention
• Prior research experience in speculative decoding, efficient inference, or efficient machine learning

Tech stack

• vLLM