Responsibilities
Research and prototype new efficient speculative decoding algorithms and draft model architectures
Train, fine-tune and distill draft models, and design rigorous experiments to validate hypotheses.
Implement optimized version for serving engines to show efficacy in real world workloads
Communicate the plans, progress, and results of projects to the broader team
Share findings through technical write-ups and blog posts, and help bring successful ideas into production
Requirements
Currently pursuing a Master's or Ph.D. degree in Computer Science, Electrical Engineering, or a related field
Strong knowledge of Machine Learning and Deep Learning fundamentals
Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP)
Familiarity with modern model architectures, such as Mixture-of-Experts and hybrid attention
Preferred Qualifications
Prior research experience in speculative decoding, efficient inference, or efficient machine learning
Publications at leading conferences in machine learning or systems (e.g., NeurIPS, ICML, ICLR, MLSys)
Experience with knowledge distillation, post-training, or reinforcement learning
Familiarity with LLM serving engines (e.g., vLLM, SGLang, TensorRT-LLM)
Contributions to open-source machine learning projects
About Together AI
Together AI, the AI Native Cloud, is purpose-built for AI engineers. AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence, all around a marketplace of leading open models that teams can run, adapt, and own. Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom, Together serves 400+ trillion tokens a month.
Internship Program Details
Our internship program runs 12 to 14 weeks, giving you the opportunity to work alongside industry-leading engineers and researchers across multiple teams. This cohort's internship dates span May 17th to August 6th or June 14th to September 3rd.
Compensation
We offer competitive compensation, housing stipends, and other competitive benefits. The estimated US hourly rate for this role is $58 - $70 an hour. Our hourly rates are determined by location, level and role. Individual compensation will be determined by experience, skills, and job-related knowledge.
Application Policy
Please read each role carefully and apply to the position best suited to you. Candidates are limited to 6 applications within a 90-day period, and applications beyond that limit will be automatically rejected. You may apply to additional roles 90 days after your last application.
Equal Opportunity
Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.
For more information about our privacy policy, please visit: https://www.together.ai/privacy
