Research Engineer, Large-Scale Training

Job summary

Responsibilities

The Model Shaping team at Together AI works on products and research for tailoring open foundation models to downstream applications. We build services that allow machine learning developers to choose the best models for their tasks and further improve these models using domain-specific data.

• Design, implement, and optimize core components of Together's large-scale training infrastructure.
• Integrate new model architectures, validate training correctness and convergence, and optimize performance for production fine-tuning workloads.
• Profile distributed training workloads to identify and eliminate bottlenecks across compute, memory, and communication.
• Design and execute experiments to validate performance hypotheses and benchmark new approaches against state-of-the-art methods.
• Partner closely with Research Scientists to productionize novel training methods and contribute to publications and open-source releases.

What they're looking for

• Demonstrated ability to independently take ambiguous performance or infrastructure problems from investigation through deployment.
• Strong programming skills in Python and PyTorch, with an emphasis on writing efficient, maintainable code.
• Hands-on experience training or fine-tuning large neural networks in multi-GPU or multi-node environments.
• Solid understanding of ML systems fundamentals, including GPU architecture, mixed-precision training, and distributed training paradigms such as data, tensor, pipeline, or expert…
• Strong communication skills and the ability to collaborate effectively with both researchers and engineers.

Tech stack

• Python
• PyTorch
• CUDA
• Triton