Research Engineer, Mid-Training

Job summary

Responsibilities

Mid-training sits at the seam between pre-training and post-training and is one of the highest-leverage points in the entire model pipeline. This is where raw base model capability is sharpened into something that can reason deeply, generalize reliably, and serve as the foundation that…

• Data Mix and Quality Uplift: Design and iterate on high-quality data mixtures for late-stage and annealing training runs.
• Capability Injection: Drive targeted improvements in coding, mathematics, and long-horizon reasoning through curated data strategies and training interventions.
• Synthetic Data Research: Develop and evaluate synthetic data pipelines that generate training signal at scale.
• Annealing and Schedule Design: Research and optimize multi-stage learning rate schedules, warmup strategies, and compute allocation across training phases.
• Context Length Extension: Research and implement methods for extending effective context length without degrading short-context performance.

Tech stack

• Python
• PyTorch