Member of Technical Staff - Pretraining

Job summary

Responsibilities

We're building the foundation models that power the next wave of visual intelligence — and pretraining is where that work begins. This role sits at the center of our research effort.

• Lead large-scale pretraining experiments for our multimodal (image, video, audio) foundation models (architecture, objective functions, scaling strategies)
• Develop and evaluate novel ideas across architecture, optimizers, and training algorithms.
• Contribute across the full stack: low-level GPU and systems optimizations, research code, and high-level model design
• Lead focused research projects independently and drive larger cross-team initiatives

What they're looking for

• You've led or co-owned pretraining for a foundation model (image, video, LLM, or multimodal) that shipped to production or a major release
• Own architectural calls that move the model: attention patterns, modulation schemes, loss formulations, tokenization strategies
• Deep experience with large-scale distributed training: FSDP/TP/PP, multi-node runs at 500+ GPUs, debugging loss spikes, NaNs, throughput regressions, and silent correctness…
• Strong intuition for architecture and objective design — you've made calls on attention patterns, modulation schemes, or loss formulations that moved a real model
• Track record of shipping: top-venue publications (NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV) paired with production impact, or unambiguous production wins at a frontier lab

Tech stack

• Python
• PyTorch