Research Engineer, Infrastructure, Numerics

Job summary

Responsibilities

We’re looking for an infrastructure research engineer to design and build the core systems that enable efficient large-scale model training with a focus on numerics. You will focus on improving the numerical foundations of our distributed training stack, from precision formats and kernel…

• Design and optimize distributed training infrastructure for large-scale LLMs, focusing on performance, stability, and reproducibility across multi-GPU and multi-node setups.
• Implement and evaluate low-precision numerics (for example, BF16, MXFP8, NVFP4) to improve efficiency without sacrificing model quality.
• Develop kernels and communication primitives that use hardware-level support for mixed and low-precision arithmetic.
• Collaborate with research teams to co-design model architectures and training recipes that align with emerging numeric formats and stability constraints.
• Prototype and benchmark scaling strategies such as data, tensor, and pipeline parallelism that integrate precision-adaptive computation and quantized communication.

What they're looking for

• Bachelor’s degree or equivalent experience in computer science, electrical engineering, statistics, machine learning, physics, robotics, or similar.
• Understanding of deep learning frameworks (e.g., PyTorch, JAX) and their underlying system architectures.
• Thrive in a highly collaborative environment involving many, different cross-functional partners and subject matter experts.
• A bias for action with a mindset to take initiative to work across different stacks and different teams where you spot the opportunity to make sure something ships.
• Strong engineering skills, ability to contribute performant, maintainable code and debug in complex codebases in areas such as floating-point numerics, low-precision arithmetic…

Tech stack

• PyTorch
• JAX