Research, Pre-Training Data

Job summary

Responsibilities

The role of pre-training researchers sits at the core of our roadmap. This work blends research with large-scale data engineering to help assemble the pre-training datasets and data systems that underpin the next generation of AI models.

• Design and implement techniques for curating, sourcing, and filtering large-scale text, code, and multimodal data.
• Develop data quality metrics and analysis to measure coverage, diversity, and representativeness across sources.
• Collaborate with research and infrastructure teams to scale data processing systems efficiently and reproducibly.
• Investigate and mitigate data risks, including privacy, safety, and licensing concerns, to ensure responsible and ethical data use.
• Continuously evaluate dataset improvements by analyzing their downstream effects on model learning and behavior.

What they're looking for

• Proficiency in Python and familiarity with at least one deep learning framework (e.g., PyTorch, TensorFlow, or JAX).
• Bachelor’s degree or equivalent experience in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline with strong theoretical and empirical grounding.
• Clarity in communication, an ability to explain complex technical concepts in writing.
• A strong grasp of probability, statistics, and ML fundamentals. You can look at experimental data and distinguish between real effects, noise, and bugs.
• Experience with curation, preprocessing, and analysis of large-scale text, code, or multimodal datasets.

Tech stack

• Python
• PyTorch
• TensorFlow
• JAX