Research, Post-Training Evals

Job summary

Responsibilities

We’re looking for a researcher to help develop reliable model evaluations for research signals. This role spans evaluation creation, usability, auditing, and efficiency.

• Create internal evaluations and research signals for capabilities and behaviors important to model research and post-training.
• Develop usability evaluations that measure whether models are genuinely useful in real research and product workflows, and partner with the data flywheel to turn evaluation…
• Improve evaluation robustness, including grader reliability, ambiguous ground truth, evaluator disagreement, false positives and negatives, and gaps between measured and intended…
• Build benchmark auditing methodologies that help researchers understand, trust, and appropriately use evaluation signals.
• Develop specialized agentic evaluation environments and user simulators, including supporting harness development and studying cross-user, cross-harness and cross-environment…

What they're looking for

• Bachelor’s degree or equivalent experience in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline with strong theoretical and empirical grounding.
• Experience designing, building, or analyzing evaluations, benchmarks, datasets, graders, or other measurement systems.
• Strong written and verbal communication skills, with the ability to collaborate effectively across research and engineering teams.
• Experience with LLMs, post-training, reinforcement learning, or agentic systems.
• Experience with evaluation auditing, human evaluations, LLM-judges, or open-ended task evaluation.

Tech stack

• Python
• PyTorch
• TensorFlow
• JAX