Research Engineer, Post-Training Model Evaluations

Job summary

Responsibilities

Anthropic's production models undergo sophisticated post-training processes to enhance their capabilities, alignment, and safety. Decisions in production training depend on knowing how the model is really behaving.

• Steer the eval strategy for production Claude models: study and optimize the eval mix so it gives the most accurate and complete assessment of model quality
• Run and monitor the eval fleet live to understand how each Claude model is developing as it trains
• Advise teams across post-training on eval methodology, and help eval authors bring new evals up to the bar for production
• Investigate regressions in production runs and inform training interventions when appropriate
• Build the dashboards, alerts, and reports that researchers and leadership use to track model quality

Tech stack

• Python