Cyber Evaluations Engineer

Job summary

Responsibilities

We're hiring Cyber Evaluations Engineers to build and run the evaluations that measure cyber-relevant capabilities and safeguard robustness in our models. You'll design new evals, run per-release robustness testing, and dig into data on jailbreaks and prompt bypasses to understand where our…

• Design and run capability, uplift, and safety evaluations to assess cyber-relevant risk in new models
• Execute per-release safeguard-robustness testing ahead of major model launches
• Analyze evaluation results and communicate findings clearly to the team and to stakeholders
• Design, prototype, and tune detection probes for cyber misuse
• Work with the cyber policy team to turn policy lines into a layered, robust abuse-detection architecture, and measure its precision and coverage over time

What they're looking for

• Experience building or running evaluations, benchmarks, or test suites for software or ML systems, including delivering results on short, fixed timelines
• Hands-on cybersecurity experience (e.g., CTF participation, vulnerability research, exploit development, or security research)
• Proficiency in Python
• Strong ability to communicate evaluation results with multiple cross-functional stakeholders or potential policy stakeholders
• Deep offensive-security or security-research experience, including experience building AI security benchmarks

Tech stack

• Python