ML/Research Engineer, Safeguards

Job summary

Responsibilities

We are looking for ML Engineers and Research Engineers to help detect and mitigate misuse of our AI systems. As a member of the Safeguards ML team, you will build systems that identify harmful use—from individual policy violations to sophisticated, coordinated attacks—and develop defenses that…

• Develop classifiers to detect misuse and anomalous behavior at scale. This includes developing synthetic data pipelines for training classifiers and methods to automatically…
• Build systems to monitor for harms that span multiple exchanges, such as coordinated cyber attacks and influence operations, and develop new methods for aggregating and analyzing…
• Evaluate and improve the safety of agentic products—developing both threat models and environments to test for agentic risks, and developing and deploying mitigations for prompt…
• Conduct research on automated red-teaming, adversarial robustness, and other research that helps test for or find misuse

What they're looking for

• Language modeling and transformers
• Building classifiers, anomaly detection systems, or behavioral ML
• Adversarial machine learning or red-teaming
• Interpretability or probes
• Reinforcement learning

Tech stack

• Python
• Transformers