Staff+ Software Engineer, ML Sampling Path

Job summary

Responsibilities

The Safeguards ML Sampling Path team builds and operates the production services that power Claude's safety systems. These services sit on the token generation path across every platform Claude runs on: every request must pass through them, and each millisecond of added latency is wait time for…

• Design, build, and operate the backend systems that process every token on the generation path for Claude requests, including the streaming contract with the API and inference…
• Own latency and reliability end to end: define and maintain SLOs and error budgets for added latency, time-to-first-token, and availability, and lead incident response and…
• Ship changes to the hot path rapidly but safely — canaried and gradual rollouts, error budget and latency gating, fast rollbacks — and drive per-token performance: chase tail…
• Set technical direction for the sampling path: lead design reviews, make latency, reliability, and cost trade-off calls with the inference and research teams, mentor engineers…