Technical Program Manager, Inference

Job summary

Responsibilities

Anthropic’s Inference organization serves Claude to millions of users and enterprise customers, across our own products and the world’s largest cloud platforms, with the speed, reliability, and efficiency that frontier AI demands.

• Scaling and infrastructure. Routing, autoscaling, and capacity management for the serving fleet, and bringing new capacity online.
• Velocity. The toolchains, development environments, and CI/CD that let inference engineers build, test, and ship across every accelerator platform.
• Inference engine. The software between the accelerator kernels and the routing layer that serves every request.
• Performance. Improving throughput, latency, cost, and reliability across the stack, with correctness treated as part of performance.
• Own the programs that scale Claude’s serving fleet across accelerator and cloud platforms, from early direction through delivery.

What they're looking for

• 7+ years of experience as a TPM, or in an equivalent engineering or SRE lead role, owning large multi-team infrastructure programs end to end in production distributed systems…
• Technical fluency in large-scale serving infrastructure, including request routing and load balancing, autoscaling, capacity planning, cluster orchestration, deployment and…
• A working understanding of how LLM inference uses hardware (prefill and decode, batching, caching, compute versus memory bandwidth versus interconnect), or a track record of…
• A track record of building program mechanisms from scratch on ambiguous, fast-moving efforts, and of retiring process that has stopped earning its keep.
• The ability to drive outcomes across organizations you don't manage, with competing priorities and unclear decision ownership.

Tech stack

• Triton
• Kubernetes