Engineering Manager, Inference Infrastructure

Job summary

Responsibilities

Every request that hits Claude — from claude.ai, the API, our cloud partners, or internal research — depends on a set of decisions made before it ever reaches a model: where each request should be served and how much capacity each model needs right now.

• Own the technical roadmap for how the inference fleet is coordinated — where traffic goes, where capacity lives, how caches are placed, how fast the system reacts to demand, and…
• Partner with the product, inference engine, performance, and capacity teams to identify throughput, latency, utilization, and cost wins, then turn those into shipped improvements…
• Build the group's habit of quantitative modeling: claim a win only when you can measure it, and know before you ship what the expected effect is
• Set technical strategy for how the control plane evolves across heterogeneous hardware, across multiple cloud providers, and across all our serving surfaces
• Run the group's operational backbone — on-call rotations, incident response, postmortem review, deploy safety — so the teams can ship aggressively without the system becoming…

What they're looking for

• Engineering management experience leading teams on critical-path production infrastructure at scale
• A deep systems background — load balancing, scheduling, cluster orchestration, autoscaling, cache-coherent distributed state, high-performance networking, or similar — with…
• Experience shipping performance or efficiency improvements in large-scale systems, and the ability to explain, with numbers, what the impact was — including the cost side, not…
• Experience running production infrastructure with real operational stakes: on-call, incident response, capacity events, deploy discipline
• A results-oriented, impact-driven approach, and comfort working in a space where throughput, latency, cost, stability, launch timelines, and feature velocity all pull in…

Tech stack

• Kubernetes