Product Manager, Inference Platform

Job summary

Responsibilities

Product at Baseten is a nascent function. Our company today has a strong engineering culture, is heavily customer-obsessed, and moves fast. We're building the product function now, and you'd be one of the people who defines it.

• Own how workloads scale and where they land, from autoscaling to demand (up under load, down to zero when idle) to a single placement policy that expresses region, compliance…
• Give compliance-bound workloads right-of-way on sensitive capacity.
• Make production inference reliable by default, so every request reaches a healthy replica and rolling deploys never drop traffic.
• Define region-aware routing, with multi-region and active-active failover as first-class policy, plus health-aware recovery from stuck or bad replicas.
• Build the release engine beneath safe rollouts: the traffic-shifting that powers canary, shadow, and A/B, along with warm-ups, drain, and probes.

What they're looking for

• 8+ years in product management, including deep experience with infrastructure, distributed systems, or ML serving.
• Fluency reasoning about scaling, routing, failover, and the cost/performance frontier, at a level that earns the respect of staff engineers.
• Experience owning capabilities end to end, backend through UX, rather than a single slice.
• Track record driving cross-team roadmaps and the dependencies beneath them.
• Comfort defining a category that doesn't fully exist yet.

Tech stack

• Spark
• Kubernetes
• vLLM