(Staff / Senior Staff) AI Infra Engineer, Inference Infrastructure

Job summary

Responsibilities

We're looking for a software engineer, reliability-minded but still a builder, to own what happens in production to the Kubernetes-native control plane that provisions and runs our GPU inference fleet.

• Operate the provisioning state machine in production: own the health of the host lifecycle pipeline, discovery, inference bring-up, GPU driver/CUDA validation, health checks…
• Triage and extend automated self-healing: when the node-scanning and remediation platform misses something, a new failure mode, an InfiniBand issue in disaggregated…
• Own end-to-end reliability of the pipeline: hold the SLOs and error budgets for provisioning and fleet health, lead incident response when idempotency, retry, rollback, or drift…
• Partner with the inference/ML platform team from the production side: be the one who knows how real cluster topologies, interconnects, and scheduling constraints actually fail in…
• Work alongside machine learning researchers, engineers, and product managers to bring our latest technologies into production, enabling advanced research through awesome…

What they're looking for

• Experience operating a provisioning or host-lifecycle pipeline in production, comfortable troubleshooting stuck or failed state transitions (discovery, bring-up, GPU driver/CUDA…
• Deep experience owning production reliability for large-scale distributed systems; GPU/ML-serving infrastructure experience is a strong plus.
• Hands-on experience troubleshooting InfiniBand/RDMA networking issues, ideally in a multi-node training or disaggregated-inference context — comfortable reading IB port counters…
• Familiarity with the operational realities of multi-tenant model serving (LoRA/adapter-based serving) and with running differentiated SLOs across model pools of different priority.
• A demonstrated track record of eliminating operational toil through automation and tooling you built yourself, not just process or documentation.

Tech stack

• Kubernetes
• CUDA