Inference Systems Performance Architect

Job summary

Responsibilities

You will be the architect for end-to-end inference performance: how a request moves through tokenization, prefill, decode, and the fabric between them, and how a deployment is sized against customer SLOs. The work has two coupled pillars.

• Define and drive the technical strategy for inference-systems performance including workload capture, benchmarking, modeling, and simulation, while developing and architecture…
• Build the workload-capture and agentic-benchmarking capability - capture representative production traffic and enforce the discipline of interrogating results, spotting…
• Own the performance-modeling and simulation practice - models that predict how a configuration change moves the output, informing capacity planning against customer SLOs and…
• Attack the end-to-end profiling gap - drive tooling that produces accurate, actionable profiles of a distributed inference pipeline so bottlenecks can be localized across host…
• Serve as the senior technical voice across model-optimization, systems, hardware, and product, tying together multiple engineering activities and teams, and weighing trade-offs…

What they're looking for

• B.S. in Computer Science, Computer Engineering, or Related Field
• 12+ years of experience in performance engineering, with a demonstrated record of technical leadership on large-scale, complex systems
• Deep expertise in end-to-end performance analysis of distributed systems with many moving parts and the ability to localize bottlenecks that others cannot
• Proven command of realistic workload generation and simulation and of performance modeling, including calibrating models against real, variable workloads
• Demonstrated ability to enter an unfamiliar domain and apply core performance methods with transferable discipline expertise