Engineering Manager, Inference Performance

Job summary

Responsibilities

We're looking for an Engineering Manager to lead part of our Inference Performance team. This team makes the world's most demanding AI workloads run faster and more efficiently on GPUs.

• Lead, mentor and grow a team of inference performance engineers through regular 1:1s, clear feedback, career development and performance reviews.
• Hire top GPU and inference engineering talent, and build a strong, collaborative team culture as the runtime team scales.
• Own the technical roadmap and execution for runtime performance work, balancing customer needs, new model launches and long-term platform investments.
• Stay close to the technical work. Review designs, guide profiling and optimization efforts, and help the team reason from first principles about where time and memory go.
• Drive the productionization of inference techniques such as quantization, speculative decoding, KV-cache reuse, chunked prefill and custom scheduling.

What they're looking for

• Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or a related field.
• Experience managing engineers, including hiring, mentoring, giving feedback and running performance reviews.
• Experience leading or closely supporting GPU optimization teams in training, inference or recommendation systems.
• Strong technical depth in GPU workloads, with a solid understanding of GPU architecture and performance tradeoffs.
• Familiarity with ML libraries such as PyTorch, TensorRT or TensorRT-LLM.

Tech stack

• Spark
• PyTorch
• vLLM
• CUDA
• Triton