Member of Technical Staff - Research, Inference

Job summary

Responsibilities

Most of the value of owning a model shows up at serving time. We’re building a platform that covers the whole life of an LLM (train it, deploy it, observe it), and inference is where teams feel the difference every day.

• Own end-to-end inference research bets: speculative decoding, disaggregated prefill/decode, quantization (FP8, INT4), KV-cache and memory management, autoscaling for spiky…
• Train custom speculators against real production traffic and feed what you learn back into target models; acceptance length is the metric that decides the win
• Work directly with customers alongside our Forward Deployed Engineers to deploy and tune models, and bring what you learn back into the research
• Carry and expand collaborations with outside research labs, for example:
• our work with ZLab on DFlash, a speculator design built on KV injection and blockwise parallel drafting