Staff Applied AI Inference Engineer

Job summary

Responsibilities

You will spend your time making large language models run faster, cheaper, and more reliably in production. That means owning the inference stack end to end: profiling where time and cost go, bringing modern optimization techniques into real deployments, and getting deep into the serving code when…

• Bring current inference techniques into production and refine them.
• Design and optimize serving architectures, including prefill and decode disaggregation, request routing, and related approaches.
• Work down into the serving stack, from frameworks like vLLM and SGLang to the CUDA kernels underneath, profiling and running in-depth analysis to find and fix performance problems.
• Adapt and scale optimization methods across many kinds of ML models, with an emphasis on large language models.
• Profile and tune deployments against clear targets for latency, throughput, and cost, and keep them dependable under real traffic.

What they're looking for

• A Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field.
• Hands-on experience shipping code in production with one or more general-purpose languages, such as Python or C++, with a strong preference for Python.
• Familiarity with methods for optimizing LLMs for high throughput / low latency inference.
• Comfort with modern LLM serving frameworks such as vLLM or SGLang, and with profiling and analyzing performance down to the kernel level.
• A firm grasp of how GPUs are built and how they behave.

Tech stack

• Go
• vLLM
• CUDA
• Python
• C++
• Docker
• Kubernetes