Research Engineer, Inference Foundation

Job summary

Responsibilities

The Inference Foundation team owns the core of Mistral's inference stack: the inference engine and its orchestration, from the feature set and configuration that serve our models in production to the release machinery that keeps the stack current and production-grade.

• Optimize the inference stack at scale — feature development and fixes in the engine and orchestrator, squeezing more throughput and lower latency out of every GPU under strict…
• Make capacity elastic — scaling up and down should be cheap and fast, not a performance cliff.
• Power the training of our frontier models — high-performance serving that keeps RL and post-training loops running at full speed.

What they're looking for

• Experience building and running ML/LLM services at scale, with clear latency and availability targets
• Hands-on experience with inference engines such as vLLM, SGLang, TensorRT-LLM, or others
• A solid grasp of inference internals: prefill vs. decode, KV-cache behavior, batching, scheduling, speculative decoding, parallelism strategies
• Familiarity with distributed and disaggregated serving architectures
• Comfortable debugging across the full stack — CUDA/NCCL, kernels, containers, networking, storage

Tech stack

• vLLM
• CUDA
• Python
• PyTorch
• Kubernetes
• Triton
• Rust
• C++