Staff Software Engineer, Inference API

Job summary

Responsibilities

Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine.

• Build production ML inference APIs. Design, implement, and maintain APIs for chat completions, text generation, streaming, model configuration, tool calling, structured outputs…
• Deliver a unified serving experience. Create consistent request and response semantics across GPU prefill, Cerebras decode, and other heterogeneous inference backends.
• Enable new models and capabilities. Integrate emerging foundation models, tokenizers, prompt formats, sampling methods, attention variants, multimodal inputs, and model-specific…
• Own API compatibility and evolution. Maintain compatibility with widely adopted inference interfaces while designing Cerebras-specific extensions.
• Integrate with model-serving runtimes. Extend and integrate custom inference services with vLLM, PyTorch, Hugging Face libraries, the AMD ROCm stack, and Cerebras runtime…

What they're looking for

• 5+ years of software engineering experience, including substantial individual-contributor ownership of production software or distributed systems.
• Strong programming ability in Python and Go plus experience developing performance-sensitive or highly concurrent services in C++, Rust, or a similar systems language.
• Experience building stable APIs with clear validation, error handling, observability, compatibility, and versioning practices.
• Experience integrating software across service, framework, runtime, and infrastructure boundaries.
• Experience designing or maintaining OpenAI-compatible, gRPC, REST, or streaming inference APIs.

Tech stack

• vLLM
• PyTorch
• Python
• C++
• Rust
• Kubernetes
• Transformers
• Triton