Senior Software Engineer, ML API – AI Inference

Job summary

Responsibilities

Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine.

• Own features end to end. Design, implement, test, and operate features for chat completions, text generation, streaming, model configuration, tool calling, structured outputs…
• Lead multi-component delivery. Write clear design documents, evaluate technical tradeoffs, break projects into deliverable milestones, and coordinate implementation with a small…
• Deliver consistent API behavior. Implement clear request and response semantics across GPU prefill, Cerebras decode, and other inference backends.
• Enable new models and capabilities. Integrate foundation models, tokenizers, chat templates, generation configuration, sampling methods, and model-specific features into the…
• Integrate serving components. Extend inference services and their integration with frameworks and runtimes such as vLLM, PyTorch, Hugging Face libraries, LLGuidance, and Cerebras…

What they're looking for

• 5+ years of software engineering experience, including independent ownership of production services, distributed systems, or substantial software components.
• Strong programming ability in Python and proficiency in Go, C++, Rust, or another language used to build production services.
• Experience designing and delivering features that span multiple components, including documenting designs, evaluating tradeoffs, and coordinating implementation with other…
• Experience building and evolving production APIs, with sound practices for validation, error handling, compatibility, and testing.
• Experience integrating software across service, framework, runtime, or infrastructure boundaries.

Tech stack

• vLLM
• PyTorch
• Python
• Go
• C++
• Rust
• Kubernetes
• Transformers