Software Engineer, GPU Inference

Job summary

Responsibilities

Cerebras is building a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine.

• Productionize the GPU inference stack. Design, build, deploy, and maintain the complete GPU prefill path, spanning API services, model-serving workers, vLLM, PyTorch, ROCm, GPU…
• Own GPU operational readiness. Establish deployment, upgrade, rollback, health-checking, capacity-management, and failure-recovery practices for the AMD GPU fleet.
• Drive reliability in production. Define service-level indicators and objectives for GPU-backed inference. Improve fault isolation, graceful degradation, automated recovery…
• Improve inference performance. Profile and optimize time to first token, request throughput, tokens per second per GPU, tail latency, GPU utilization, memory efficiency, and…
• Optimize model-serving behavior. Tune and improve scheduling, continuous batching, prefix caching, KV-cache management, tensor and expert parallelism, request admission…

What they're looking for

• 5+ years of software engineering experience, including substantial individual-contributor ownership of complex production systems.
• Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
• Strong programming ability in C++ and Python, including experience with multithreading, concurrency, memory management, and performance-sensitive software.
• Hands-on experience with a high-performance model-serving framework such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or an equivalent internally developed system.
• Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead, and profiling…

Tech stack

• vLLM
• PyTorch
• C++
• Python
• Triton
• Kubernetes
• CUDA