Senior ML Infrastructure Engineer

Job summary

Responsibilities

The Senior Software Engineer, ML Infrastructure will be responsible for designing, building, and operating the production-grade inference infrastructure that powers SambaNova's serving stack on our Reconfigurable Dataflow Unit (RDU) architecture.

• Design and productionize advanced inference techniques on RDU to optimize for performance and cost. Key areas include speculative decoding, constrained decoding, function/tool…
• Own SambaNova's integration with vLLM and adjacent serving frameworks, adapting them to RDU's architecture.
• Own the public inference API surface exposed through SambaStack and SambaCloud.
• Build and maintain the accuracy verification and regression infrastructure that gates every inference feature shipped to customers.
• Partner with ML, compiler, runtime, and product teams to take inference features from prototype to production.

What they're looking for

• B.S. in Computer Science, Electrical Engineering, or related field
• 5+ years of industry experience building and operating large-scale distributed systems, ideally in ML serving
• Strong software engineering fundamentals: algorithms, data structures, concurrency, and systems design
• Experience designing and maintaining production services with strict latency, throughput, and availability requirements
• Working knowledge of modern LLM inference techniques and familiarity with open-source serving stacks such as vLLM, TensorRT-LLM, or SGLang

Tech stack

• vLLM
• Python