Software Engineer, Inference AI/ML

Job summary

Responsibilities

Join the Inference team to ship production features that improve latency, reliability, and cost for model serving on our GPU platform. As an IC1, you’ll implement well-scoped changes, learn our operational practices, and grow quickly with mentorship from experienced engineers.

• Implement well-scoped features and fixes in Python/Go/C++ for model-serving services (e.g., Triton, vLLM, TensorRT-LLM, Ray Serve).
• Write tests, code comments, and short design docs; participate in code reviews.
• Add basic metrics and dashboards; assist with alarms and runbooks.
• Follow on-call runbooks and learn incident response in a guided rotation.
• Contribute to performance experiments (e.g., request batching, concurrency, caching) with guidance.

What they're looking for

• BS/MS in CS, EE, or related field, or equivalent practical experience.
• Foundations in data structures, algorithms, and networked services.

Tech stack

• Python
• Go
• C++
• Triton
• vLLM
• Kubernetes
• PyTorch
• TensorFlow