Software Engineer, Training/Inference (C++)

Job summary

Responsibilities

At SpaceXSI we are building the high-performance inference platform that serves Grok to millions of users every day. As a Software Engineer, Training/Inference (C++), you will design and optimize large-scale model serving systems end-to-end.

• Architect and implement scalable distributed infrastructure for model serving (load balancing, auto-scaling, batch scheduling, global KV cache)
• Optimize latency and throughput of model inference under real production workloads
• Build and operate high-concurrency serving systems with excellent tail latency and high reliability at the scale required to serve millions of users
• Benchmark, fine-tune, and accelerate inference engines (including low-level GPU kernel work and code generation)
• Develop custom tools to trace, replay, and fix issues across the full stack — from orchestration down to GPU kernels

What they're looking for

• Bachelor’s degree in computer science, computer engineering, or science discipline
• Hands-on low-level system programming experience in C/C++ or Rust
• Experience with large-scale, high-concurrent production serving
• Experience with GPU inference engines (e.g., vLLM, SGLang, Triton, TensorRT-LLM)
• Strong background in system optimizations: batching, caching, load balancing, parallelism

Tech stack

• C++
• Rust
• vLLM
• Triton