Systems Generalist, GPT Infrastructure

Job summary

Responsibilities

We are seeking an experienced systems generalist who can work comfortably across the stack to help build an automated inference optimization platform. Given a workload, target hardware profile, compiler and runtime context, and a trusted verifier, the system runs durable optimization campaigns…

• Design, build, and operate durable APIs and control-plane services for multi-hour or multi-day optimization campaigns, including scheduling, retries, budgets, checkpoints…
• Build secure partner-side runner and grader software that can compile, execute, verify, and benchmark candidate artifacts on third-party accelerator hardware.
• Integrate hardware profiles, ISA and toolchain context, compilers, runtimes, and inference-serving engines into a repeatable optimization workflow.
• Turn research prototypes into reliable product surfaces with clear contracts, debuggable failure modes, reproducible outputs, and excellent developer ergonomics.
• Develop correctness and performance evaluation systems spanning latency, throughput, memory use, utilization, and cost efficiency.

What they're looking for

• 8+ years of professional software engineering experience building large-scale distributed systems, infrastructure platforms, or cloud services, or equivalent depth of experience.
• Strong programming skills in one or more of C++, Python, Go, or Rust.
• Experience designing and operating highly available backend systems, APIs, job orchestration systems, or durable workflows for production workloads.
• Strong understanding of distributed systems, Linux, networking, storage, containers, and modern cloud architectures.
• Experience debugging complex systems and using measurement, profiling, and benchmarks to guide engineering decisions.

Tech stack

• C++
• Python
• Go
• Rust
• Triton
• CUDA
• vLLM