Software Engineer, GPT Infrastructure

Job summary

Responsibilities

We are seeking a software engineer to help build the platform that qualifies and optimizes inference workloads across heterogeneous compute environments.

• Design, build, and operate APIs and control-plane services for long-running workload qualification and optimization campaigns, including scheduling, retries, checkpointing…
• Build secure partner-side execution and evaluation software that can compile, run, verify, profile, and benchmark candidate artifacts on accelerator hardware.
• Integrate model workloads, hardware profiles, compiler toolchains, runtimes, serving engines, and distributed-execution backends into a repeatable platform.
• Develop correctness and performance evaluation systems spanning output fidelity, latency, throughput, memory footprint, accelerator utilization, communication efficiency, scaling…
• Automate the generation, evaluation, and improvement of kernels, runtime configurations, parallelization strategies, and serving-stack changes.

What they're looking for

• Strong software engineering experience building distributed systems, infrastructure platforms, production services, developer platforms, or orchestration systems.
• Proficiency in one or more systems-oriented languages such as Python, C++, Go, or Rust.
• Experience designing and operating APIs, job orchestration systems, durable workflows, or large-scale backend services.
• Strong understanding of Linux, networking, storage, containers, distributed execution, and modern infrastructure architectures.
• Ability to reason about model execution and diagnose problems across software and hardware boundaries.

Tech stack

• Python
• C++
• Go
• Rust
• CUDA
• Triton
• vLLM