Member of Technical Staff - Inference Runtime

Job summary

Responsibilities

Modal’s Inference Runtime team owns the container runtime stack used to run inference and training workloads across our fleet. We work at the boundary of Linux, containers, filesystems, storage, GPU drivers, and distributed systems.

• Make container startup, checkpoint, and restore dramatically faster for large inference and training workloads
• Build multi-GPU and accelerator-aware snapshotting, including efficient handling of GPU memory and RDMA-enabled workloads
• Design zero-copy and low-copy data paths between container memory, filesystems, storage, and Modal’s runtime
• Optimize large snapshot pipelines using techniques such as parallel uploads, direct I/O, incremental snapshots, and more efficient memory handling
• Improve container image and filesystem performance across EROFS, FUSE, page caches, overlay filesystems, and remote storage

Tech stack

• Rust
• Go
• CUDA