Software Engineer, Inference - Multi Modal

Job summary

Responsibilities

We’re looking for a software engineer to help us serve OpenAI’s multimodal models at scale. You’ll be part of a small team responsible for building reliable, high-performance infrastructure for serving real-time audio, image, and other MM workloads in production.

• Design and implement inference infrastructure for large-scale multimodal models.
• Optimize systems for high-throughput, low-latency delivery of image and audio inputs and outputs.
• Enable experimental research workflows to transition into reliable production services.
• Collaborate closely with researchers, infra teams, and product engineers to deploy state-of-the-art capabilities.
• Contribute to system-level improvements including GPU utilization, tensor parallelism, and hardware abstraction layers.

Tech stack

• vLLM