Software Engineer, Inference

Job summary

Responsibilities

We're hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of the systems that serve our models to real users. Our research and inference teams push the limits of model performance and serving efficiency; this role makes sure those gains reach production safely and…

• Operate and scale the production inference systems that serve live traffic, including Tinker's multi-tenant serving platform
• Own the rollout process for new models, model versions, and inference optimizations, ensuring safe, incremental deployment to production
• Build and improve observability, alerting, and capacity planning so the team can detect, diagnose, and resolve production issues quickly
• Partner with inference and research teams to productionize new serving techniques without compromising reliability
• Lead incident response for production inference issues, driving root cause analysis and durable fixes

What they're looking for

• Experience operating large-scale, latency-sensitive production systems
• Proficiency in Python and Go or another systems language
• Experience with observability, monitoring, and incident response for production services
• Strong understanding of distributed systems and how they fail at scale
• Experience running production inference for large language models or other large-scale ML systems

Tech stack

• Python
• Go