Senior Software Engineer, Inference Cloud

Job summary

Responsibilities

We’re hiring a Senior Software Engineer for our Inference Cloud team. This team owns the cloud layer behind our Inference Service, including availability, latency, reliability, and global scale.

• Kubernetes Infrastructure. Build and improve production Kubernetes environments on Amazon EKS. Own improvements to workload scheduling, autoscaling, cluster upgrades, resource…
• Service Networking & Traffic Control. Develop and operate traffic management infrastructure using Envoy and Istio.
• Databases. Build database-backed services and improve the reliability and performance of their data stores. Troubleshoot query performance, connection pooling, and availability…
• Reliability & Performance. Identify bottlenecks across applications, Kubernetes, networking, and databases. Improve latency, throughput, and capacity efficiency in high-QPS…
• Infrastructure & Deployment Automation. Use Terraform, Argo CD, and Kustomize to automate infrastructure provisioning and application delivery.

What they're looking for

• 5+ years of software engineering experience, including building and operating production services or cloud infrastructure.
• Strong Kubernetes expertise, including workload lifecycle, networking, service discovery, storage, autoscaling, and troubleshooting cluster and application failures.
• Hands-on experience with Amazon EKS or a comparable managed Kubernetes service.
• Experience configuring and troubleshooting Envoy and Istio, or comparable proxy and service mesh technologies. Solid understanding of HTTP, gRPC, DNS, TLS, and mTLS.
• Experience developing services that use production databases, with practical knowledge of query optimization, indexing, connection management, and database reliability.

Tech stack

• Kubernetes
• Terraform
• Go
• AWS
• C++