Software Engineer - Training Infrastructure

Job summary

Responsibilities

As a Software Engineer on the Training Infrastructure team, you'll architect and lead development of our training platform, supporting top tier research engineers and model developers.

• Design and architect scalable infrastructure systems for our ML training platform (e.g. scheduling, storage, and networking)
• Partner closely with developers and research engineers to translate complex training requirements into technical solutions
• Design and architect a global training scheduler
• Design and architect reinforcement learning systems and continuous learning pipelines
• Drive long-term improvements to increase reliability of systems and velocity of development

What they're looking for

• Bachelor’s degree or higher in Computer Science or related field
• 5+ years of experience
• Proficiency in Go, with
• Deep expertise with Kubernetes in production environments
• Advanced understanding of distributed systems concepts and performance tuning

Tech stack

• Spark
• Go
• Kubernetes
• Python
• AWS
• GCP
• Airflow
• PyTorch