Engineering Manager, Research Data Platform

Job summary

Responsibilities

Anthropic's researchers generate and depend on enormous amounts of data — training runs, evaluations, RL transcripts, annotations etc... The Research Data Platform team builds the systems that make that data easy to produce, find, query, and trust.

• Work directly with researchers and the engineers supporting them to understand their workflows, identify the highest-leverage opportunities, and shape what the team builds next
• Set the technical direction for the team across our platform and our datasets
• Design and build platform components that other teams plug into — libraries, services, and interfaces such as the metrics library used by training frameworks
• Own core datasets end to end: the pipelines that produce them, the schemas that define them, and the documentation and guarantees that make researchers trust them
• Drive convergence toward canonical datasets — including the core data model for RL transcripts — that research teams standardize on

Tech stack

• Spark
• BigQuery