Member of Technical Staff, Data & Evals

Job summary

Responsibilities

As a Member of Technical Staff, Data & Evals in Harvey Research, you’ll build the data and measurement systems that power our post-training, benchmarking, and agent capability efforts.

• Build synthetic data pipelines that generate, filter, and curate training and evaluation samples for complex legal reasoning and agentic workflows.
• Design evaluations and verifiers that reflect real legal work, calibrate them against expert judgment, and investigate where they fail.
• Build and maintain capability benchmarks like our Legal Agent Benchmark (LAB), along with product-focused evals that track how models perform on real customer workflows.
• Work with legal domain experts to define quality metrics, rubrics, and other grading mechanisms for training and evaluation.
• Own training and evaluation data quality, including provenance, coverage, and safeguards against contamination.

What they're looking for

• Experience building data pipelines, ML evaluation systems, or experimentation tooling, with strong Python skills and ownership of production-quality code.
• A rigorous approach to measurement, including statistical reasoning, experimental design, and understanding when a metric can mislead.
• The ability to translate ambiguous goals or expert judgment into structured datasets, rubrics, and measurable criteria.
• Experience with synthetic data, LLM evaluations, automated raters, or preference data is a plus. Strong data and measurement engineering experience close to ML model development…

Tech stack

• Python