Staff Software Engineer, AI Platform

Job summary

Responsibilities

Harvey’s products all depend on a shared AI foundation: the model layer and agent infrastructure that determine the quality of work our agents deliver. Legal is one of the hardest domains for AI: documents run to hundreds of pages, matters can span millions of files, and there is zero margin for…

• Design and build abstractions and platform-level systems that improve all of Harvey’s agentic products.
• Own infrastructure for model integration, routing, and evaluation that helps Harvey choose and deploy the right foundation model for any given context.
• Build evaluation frameworks and tooling that let every team across Harvey iterate on AI quality effectively.
• Partner closely with product engineering teams, PMs, and design to launch cutting-edge AI products.
• Evaluate, prototype, and integrate the latest advancements in AI and agentic systems as they emerge.

What they're looking for

• 10+ years of experience building backend systems, with at least 1+ year focused on AI/ML engineering and a track record of technical leadership across teams.
• Experience building and shipping multi-model or multi-provider AI systems in production.
• Familiarity with context management, session state, or memory systems in AI or distributed systems. You’ve thought about what the model sees and why it matters.
• A track record of building internal platforms, SDKs, or shared infrastructure that other engineering teams actually adopted - and an understanding of why developer experience…
• Strong judgment about abstractions. Opinionated about good design but pragmatic about shipping incrementally.