Member of Product Staff, AI/ML (Evals)

Job summary

Responsibilities

Abridge's has multiple products - the core product is ambient clinical notes, but we’ve expanded surfaces: billing, clinical decision support, orders, nursing, etc. Measuring quality reliably and iterating fast without breaking clinician trust sit at the core of every single product’s success.

• Drive product strategy and execution for the evals platform. Own the roadmap across the eval lifecycle and own outcomes against it.
• Build the shared measurement infrastructure. Help build the systems that let any pod define quality, run experiments, compare models, and watch production.
• Make model selection fast and routine. Give teams a repeatable way to evaluate a new frontier model within days of release, and a defensible framework for when a post-trained…
• Own the operating model across pods. Define the eval gates from early build through GA and steady-state monitoring, which gates are hard versus advisory, and who owns…
• Cross functional execution. Work with Data Engineering on the de-identification pipeline, Data Science on bootstrapping judge quality with less human annotation, Clinical Science…

What they're looking for

• 5+ years of product management experience with significant ownership of ML powered products or platform systems.
• Deep understanding of how to measure and improve model quality, including evaluation frameworks, annotation pipelines, and benchmark design.
• Strong technical fluency across ML, data pipelines, and distributed systems.
• Experience working closely with ML researchers and engineers to drive impact in production.
• Ability to balance long term architectural investments with near term quality improvements.