Senior Product Quality Analyst - AI Voice

Job summary

Responsibilities

The Personalization team makes deciding what to play next easier and more enjoyable for every listener. From Blend to Discover Weekly, we’re behind some of Spotify’s most-loved features. We built them by understanding the world of music and podcasts better than anyone else.

• Own the quality bar for Spotify’s AI voice models, defining what “good enough” means for different products, audiences, languages and use cases.
• Independently design evaluation rubrics and build an evaluation approach that measures qualities such as naturalness, tone, pacing, pronunciation, accent, audio quality and…
• Plan and carry out evaluation campaigns, turning qualitative and quantitative findings into clear recommendations for Voice Production, Speak and product teams.
• Establish an ongoing evaluation rhythm, including when and how we reassess model quality as our technology and products evolve.
• Benchmark Spotify’s AI voices against third-party models and competitors to understand where our experiences lead, where they differ and where there are opportunities to improve.

What they're looking for

• You have experience in product quality, evaluation, data curation or a closely related field where human judgment is used to assess model or product performance.
• You know how to independently define evaluation criteria and turn subjective qualities into clear, repeatable rubrics.
• You are comfortable designing and running both qualitative and MOS-style evaluations and synthesizing the results into actionable insights.
• You have an understanding of voice or audio quality, including qualities such as naturalness, pacing, tone, accent and pronunciation, or the ability to develop that expertise…
• You have strong judgment and can maintain a consistent quality bar while balancing different product requirements, markets and stakeholder perspectives.