Navigating the challenges and opportunities of synthetic voices

We recognize that generating speech that resembles people's voices has serious risks, which are especially top of mind in an election year. We are engaging with U.S. and international partners from across government, media, entertainment, education, civil society and beyond to ensure we are incorporating their feedback as we build. The partners testing Voice Engine today have agreed to our usage policies⁠, which prohibit the impersonation of another individual or organization without consent or legal right. In addition, our terms with these partners require explicit and informed consent from the original speaker and we don’t allow developers to build ways for individual users to create their own voices. Partners must also clearly disclose to their audience that the voices they're hearing are AI-generated. Finally, we have implemented a set of safety measures, including watermarking to trace the origin of any audio generated by Voice Engine, as well as proactive monitoring of how it's being used. We believe that any broad deployment of synthetic voice technology should be accompanied by voice authentication experiences that verify that the original speaker is knowingly adding their voice to the service and a no-go voice list that detects and prevents the creation of voices that are too similar to prominent figures.

Voice Engine is a continuation of our commitment to understand the technical frontier and openly share what is becoming possible with AI. In line with our approach to AI safety⁠ and our voluntary commitments⁠, we are choosing to preview but not widely release this technology at this time. We hope this preview of Voice Engine both underscores its potential and also motivates the need to bolster societal resilience against the challenges brought by ever more convincing generative models. Specifically, we encourage steps like:

• Phasing out voice based authentication as a security measure for accessing bank accounts and other sensitive information
• Exploring policies to protect the use of individuals' voices in AI
• Educating the public in understanding the capabilities and limitations of AI technologies, including the possibility of deceptive AI content
• Accelerating the development and adoption of techniques for tracking the origin of audiovisual content, so it's always clear when you're interacting with a real person or with an AI

It's important that people around the world understand where this technology is headed, whether we ultimately deploy it widely ourselves or not. We look forward to continuing to engage in conversations around the challenges and opportunities of synthetic voices with policymakers, researchers, developers and creatives.