Resemble Ai Release Notes

Rapid Voice Cloning

We’re excited to announce the launch of our groundbreaking new feature: Rapid Voice Cloning. This innovative technology allows you to create high-quality voice clones faster and easier than ever before, unlocking new possibilities for your voice-enabled projects.

Rapid Voice Cloning is a game-changing feature that streamlines the voice cloning process by enabling the creation of voice clones from remarkably short audio samples. With Rapid Voice Cloning, you can now generate a voice clone using as little as 10 seconds of audio data.

This revolutionary approach to voice cloning makes the technology more accessible and efficient than traditional methods that require lengthy audio recordings. By reducing the barrier to entry, Rapid Voice Cloning empowers more users to leverage the power of custom voices in their applications.

How Does Rapid Voice Cloning Work?

Under the hood, Rapid Voice Cloning employs cutting-edge algorithms to analyze and replicate the unique characteristics of a source voice from just a brief audio sample. Our advanced machine learning models are able to capture the essence of a voice and generate a high-fidelity clone in a matter of moments.

The process is designed with simplicity in mind. All you need to do is provide a clear audio sample of the target voice, lasting anywhere from 10 seconds to 1 minute. Our intelligent system takes care of the rest, delivering a fully-functional voice clone that’s immediately ready to use.

Rapid Voice Cloning offers a range of compelling benefits:

• Speed: Generate voice clones in seconds, enabling rapid iteration and deployment in your projects.
• Accessibility: With dramatically reduced audio requirements, voice cloning is now within reach for a wider user base.
• Seamless Integration: Rapid Voice Clones work flawlessly with our Web UI and API, allowing for frictionless use across your applications.
• Efficiency: Save valuable time and resources by eliminating the need to record and process lengthy voice samples.

Comparison to other State of the Art Models

We compare our results to other state of the art AI Voice cloning models including Microsoft’s VALL-E and XTTS-v2. Like VALL-E and XTTS-v2, our model has never seen any of the Voice Prompt speakers during training.

Breakthrough in Accent Retention

Resemble AI’s Rapid Voice Cloning technology is a true game-changer when it comes to capturing and reproducing accents with unparalleled accuracy. While other state-of-the-art models often struggle to replicate the nuances and subtleties of different accents, Resemble AI’s advanced machine learning algorithms excel in this area. By analyzing and learning from just a 10-second voice sample, our Rapid Voice Cloning can create an AI-generated voice that faithfully mimics the unique intonations, pronunciations, and cadences of the original speaker’s accent.

Applications of Rapid Voice Cloning

Rapid Voice Cloning is a versatile technology with a wide array of potential applications across industries. Its ability to create high-quality voice clones from short audio samples opens up exciting possibilities for content creation, personalization, accessibility, and more. Let’s dive deeper into some of the key use cases.

Content Creation

Rapid Voice Cloning can be a game-changer for content creators, allowing them to efficiently generate voiceovers, narration, and dialogue. Whether you’re producing podcasts, videos, audiobooks, or e-learning materials, this technology enables you to quickly create compelling voice content without the need for extensive recording sessions.

Imagine being able to clone the voices of multiple characters for an animated series, or generate localized versions of a product demo in various languages – all with just a few clicks. Rapid Voice Cloning makes these scenarios possible, empowering content creators to scale their productions and explore new creative avenues.

Personalized User Experiences

In today’s digital landscape, personalization is key to engaging and retaining users. Rapid Voice Cloning allows businesses to take this to the next level by integrating custom voices into their applications and services.

For example, a fitness app could use Rapid Voice Cloning to create a personalized AI coach that speaks to each user in a familiar voice, providing encouragement and guidance. Similarly, a virtual assistant could adapt its voice to match the user’s preferences, creating a more intimate and tailored interaction.

By leveraging the power of voice cloning, businesses can create more immersive and memorable user experiences that foster deeper connections with their audience.

Accessibility and Assistive Technologies

Rapid Voice Cloning has the potential to make a significant impact in the realm of accessibility and assistive technologies. For individuals with speech impairments or those unable to speak, voice cloning can provide a means of communication that preserves their unique vocal identity.

By cloning a person’s voice from a short sample recorded before the onset of their condition, assistive devices can generate speech that sounds authentically like the individual. This can be incredibly empowering, allowing people to express themselves in a way that feels true to who they are.

Moreover, Rapid Voice Cloning can be used to create more engaging and inclusive educational materials. By cloning the voices of educators or subject matter experts, learning content can be made more accessible to students with diverse needs, such as those with visual impairments or language barriers.

Prototyping and Product Development

Rapid Voice Cloning can significantly accelerate the prototyping and development process for voice-enabled products and features. By quickly generating voice clones, teams can test and iterate on their ideas faster than ever before.

For instance, a company developing a voice-controlled smart home device can use Rapid Voice Cloning to experiment with different voice personas and interaction styles, gathering user feedback early in the design process. This agile approach allows for more informed decision-making and can ultimately lead to better, more user-centric products.

Creative Industries

The creative industries are another domain where Rapid Voice Cloning can make a significant impact. In film and television production, voice cloning can be used to create dubbed versions of content for international markets, or to generate dialogue for computer-generated characters.

In the music industry, Rapid Voice Cloning opens up new possibilities for artistic expression. Musicians can experiment with incorporating cloned voices into their compositions, creating unique vocal textures and harmonies. DJs and producers can use voice cloning to generate bespoke vocal samples and hooks, adding a new dimension to their tracks.

Ethics and Responsible Use

At Resemble AI, we’re committed to developing voice cloning technology that is not only powerful and accessible but also grounded in strong ethical principles. We recognize the potential for misuse and are proactively taking steps to ensure that Rapid Voice Cloning is used responsibly and transparently.

Consent and Authorization

One of the cornerstones of ethical voice cloning is consent. We require that all users obtain explicit permission from the individuals whose voices they intend to clone. This consent must be freely given and informed, with the voice owner fully understanding how their voice will be used.

To facilitate this, we’ve built consent and authorization mechanisms directly into our platform. When creating a Rapid Voice Clone, users must confirm that they have obtained the necessary permissions and agree to use the voice clone in accordance with our terms of service.

Join Us in Responsible Innovation

To our users and the wider community, we invite you to join us in this commitment to ethical voice cloning. By using Rapid Voice Cloning responsibly, transparently, and with respect for voice owners’ rights, you can help shape a future where this transformative technology is used to create positive change.

Together, we can unlock the incredible potential of voice cloning while ensuring that it remains a tool for good. Let’s build a future where every voice matters and every innovation is grounded in ethics.

Original source Report a problem

Rapid Voice Cloning

Resemble AI is launching Rapid Voice Cloning, a new feature of its platform that significantly expedites the process of generating voice clones. The company works in the elusive AI voice category focused on enterprise users.

Available today, Rapid Voice Cloning can duplicate voices from relatively short datasets and produce an output in just about a minute. The move, Resemble says, marks a significant development and will make voice cloning technology more accessible, empowering more users to create custom voices for their applications. The company believes it will make an impact across fields such as content creation, personalization and accessibility.

Resemble published multiple voice clone samples showcasing the prowess of the new technology. VentureBeat also tested the feature to see how it really works.

How does the new AI voice Cloning feature work?

When using Resemble’s web platform, users can create a digital replica of their voice by uploading an audio sample or recording a series of sentences. The company has been offering this feature for a while, but the process took time. Users had to record around 25 sentences or upload at least three minutes of voice content to set up the system, which would then take another hour or so to provide a clone.

Now, with the launch of Rapid Voice Cloning, users can get started with the technology more easily. All they have to do is give a clear audio sample of the target voice, lasting anywhere from 10 seconds to 1 minute. The company’s model under the hood instantly captures all the parameters, including accents, from the sample and gives the result for downstream use cases in a minute.

“While other state-of-the-art models often struggle to replicate the nuances and subtleties of different accents, Resemble AI’s advanced machine learning algorithms excel in this area. By analyzing and learning from just a 10-second voice sample, our Rapid Voice Cloning can create an AI-generated voice that faithfully mimics the unique intonations, pronunciations, and cadences of the original speaker’s accent,” the company noted in a blog post announcing the feature.

The company published a bunch of samples comparing its offering with Microsoft’s VALL-E and XTTS-v2 voice cloning models, complete with the input voice sample and the text used for the clone. The results were pretty impressive. However, when we created a free test account to see how the tech works for real, there were some clear gaps.

In our tests, the system mandated recording at least three long sentences, with no option to record a smaller 10-second sample. The processing was swift but it couldn’t recognize the speaker’s Indian accent and took the input by default as a voice sample in American English. This affected the accent of the output voice. However, it is expected to be fixed, since according to the company Rapid Voice Cloning will support most English accents.

Notably, the company will continue to provide the original cloning feature under the name of professional voice cloning. This option, with lengthy input requirements, will take time but support all English accents with support for text-to-speech and speech-to-speech use cases. Rapid cloning will only support text-to-speech generation.

Use across different categories

With Rapid Voice Cloning’s speed and dramatically reduced sample requirements, Resemble AI expects to see more users using the technology with faster iterations and deployments. The biggest adoption is expected from content creators who may use the tech to generate voiceovers, dubbing, narration and dialogue for their podcasts, videos, audiobooks or e-learning materials. The company also says businesses can create enhanced accessibility and personalization experiences with the technology.

“For example, a fitness app could use Rapid Voice Cloning to create a personalized AI coach that speaks to each user in a familiar voice, providing encouragement and guidance. Similarly, a virtual assistant could adapt its voice to match the user’s preferences, creating a more intimate and tailored interaction,” the company stated.

While it remains to be seen how the tech gets adopted, it is important to note that Resemble is not the only player cutting down the time to generate voice clones. ElevenLabs, another major player in the category, offers a feature called Instant Voice Cloning that needs at least a minute of clear audio to generate a clone almost instantly. Like Resemble, ElevenLabs also offers a professional version of the tool, which covers more languages and accents.

As of now, Resemble AI allows users to create one free voice clone. For more, users would have to take up a paid plan from the company, which starts from $29/month and goes up to $499/month. There is also the option of a pay-as-you-go personal plan or a bigger enterprise plan with custom pricing.

Original source Report a problem

Resemble Enhance

While many AI companies race to find ways to use the technology to enhance or even create video, Resemble AI is focused on audio fidelity. The startup—which also offers an AI voice generator for businesses to use to create realistic human–like voiceovers—has launched 'Resemble Enhance,' an open-source tool designed to significantly upgrade the quality of historical audio.

The new service can take a distorted, fuzzy recording of a long-lost historical speech and then apply AI to make it sound like it was recorded or broadcast yesterday.

The Canadian company says Resemble Enhance is distinguished by its dual-module approach, combining a sophisticated denoiser—which removes static background hums and hisses—and an AI-powered speech enhancer. This combination not only removes unwanted noise but also enriches the overall quality of the audio.

Even though there are other audio restoration products on the market, Resemble’s combination of techniques could be a meaningful differentiator.

How does it work?

• The Resemble AI denoiser uses UNet, an AI model that helps to separate the different types of sounds that appear on a recording. It excels at filtering out unwanted noise from audio tracks, leaving just the speech as the focus.
• Once UNet does its job, the enhancer module kicks in, extending audio bandwidth and correcting distortions. This dual functionality, the company says, ensures that the final output is not just noise-free but also possesses the richness of contemporary recordings.

As an open-source tool, Resemble Enhance is accessible at no cost, a compelling option in the traditionally expensive market of media restoration services. The primary beneficiaries of Resemble Enhance are industries reliant on clear audio quality, such as podcasting, entertainment, and education. Additionally, this tool offers a new lease on life to historical recordings, potentially providing clearer insights into the past.

The tool's release comes when the demand for high-quality digital content is at an all-time high. Meanwhile, the open-source nature of Resemble Enhance positions it as a potentially disruptive force in a market currently dominated by high-cost proprietary solutions.

The convergence of AI in audio and video enhancement will likely pave the way for more comprehensive media restoration solutions.

By combining this tool with other video enhancers that use generative AI or other models to upscale and enhance images and faces—like GPEN or the well-known GFPGan—users can now achieve professional results with their own computers for a minimum investment.

To experiment with Resemble Enhance, users can visit the official Resemble AI website or download their models from the project’s official Github page.

Edited by Ryan Ozawa.

Original source Report a problem

Generative AI in Film & TV (Special Report)

Over the last year, the entertainment industry narrative around generative AI has been intensely fraught. Now that the writers and actors strikes are over, Hollywood needs to consider exactly how studios and creatives will — and perhaps should — use gen AI models and software tools in the many varied creative processes involved in producing film or TV.

As the industry grapples with the game-changing technology in the coming months, Variety Intelligence Platform’s special report “Generative AI in Film & TV” examines the capabilities and limitations of gen AI models and emerging software through the lens of their present, near-term and future uses in film and television creation.

Generative AI offers an expansive range of possibilities at stages throughout the production value chain. In particular, we analyze the value and usability of this expanding, diverse and powerful set of AI models and tools for tasks in screenwriting, VFX, previsualization, content localization and sound editing.

And it’s already beginning to disrupt traditional methods, with generative AI tools currently used to automate some creative tasks. Still, its impact stands to be positive, as it eliminates rote work, speeds project timelines and allows productions to pursue previously impossible creative paths prohibited by constraints on cost, time and even physical reality. At the same time, its use promises to reduce the need for certain processes and as many workers to achieve the same level of output.

Since the first major public release of generative AI user applications last year, the exponential pace of research and development in the AI community has further improved the capabilities of these systems.

Alongside a rising tide of major tech companies variously developing, open-sourcing and productizing generative AI, several startups have emerged, offering best-in-class software and tools catering to entertainment production and post-production uses.

Research for this report partly draws from over 20 background conversations primarily conducted in August through November 2023 with media & entertainment (M&E) advisers; leaders and founders at generative AI startups; and independent filmmakers involving AI tools and techniques in their processes.

Company participants included Runway, Synthesia, Metaphysic, Wonder Dynamics, Digital Domain, Monsters Aliens Robots Zombies (MARZ), ElevenLabs, Deepdub, Resemble AI, DeepBrain AI, Luma AI and Fable Studio.

For the purposes of this report, our focus is the impact of generative AI in film and TV production. For additional research, we point back to our October edition, “Generative AI & Entertainment: Part 2,” examining the legal and labor risks of gen AI for the film and TV industry as well as potential mitigations. Our April 2023 report, “Generative AI & Entertainment,” presented the full scope of use cases across entertainment domains, including film, TV, music and gaming.

Read on to learn about:

• A full-page chart on generative AI capabilities, software and production uses

• How the tech will be used in screenwriting, VFX, content localization, more

• Ways advances in AI software development will impact content creation

Original source Report a problem

With the launch of its new speech-to-speech feature, Resemble AI (https://www.resemble.ai/) is bringing natural-sounding AI voices to millions of developers and creators and continuing to expand the creative possibilities for human voices and beloved characters.

TORONTO and SAN FRANCISCO, Sept. 8, 2022 /PRNewswire-PRWeb/ -- Resemble AI today announced the launch of its speech-to-speech feature, to capture the unique style of human voices at scale and bring natural-sounding AI voices to millions of developers and creators across gaming, entertainment, e-learning, and more. Resemble AI's generative audio continues to expand the creative possibilities for human voices and beloved characters.

Now AI voices can perform a wide range of emotions, speaking styles or even singing using non-speech vocalizations. If the input audio is spoken in a different language, the resulting target voice will be able to speak in that language. To see speech-to-speech in action, watch this video.

“Resemble AIʼs mission is to make interactions with digital products as human and natural as possible,” says Resemble AI founder & CEO Zohaib Ahmed. “Now, developers and storytellers can create content in any language and any voice, without the expense or travel associated with recording studios.”

"We're really happy with the quality of voices we're able to develop for 'Animals Anonymous' using Resemble AI," says Fika Agency co-founder Adam Altman. "Now our entire team can record new episodes that feel consistent with the same voices our listeners are used to hearing."

The team at Resemble AI refined speech-to-speech when it used 3 minutes and 12 seconds of Andy Warhol's original voice recordings from the 1970s and 80s to produce synthetic voice narration for the Emmy-nominated and Dorian Award-winning Netflix docu-series, The Andy Warhol Diaries. The team made adjustments for emotion and pitch to the AI output of Andy Warhol's voice, and added human-like imperfections using reference audio clips of another speaker, as seen in this video about how Resemble AI's Style Transfer works.

"Resemble AIʼs mission is to make interactions with digital products as human and natural as possible," says Resemble AI founder and CEO Zohaib Ahmed. "We've dramatically accelerated and simplified the process of creating human-like AI voices by building hyper realistic synthetic voices that can match or expand the reach of voice actors. This means that developers, creators and storytellers can create content in any language, and with any voice, without the need for expense or travel associated with recording studios."

Starting today, Resemble AI customers on the Basic, Pro (new) and Enterprise plans will see a new option for speech-to-speech when creating a sentence in a clip within a project. Instead of using text as input, they can provide a spoken sentence by uploading a pre-recorded audio file or record directly through the interface. This enables high quality AI voices and allows the target voice to speak in a different language than the original voice–which must be the same voice that records a consent line at the start of any project.

Consent is a requirement for all Resemble AI projects and transparency is maintained throughout the process, as seen in this video about how Resemble AI voice cloning works. To learn more about how Resemble AI maintains ethical standards across the industry, visit https://www.resemble.ai/ethics/.

About Resemble AI

Resemble AI's technology is being used by some of the largest media companies in the world to create content that was previously impossible. Whether it's transferring a voice into dozens of other languages, creating thousands of dynamic personalized messages from celebrities, or creating unique real-time conversational agents, Resemble AI is changing how content is created.

With Resemble AI, creating engaging and high-quality voice content is now easier than ever, enabling content creators to add a whole new level of authenticity to their work, and will add a new level of immersion for the audience. Learn more at https://www.resemble.ai/our-solution/.

Media Contact

Amy Jackson, TaleSplash for Resemble AI, 1 4156092435, [email protected]

SOURCE TaleSplash for Resemble AI

Original source Report a problem

More than 65,000 Users have Cloned 42,000 Voices

Toronto, Canada, Oct. 15, 2020 /PRNewswire-PRWeb/ -- Resemble.ai, a leader in generative deep learning voice technology today announced that it has created Localize -- voice AI technology to localize speech for the first time. Until now, entertainment companies, ad agencies, call centers and companies that needed to translate voices would have to use a different voice in each language. With Resemble.ai, voices can be cloned into any language - so George Clooney sounds like George Clooney, even when a movie has been dubbed into another language.

Resemble.ai clones voices at scale in seconds, as opposed to weeks. It has democratized this previously laborious, expensive process and cloned 42,000 voices for 65,000 users, including two of the largest global telecoms, two of the largest consulting companies, a top global broadcasting co, two of the largest entertainment conglomerates, one of the largest toy makers, and the leader in airport communications systems.

Localize fundamentally changes the way we think about speech. Much like text, video, and other mediums that have gone across borders, speech remains stuck to a single language. With deep learning and custom synthetic AI voices, we're breaking that barrier down.

Localize is able to keep any character's voice consistent across video games, movies, call centers, company videos, and more as they are translated to and from languages including French, German, Dutch, Italian and Spanish. A majority of the world speaks one of these six languages and the company also has near-term plans to introduce Localize for Korean, Japanese and Mandarin.

Normally, voice talent translation takes an average of two months and can tally hundreds of thousands of dollars. With Resemble.ai, it's accomplished in a week with maximum creative flexibility and efficiency. For entertainment companies, dubbing a script is logistically challenging and the fidelity of the production is oftentimes lost in translation. Resemble.ai offers a more attractive solution in a fraction of the time using the same talent whom production houses have already paid for.

Resemble Co-founder and CEO Zohaib Ahmed said, "Localize fundamentally changes the way we think about speech. Much like text, video, and other mediums that have gone across borders, speech remains stuck to a single language. With deep learning and custom synthetic AI voices, we're breaking that barrier down."

"It's hard to overstate how important audio has become in recent years -- or just how much bigger it's going to get in an AirPods-first world," said Peter Rojas, Partner at Betaworks Ventures. "Synthetic voice is going to be key to all this by transforming how audio is created. Demand for localized and translated spoken word content, whether it's in the form of podcasts or audiobooks, is exploding, and AI-based tools like Localize are the way to satisfy that demand."

How it works:

An audio recording is translated so that it accurately reflects not only the words and their specific meanings, but also colloquialisms and grammatical structures that are particular to that region and language. The service allows anyone to translate and hear it simultaneously - so you can find out immediately "how does Will Smith sound when he says this sentence?" Other examples include:

• Gaming - Resemble.ai makes it possible for a character's voice to sound the same in different languages. It retains the characters' voices for a different language.
• Influencers - YouTubers and TikTok stars can reach non-English speaking audiences and vice versa
• Call Centers, which serve scores of markets, can create one universal conversational chatbot with similar voices in numerous languages. Previously they had to hire talent internationally.
• Movies can expand internationally faster with the voices they originally intended because dubbing isn't needed.

About Resemble AI

Resemble AI, an artificial intelligence company that creates synthetic voices, is creating synthetic speech that is focused on human emotion. With state of the art Artificial Intelligence, its products Resemble Clone and Resemble Localize are designed to help entertainment, gaming, call center and other industry creatives synthetically produce, clone and dub high-quality voices. Founded in 2019, Resemble AI is headquartered in Toronto, Ontario and has secured $2M in funding from Craft Ventures, firstminute Capital, AET Fund, and Betaworks. To learn more, please visit https://www.resemble.ai.

SOURCE Resemble AI

Original source Report a problem