For Voice Actors, the Race Against AI Has Already Begun

Last year, Hulu won the rights to “Vanda,” a Portuguese crime drama series distributed by Legendary. Hulu viewers in the U.S. will automatically hear the English dub play over a cast led by Gabriela Barros, who plays a hairstylist who turns to robbing banks after the 2008 recession.

But after the main credits roll, the usual voice dubbing credits show a text card that reveals it was done not by human voice actors, but by the company Deepdub AI.

Artificial intelligence has taken root in the recording booth — and there’s no going back. The technology is already taking jobs away from voice actors in the U.S., and the pressure is even greater overseas (especially in the video game industry) for productions with lower budgets to turn to AI dubbing, industry experts told TheWrap.

“For a lot of projects, they’re not looking for top quality dubbing,” Keith Arem, a voice recording producer and head of video game voiceover company PCB Productions, told TheWrap. “They’re looking for ‘just good enough.’”

That could have sweeping implications for a part of the Hollywood production machine that employs hundreds of thousands of actors and technicians around the world. Deepdub AI says it has already provided its services for over 1,000 hours worth of streaming content.

“Our organization has already spoken directly to at least a dozen voice actors who say they’ve lost a potential job to AI dubbing,” Tim Friedlander, president and co-founder of the National Association of Voice Actors (NAVA), told TheWrap. “It’s hard to track because like with ‘Vanda,’ we don’t know which projects get dubbed by AI until it gets released and the credits have an AI company instead of a dubbing cast.”

Efficiences and Deepdub

On Deepdub AI’s website, the Israeli company promises efficiencies and cost savings that are at the heart of what’s driving Hollywood toward using the emerging technology: 70% faster recording than with human voice actors, and up to 50% lower costs, while still offering royalty fees to voice actors who lend their voices to the software.

Deepdub did not respond to TheWrap’s request for comment.

Friedlander and his team at NAVA, along with a wide range of voice actors, producers and companies, are in a race to develop a new normal in which artificial intelligence is incorporated into the recording process while keeping the human element at its core. But some voice actors see even using AI in that manner as a bridge too far.

Protecting actors and writers from the unauthorized use of AI was a central issue in the negotiations between WGA and SAG-AFTRA and the studios in their respective strikes last year. While protections around consent and compensation in SAG-AFTRA’s TV/Theatrical contract were focused primarily on live-action actors and productions, the TV animation contract negotiated this past month by the union provides more specific protections for voice actors in the guild.

Along with requiring consent and compensation for any digital replica of an actor’s voice, SAG-AFTRA also negotiated the elimination of the requirement that the digital replica recognizably sound like the actor’s natural voice to be eligible for such protections. SAG-AFTRA members would also be eligible for residuals if AI is used to alter their voice into another language, and cannot have their names used to prompt a generative AI system to create their voice without their consent.

While these protections are on their way for union voice action jobs, 80% of such jobs are non-union, Friedlander estimated.

NAVA’s push to ensure a human element in dubbing was seen when SAG-AFTRA announced a deal in January with AI voice company Replica Studios that would require the company to have the consent of any actor whose voice it replicates with AI and to compensate them for that use. The union said the deal was “approved by affected members of the union’s voiceover performer community.” A backlash on social media from voice actors ensued.

“How has this agreement passed without notice or vote? Why can’t the actual actor be used for the video game?” tweeted “Pokemon” voice actor Veronica Taylor. “Every job brings a unique opportunity for an actor to … act. Encouraging/allowing AI replacement is a slippery slope downward.”

While Friedlander and NAVA are fighting to keep open as many opportunities for voice actors to perform as possible, he said he supports SAG-AFTRA’s agreement as it set a precedent in the fight for an industry standard around AI.

“It should not be taken lightly that an AI company has signed an agreement with a union that will now hold it accountable,” he said. “That is what we need throughout the industry.”

Artists and unions are playing catch-up

Friedlander believes that artists are “10 years late on this technology” which was developed by tech companies long before SAG-AFTRA was able to provide any serious input. “But we still can and must build a system of consent, control and compensation to protect artists, because this software was built from data sets taken from artists’ work without their knowledge,” he said.

To that end, NAVA has engaged in an educational campaign to help voice actors both union and non-union understand AI technology and what to look for in contracts for potential jobs.

The organization warns that phrases like “simulation” “machine learning,” and “digital double” are red flags that should be brought to SAG-AFTRA reps — or an agent in a non-union case — to ensure that actors aren’t signing away their right to not allow their voice to be replicated by AI without their knowledge. NAVA also created an AI rider that actors can have added to a contract to maintain control of their performances.

“Until we have firm federal legislation to protect actors’ rights to their work, we need to get everyone in the industry informed on how to protect themselves,” Friedlander said.

SAG-AFTRA is one of multiple entertainment unions lobbying for the NO FAKES Act, a Senate bill introduced by Democrats Chris Coons and Amy Klobuchar with Republicans Thom Tillis and Marsha Blackburn which would hold individuals and companies liable if they created a digital replica of a person’s voice or likeness without their consent. A similar bill called the No AI Fraud Act was introduced in the House by Republican Maria Elvira Salazar and Democrat Madeleine Dean.

Using AI for local productions

While the effort to protect performers is underway, the wheels are already in motion when it comes to AI taking jobs from voice actors. This is particularly the case when it comes to localization, as projects like “Vanda” are turning to AI for cheaper, faster ways to dub productions into overseas languages.

Keith Arem, a voice recording producer and head of video game voiceover company PCB Productions, estimated that around a quarter of the overseas clients he works with on video game localization have told him that they expect to lose projects with game companies to AI automation.

While major Hollywood studios are subject to more scrutiny from unions, overseas productions with lower budgets are feeling pressure to turn to AI dubbing to keep costs down, Arem told TheWrap.

“Streaming and video games have created a huge demand for localization, but many of those projects don’t have the budget to really invest in dubbing that does justice to the original performances,” Arem said.

Now, AI dubbing companies like Deepdub are offering projects with shoestring budgets a higher standard of “just good enough” through automation. Deepdub touts on its website the Italian dub of “Forensic Files” and English dubs of the Danish crime series “Follow the Money” and the French police procedural “Spiral.”

Finding a middle ground

One company is trying to offer an alternative for localization that doesn’t take humans out of the recording booth. Flawless AI, co-founded by DGA-member director/producer Scott Mann, offers producers a suite of AI software that streamlines the dubbing process by altering the lip movements of actors in recorded footage to better align with recording voice tracks, whether those tracks are done by the actors onscreen in re-recording or by voice actors in other languages.

The important difference is that Flawless’ software doesn’t generate a voice. Mann believes that his software can solve both an old and a new issue: the old one being poor dubbing where the voices are out-of-sync with actors’ lip movements, and a new one in which voice actors are pushed out of the recording process entirely.

“Our experience is that no AI program can match the nuance that a voice actor can bring to a performance, but because dialogue can be difficult to translate into certain languages, either the lines have to be altered in a way that can harm the artistic intent or they end up out-of-sync with what is being seen onscreen,” Mann explained.

Using AI to better sync lip movements with dialogue “is an ethical and efficient use of technology that can keep a human performance at the center,” he said. “It can provide localization that delivers dubbing with equal value to the original language track, and that is only going to increase spending and interest in international shows and films,” Mann added.

Mann said Flawless is not competing with other companies in selling this kind of AI technology. Fewer AI startups are trying to preserve human actors than those offering full automation. And there’s plenty of other areas of voiceover work beyond localization, such as commercials and videos used by government and businesses, that are prime targets for automation.

Where will it end up?

How much farther will AI encroach upon movies, TV and video games? The voiceover actors who spoke to TheWrap agreed that it is difficult to know just how rapidly the biggest companies in entertainment will adopt AI for dubbing and whether they will take the path of full automation or the alternatives offered by companies like Flawless.

“In my field of work with video games, the sheer number of lines that we’re producing is tenfold fold compared to film and TV,” Arem said. “There’s all this dialogue that can be produced for some games for non-player characters, and while we believe what can be done by a human should be done by a human, the budgets for some of these games don’t cover all the actors needed to play them.”

Fans of animated shows, both Japanese and Western, have developed a deep appreciation for voice actors. So they would likely protest if it was revealed that an anime series was handing off its English dub to software, said Todd Haberkorn, a veteran anime and video game voice actor whose work includes the English dub of “Soul Eater” and the video game “Destiny 2.”

“There’s going to have to be a compromise between actors and studios on AI, but customers can vote with their wallets,” he said. “The unions did an incredible job during the strikes with their messaging to the public, and we will have to do the same if AI gets worse so that people know what is behind the shows they are watching.”

Not everyone shares that optimism. Ezra Weisz, a voice actor and director whose credits include multiple seasons of “Power Rangers” and anime series like “Outlaw Star” and “Code Geass,” believes that all it takes is one well-received title for AI in voice acting to be normalized.

“People will push back against AI for a couple years, but if enough fun video games that use AI voices are released, a lot of people are still going to want to play them,” Weisz said.

Weisz said he favors SAG-AFTRA making more deals to normalize a structure of consent and compensation around AI replicas of actors’ voices. And actors, he believes, need to figure out how to quickly adapt their careers to whatever changes AI brings.

“As an ADR director, I’ve had engineers show me this technology and how excited they are to use it,” he said. “AI is here to stay. We have to figure out how we live with this new neighbor in our neighborhood, even if they’re going to tear down a historic home and build a McMansion.”

The post For Voice Actors, the Race Against AI Has Already Begun appeared first on TheWrap.