Disney Research neural face-swapping technique can provide photorealistic, high-resolution video

Darrell Etherington

A new paper published by Disney Research in partnership with ETH Zurich describes a fully automated, neural network-based method for swapping faces in photos and videos -- the first such method that results in high-resolution, megapixel resolution final results according, to the researchers. That could make it suited for use in film and TV, where high-resolution results are key to ensuring that the final product is good enough to reliably convince viewers as to their reality.

The researchers specifically intend this tech for use in replacing an existing actor's performance with a substitute actor's face, for instance when de-aging or increasing the age of someone, or potentially when portraying an actor who has passed away. They also suggest it could be used for replacing the faces of stunt doubles in cases where the conditions of a scene call for them to be used.

This new method is unique from other approaches in a number of ways, including that any face used in the set can be swapped with any recorded performance, making it possible to relatively easily re-image the actors on demand. The other is that it kindles contrast and light conditions in a compositing step to ensure the actor looks like they were actually present in the same conditions as the scene.

You can check out the results for yourself in the video below (as the researchers point out, the effect is actually much better in moving video than in still images). There's still a hint of "uncanny valley" effect going on here, but the researchers also acknowledge that, calling this "a major step toward photo-realistic face swapping that can successfully bridge the uncanny valley" in their paper. Basically it's a lot less nightmare fuel than other attempts I've seen, especially when you've seen the side-by-side comparisons with other techniques in the sample video. And, most notably, it works at much higher resolution, which is key for actual entertainment industry use.

The examples presented are a super small sample, so it remains to be seen how broadly this can be applied. The subjects used appear to be primarily white, for instance. Also, there's always the question of the ethical implication of any use of face-swapping technology, especially in video, as it could be used to fabricate credible video or photographic "evidence" of something that didn't actually happen.

Given, however, that the technology is now in development from multiple quarters, it's essentially long past the time for debate about the ethics of its development and exploration. Instead, it's welcome that organizations like Disney Research are following the academic path and sharing the results of their work, so that others concerned about its potential malicious use can determine ways to flag, identify and protect against any bad actors.

More From

  • Microsoft spins out 5-year-old Chinese chatbot Xiaoice

    Microsoft is shedding its empathetic chatbot Xiaoice into an independent entity, the U.S. software behemoth said (in Chinese) Monday, confirming an earlier report by the Chinese news site Chuhaipost in June. The announcement came several months after Microsoft announced it would close down its voice assistant app Cortana in China among other countries late last year. Xiaoice has over the years enlisted some of the best minds in artificial intelligence and ventured beyond China into countries like Japan and Indonesia.

  • Tesla lowers the starting price of its Model Y electric SUV

    Tesla has lowered the price of another vehicle. This time it's the Model Y, an electric SUV the company started shipping in March. The traditional big three U.S. automakers, Ford, GM and Fiat Chrysler Automobiles, are offering 0% financing rates, in addition to deferred or longer-term payment options, while other automakers have also announced incentives and payment plans to appeal to new buyers and keep existing owners from defaulting on loans.

  • WeWork's chairman says it expects to have positive cash flow in 2021

    After aggressive cost-cutting measures, including mass layoffs and selling several of its businesses, WeWork’s chairman expects the company to have positive cash flow in 2021. Marcelo Claure, who became WeWork’s chairman after co-founder Adam Neumann resigned as chief executive officer last fall, told the Financial Times that the co-working space startup is on target to meet its goal, set in February, of reaching operating profitability by the end of next year. Claure is also chief operating officer of SoftBank Group, which invested $18.5 billion in the co-working space, according to leaked comments made by Claure during an October all-hands meeting.

  • China Roundup: Tech giants take stance on Beijing's data control in Hong Kong

    Hello and welcome back to TechCrunch’s China Roundup, a digest of recent events shaping the Chinese tech landscape and what they mean to people in the rest of the world. The law has important implications for the tech sector, providing a litmus test of business sentiment towards China's regulation over information. Google, Facebook, Twitter, Telegram, Zoom, Reddit among a roster of companies have come to voice their stance.