Inside Hollywood’s big Tom Hanks deepfake gamble

To adapt screenwriter Colin Welland’s famous acceptance speech from the 1982 Oscars, “The deepfakes are coming.” The first big budget A-list Hollywood film using so-called deepfake technology is in production. Robert Zemeckis and Tom Hanks, who won the Best Director and Best Actor Oscars respectively for 1994’s Forrest Gump, will use extensive deepfake wizardry in Here, a Miramax movie also starring Paul Bettany, Kelly Reilly and fellow Gump alumna Robin Wright.

The technology will see the actors “seamlessly transforming into younger versions of themselves” in ways that were “previously impossible”, Zemeckis said in details released today. He and Hanks ­– who also worked together on Cast Away, The Polar Express and Pinocchio – have grossed $14 billion between them at the global box office. Flop or hit, Here will put deepfakes on the map.

Deepfake technology is a sophisticated illusion. It involves a person’s appearance being digitally altered by a camera using Artificial Intelligence (AI). Here is based on a time-jumping graphic novel by Richard McGuire, and so cinema-goers will see Hanks at al transformed via what the publicity material calls “hyperreal AI-generated face replacements”.

This is just the tip of the iceberg. The film is the opening project of a tie-up between US talent agency Creative Artists Agency (CAA), which represents Julia Roberts, Steven Spielberg and Brad Pitt, and Metaphysic, a deepfake technology company. Together the groups plan to “unlock creative possibilities for artists using AI across film, television and [other] entertainment”, they said today. The very real mainstream beckons.

What’s extraordinary is the speed at which this has happened. Deepfakes only really came to public prominence last September when the final of TV show America’s Got Talent featured deepfakes of Elvis Presley and pop svengali Simon Cowell, again created by Metaphysic. Viewers saw largely unknown singers line up on the stage only to be transformed – thanks to the AI trickery embedded in the cameras in front of them – into the above-mentioned celebrities on the big screen behind them.

It was this performance that captured the attention of the high-ups at CAA and Miramax, Metaphysic says. (So much for Cowell’s apparently waning influence in the entertainment world.) Last week UK audiences were introduced to deepfakes via ITV’s tepid new comedy Deep Fake Neighbour Wars. In this, uncannily realistic fake versions of Idris Elba, Phoebe Waller-Bridge, Jay-Z, Olivia Colman and Harry Kane bicker about humdrum domestic issues like garden maintenance. It’s Stella Street meets Avatar.

“This will be the most profound use of the technology yet,” Metaphysic co-founder Tom Graham tells me from Pinewood Studios, where Here is in an unspecified stage of production. He says that while much of the film industry’s interest in deepfakes can be put down to America’s Got Talent, Tom Cruise is also to blame.

Metaphysic was behind the series of deepfake Tom Cruise videos on TikTok that went viral over lockdown. In them, the (faked) megastar did goofy things like dance to George Michael’s Freedom in his underwear. Over 250 million views helped spread the word. People now know what so-called manipulated media is. “Everybody in Hollywood knows [about] it. Everybody loves it,” Graham says. Intriguingly he says that some of Metaphysic’s investors are CAA clients, but a spokesperson won’t name names.

This is what deepfake technology does not involve. It does not involve people donning motion capture bodysuits for weeks so their movements can be filmed and then digitised (as ABBA did for their ABBA Voyage show). It does not involve teams of visual effects (VFX) artists painstakingly building 3D models on computers (a booming industry that is, reportedly, akin to a sweatshop). Nor does it involve green screens, CGI animation or multi-layered digital compositing.

Rather, it involves in-camera software that automates the process of recreating what someone looks like. How? The software is “trained” – Graham’s word – by being shown thousands of images of the target person. The camera, via algorithms, then mimics and anticipates how that person moves. It’s like a digital face filter. The technology has developed further since Cowell and The King back in September: it is now so sophisticated that an actor can watch themselves performing as a younger version of themselves on a monitor in real time (the technology is called Metaphysic Live).

This allows actors to perfect a performance on set. “If the person who’s older is trying to replicate the performance of the younger self, seeing what they look like live as the younger self has a huge impact on the feedback loop,” says Graham, referring to the on-set back-and-forth between actor, director and crew.

There are cost advantages too. Graham won’t talk about Here’s budget (“I’m new to this world, and it’s very secret”). But he says that traditional CGI techniques, such as 3D computer modelling, can be five times more expensive than deepfake technology.

Zemeckis says he “tested every flavour of face and de-aging technology available” and concluded that Metaphysic are the “global leaders”. Here sounds tantalising, but Zemeckis’s past dabbles in computer trickery haven’t been universally successful. The Polar Express, from 2004, was the world’s first all-digital capture movie: Hanks wore a motion capture suit as he played five characters (the gruelling process involved digital nodes being attached to his face, giving distinct Pinhead-from-Hellraiser vibes). The end result broke ground, certainly. But it looked like a PlayStation game. The film took $30 million at the US box office in its opening days, way below the $70 million opening weekend take of Pixar’s The Incredibles the same month.

Zemeckis also directed last year’s Pinocchio, Disney’s live-action and VFX remake of its 1940 animated classic, also starring Hanks. In his one-star review, my colleague Robbie Collin slammed it as a “garish, dead-eyed exercise” and wrote that it “can’t be over-stressed just how badly the do-it-again-but-photoreal approach backfires”. The computerised Pinocchio was so soulless as to resemble a piece of Disney merchandise, Collin wrote. The film was released direct onto the Disney+ streaming platform.

Graham says he hasn’t seen either film so can’t comment. But he says deepfakes are completely different, technologically-speaking. “These AI models could create something so realistic that nobody can tell the difference. And that’s a whole paradigm shift in terms of the software that we use from traditional VFX 3D computer graphics pipelines… It’s a generational step-change,” he says.

I ask him about director Richard Linklater’s latest film, a Paul Mescal-starring version of musical Merrily We Roll Along that will be filmed incrementally over the next 20 years as its characters age. Linklater used a similar approach in 2014’s Boyhood, which was shot over a 12-year period from 2002. But while deepfake technology didn’t exist in 2002, it does now. Is Linklater missing a trick here? Digitally-ageing his characters would save money and – literally ­– decades. “Richard should give us a call,” Graham says. More generally he says that while the concept sounds “pretty cool”, such ideas simply aren’t feasible for more commercial projects that don’t have the luxury of a 20-year production cycle.

Deepfakes were first predicted a while ago. In 2013, the director Nicolas Roeg mused about the future of filmmaking. The man behind Don’t Look Now and The Man Who Fell To Earth wrote that there were only two further technical steps forward that the industry could possibly take. These were the creation of a man-made machine that either “has an ‘opinion’ based on information that it has been processed to have, [or] an opinion of its own based on information that it has been physically ‘man-given’; but coupled with emotionally-guided reasoning.”

Roeg predicted that either of these developments would represent the “Birth of the Future”. The director died in 2018. But a version of his future has indeed been birthed. With actors providing the emotionally-guided reasoning and cameras using their highly-programmed opinions, Roeg’s prediction has come to pass with alarming accuracy.

And yet is there really so-called first-mover advantage in this sphere? I remind Graham of that expression about being able to recognise pioneers by all the arrows in their back. In other words, being first can lead to slower rivals watching you fall, pinching your ideas and beating you to your destination. Better, the logic goes, to wait until an idea is proven before setting out.

Graham disagrees. “The number of people who can do what we do on this movie is very small. Every VFX company in the world is trying to do it. And they can’t just suddenly go from being in VFX – very manual – to be a software engineering company around this emerging technology that changes every two months. A Tour de France cyclist doesn’t suddenly get into a Formula One car,” he says.

The press release about Here prominently mentions Metaphysic’s “ethics-first approach”. This could be a coded dig at ITV and StudioNeural, the technology company behind Deep Fake Neighbour Wars. Graham says that the programme has recreated versions of the world’s biggest celebrities “without their consent”, something that “opens themselves to legal challenges the likes of which could be scary”. He says that Metaphysic was approached to do something similar. “We will not create something where there is not direct informed consent from the persons whose hyperreal synthetic lacquers we’re recreating,” Graham says. It’s an ethical can of worms: if something’s fake, does it need approval from the thing it’s faking?

In a statement, ITV said that the opening credits of Deep Fake Neighbour Wars make it plain that the likes of Jay-Z and Kim Kardashian have had nothing to do with the show. There is also a “Deep Fake” logo in the corner of the screen throughout and the digital masks fall away from the actors’ faces at the end. Also, the scenarios are so surreal that they’re clearly fake. In other words, consent wasn’t sought because it wasn’t needed.

A spokesman would not comment on whether any of the real celebrities had complained. “Comedy entertainment shows with impressionists have been on our screens since television began, the difference with our show is that we’re using the very latest AI technology to bring an exciting fresh perspective to the genre. The show complies with all our obligations under the OFCOM code and all relevant laws,” the broadcaster said. StudioNeural did not reply to a request for comment.

I mention the disclaimer in the credits to Graham. “The lady doth protest too much,” he says. Ouch. Could deepfake wars be brewing before the scene has even taken off? In another sign of dissent, acting union Equity has launched the “Stop AI Stealing the Show” campaign to protect performers’ rights. It sounds more like a nest of vipers than a can of worms. Such arguments will doubtless proliferate as deepfakes become more common.

So despite the high-profile boost that Hanks and Zemeckis will give to deepfakes, it’s clear that many issues remain unresolved. More than anything else, though, the public need to believe what they’re seeing. Zemeckis made his name directing Romancing the Stone and Back to the Future. He’ll be hoping that when it comes to deepfakes, the title of his 2000 blockbuster – What Lies Beneath – will be far from viewers’ minds.