Advanced De-Aging VFX Are Crucial to ‘The Irishman,’ ‘Gemini Man,’ and ‘Captain Marvel’
Click here to read the full article.
It’s the year of digital de-aging as technology caught up with need for Martin Scorsese’s mobster epic, “The Irishman,” and Ang Lee’s sci-fi/thriller, “Gemini Man.” Industrial Light & Magic devised an unobtrusive facial capture breakthrough to make Robert De Niro, Al Pacino, and Joe Pesci look decades younger as hitman Frank Sheeran, Teamsters leader Jimmy Hoffa, and Philly crime boss Russell Bufalino, respectively. And Weta Digital has constructed the most fully realized digital human yet as 50-year-old Will Smith fights 23-year-old Will Smith in a face off between a hitman and his clone.
Meanwhile, Lola VFX, which has become Marvel’s go-to de-aging specialist, has made great strides this year with its vaunted 2D Photoshop-like procedure of skin smoothing and shape warping on “Captain Marvel,” making Samuel L. Jackson’s Nick Fury look decades younger without the use of a young double for the first time. (For good measure, Lola also tackled puberty on “It Chapter Two” to slightly de-age The Loser’s Club.)
More from IndieWire
When Netflix premiered Scorsese’s long-awaited three-and-a-half-hour saga at the New York Film Festival last week, ILM’s costly de-aging experiment (pushing the budget to $160 million) proved effective in conveying the Oscar-worthy performances from 76-year-old De Niro, 79-year-old Pacino, and 76-year-old Pesci. Not surprisingly, the director’s summary statement about “loyalty, love, trust, and ultimately betrayal,” represents his version of Sergio Leone’s similarly-themed “Once Upon a Time in America,” which also starred De Niro. Except Scorsese has replaced De Niro’s opium-induced fever dream with a grittier (though no less mournful) remembrance of things past.
Indeed, the key to Scorsese’s de-aging strategy on “The Irishman” was presenting Sheeran’s criss-crossing flashbacks (mostly the ’50s, ’60s, and ’70s) as an elderly man’s reflection on his life. It was therefore about shaping performances with youthful massaging rather than creating younger replicas of De Niro, Pacino, and Pesci. “This isn’t just about lenses and computer imagery,” Scorsese said at the New York Film Festival Q&A. “It’s about posture, it’s about movement, it’s about clarity of the eyes, everything.” Which is why during his first shooting experience with Pacino, the director had him do several additional takes, jumping up from his chair in anger while watching JFK on TV with his family until he approximated the age-appropriate 49-year-old Hoffa.
For ILM, the tech challenge was to create the lightest possible capture footprint for the trio of actors. “In the first meeting four years ago, De Niro said there was no way he was going to wear a helmet camera or facial markers,” ILM VFX supervisor Pablo Helman told IndieWire. “He wanted to be onset with the lighting, acting with other actors. And he said there will not be any controlled environment for re-shoots.
“With helmet cams you need to do calibration and that also requires two hours of makeup,” added Helman. (De Niro required prosthetic makeup only as Sheeran in his 80s.) “And the main problem for marker technology has to do with the lighting. You need to get those faces lit or else those markers don’t read. What we came up with is something that has never been used before without helmet cameras or markers.”
The camera system and companion software that ILM developed captured the actors’ facial performances on set, and then translated those unaltered performances to full CG versions of their younger selves with its proprietary models. ILM’s special three-camera rig (termed “the three-headed monster” by DP Rodrigo Pietro for its bulkiness) consisted of two witness cameras on either side of the director’s camera. The witness cameras captured the most amount of geometry from the actors in the onset lighting without obstructing the principal photography.
“It was slowly getting through performances and getting through takes, and moving on,” Helman said. “They were never waiting for us. But the post-production process was a little bit different from any other production that I have been on. We had never really showed Marty intermediate takes. He trusted us enough so that we would finish a shot, render it with the right lighting, and we would show him the performance. And if the performance had the same feeling that he had with the original performance he selected, we moved on.”
But if it didn’t, they discussed getting a better match. Scorsese, however, insisted on no keyframe-animated enhancements. ILM strictly used the raw data to slightly dial up the variation models for the three actors to achieve greater fidelity to their performances. De Niro developed the reserved Sheeran with a signature scowl, Pacino played the hot-headed Hoffa with manic exuberance, and Pesci offered a quiet menace as Bufalino. “[Scorsese] painted these characters as having a really rough life and, to him, it means that some people age differently than others, and there are all kinds of wrinkles and even body movements that echo what you have lived,” Helman said. “That is something that is completely different. And this achievement is going to be measured for what it does for the next generation of filmmakers on set with lighting.”
By contrast, “Gemini Man” offered a completely different approach to de-aging (shot in 3D at 120 frames-per-second and 4k resolution by cinematographer Dion Beebe). In fact, the filmmakers refuse to call it de-aging. “We are not de-aging,” Lee said. “I rather think that we are creating a new character, a youthful Will Smith.”
“To the layman, yes, de-aging is just making a person look younger,” added production VFX supervisor Bill Westenhofer. “But from our side, de-aging has been associated with the Lola process. Whereas this is creating a person from whole cloth. We knew we had to make a digital human and once we did, it made sense to do it everywhere.”
Smith played the aging Henry as well as his clone, Junior, channeling his younger self. Junior, therefore, represents a major character breakthrough for Weta, appearing in more than half the movie, and required to express a range of emotions performed by Smith.
Weta created the CG Junior (under the supervision of Guy Williams) by studying the morphology of aging at it applied to the actor. The wizards of Weta then made great strides particularly in the areas of skin and eye work. The animators created a new procedural software for pores that simulates areas between the pores and along the natural fall lines for a more natural look. And modeled a dark retina for the eyes to reveal more depth, and provided an additional film surface that sits across the eye for greater fidelity.
And accommodating 120 fps worked to Weta’s advantage with some of the smooth skin artifice replaced by more natural sharpness and crispness. “That’s why we [pushed] the envelope as hard as we possibly can do,” said Westenhofer, “to be the first to deliver a fully convincing digital human.”
But Lola, the de-aging expert, has come a long way since touching up Patrick Stewart and Ian McKellen in “X-Men: The Last Stand” (2006), Brad Pitt in the Oscar-winning “The Curious Case of Benjamin Button” (2008), Michael Douglas in “Ant-Man” (2015), Robert Downey Jr. in “Captain America: Civil War” (2016), Kurt Russell in “Guardians of the Galaxy Vol. 2” (2017), and Douglas, Michelle Pfeiffer, and Laurence Fishburne in “Ant-Man and the Wasp” (2018).
“One of the things that we’d always done in the past was to shoot a double and to recreate a performance so the main actor would do the piece, the other actor would watch, and then they would re-enact the thing,” said “Captain Marvel'” production VFX supervisor Christopher Townsend. “But with Sam being in two-thirds of ‘Captain Marvel,’ we couldn’t do that. It would take too much time and be too difficult to match performances for every shot.”
Fortunately, Jackson has aged very well and has great skin, so it was no problem for Lola to go without a double. They used some makeup to pull back the skin on his neck, but relied on the actor’s performance with no grafting — just slimming and tightening and smoothing over. “It’s very exciting to arrive at this point where we’re de-aging a major character for the entire length of a film,” said Lola VFX supervisor Trent Claus. “With a project of this scale, we did indeed have to modify our usual methods a bit in order to accommodate the volume of shots.”
Ultimately, though, the de-aging process must always be at the service of the actor’s performance. “You’re sculpting this whole thing,” Scorsese said. “It’s like living models in a way. Plus the truth of how they’re interpreting. It’s an extraordinary experience.”
Sign up for Indiewire's Newsletter. For the latest news, follow us on Facebook, Twitter, and Instagram.
