‘Avatar 2’ VFX Team on the Evolution of the Tulkun and Simulating Water for Performance Capture

Shortly after the first “Avatar” finished, James Cameron and the VFX team at Weta went into discussions about what could be better for its sequel. The team started with a virtual camera and stage set up at Cameron’s Lightstorm Entertainment production company in Santa Monica.

“That led to things like building a tank and doing performance capture above and below water,” Joe Letteri, visual effects artist and Weta FX director, tells Variety.

More from Variety

• 'Avatar: The Way of Water' Crosses $1 Billion Internationally, $1.48 Billion Globally

• Box Office: 'M3GAN' Aims for Impressive $20 Million Debut, but 'Avatar 2' Will Rule Again

• 'Avatar: The Way of Water' Wave Continues at U.K. Box Office, 'Whitney Houston' Makes Strong Debut

Like the first film, there were major technological breakthroughs that have made “Avatar: The Way of Water” the leading visual effects Oscar contender. Among these advances was depth compositing of the live-action characters.

“In our simul-camera, we are seeing layering of characters which allowed for better integral integration on set, better eyelines and better composition of shots,” visual effects supervisor Eric Saindon explains. “We took [existing] technology from sporting events, FlyByWire cameras, and put an eyeline system on that. So that when characters like Spider (Jack Champion) were walking around set, they had a character on set that was in the right location based on the performance capture.”

Adds Daniel Barrett, senior animation supervisor at Weta FX, “The eyeline camera helps the actors in their performances. In terms of integration in a stereoscopic film, it can be really difficult to fix an eyeline that doesn’t work, so you have to get it right on the day.”

But the biggest breakthrough was how the team developed underwater performance capture. Giant water tanks were used for the dazzling water sequences in the film. “The underwater capture was an amazing feat,” Barrett says. “The capture on set as well as the facial system that we developed internally was probably, far and away, the most significant thing for the animation team. It’s a big step forward in terms of the way we do facial work.”

While every frame in the film can be deemed a technological revelation, one effect that has the internet abuzz is the Tulkun, a majestic sea whale-like creature that roams the oceans of Pandora. “You never call it a whale,” Barrett says. “It’s a Tulkun.”

However, he admits whales and seals were very much an inspiration. “They predominantly move like a whale when they’re swimming. What makes them look slightly different is that they are quite articulate in the way they can move,” he says. “Their tails are different from whales. A Tulkun can move its head around… The performance capture that was set up in the tank was very clever. It gave Britain Dalton’s character something to work with.”

A scene Saindon is particularly proud of is the Tulkun hunt. For the scene, the film’s production designers Dylan Cole and Ben Procter built a full-size Picador boat. According to Saindon, having real boats and a motion gimbal helped enhance the actor’s performances.

“We took it out in the ocean for shots and reference. We got the wave cadence of wave jumping, of the boat moving around and of going through different waves,” Saindon says.

That data was taken back to animation in order to drive the boats on CG water. The animation team then brought it back to the live-action sets where everything could be put onto a motion gimbal. “The actors were actually reacting through the Tulkun hunt and they were really driving and jumping the boats,” Saindon says. “We were able to use that information to drive the actors’ performance and shoot those live-action plates to bring it back into CG again.”

He adds, “It allowed us to use our new water pipeline to put the boats and add the water to the live-action plates. It made the whole process feel very integrated and real.”

All in all, over 300 shots went into the Tulkun hunt sequence. “It was water simulations for every single one of those shots. And they were very hard water simulations because it’s boats getting pulled around by the Tulkun, jumping waves and the Tulkun moving through water,” he says. “It’s the connection between the boats, water and the Tulkun that had to get solved altogether.”

Best of Variety

• 19 Best Anthony Hopkins Performances: From ‘The Silence of the Lambs’ to ‘The Father’ 

• Oscars Predictions: Original Score – Could John Williams Become the Oldest Competitive Oscar Winner in Academy History?

• Oscars Predictions: Animated Feature – With 21 Films Eligible for BAFTA, Which Three Will Get a Boost?

Sign up for Variety’s Newsletter. For the latest news, follow us on Facebook, Twitter, and Instagram.

Click here to read the full article.