A Brief History of Motion-Capture, from Gollum to Caesar

This week sees the arrival of Dawn Of The Planet Of The Apes, the latest in the long-running science-fiction series, and the first sequel to 2011’s franchise-reviving Rise Of The Planet Of The Apes. The new film has been met with mostly admiring reviews, almost every one of them heaping praise on its star, Andy Serkis. Dawn also represents a high water mark in the ever-evolving motion-capture technology used to bring the film’s non-human cast to life.

Also known as performance-capture, “mo-cap” has been around in various forms for the past few decades. The technology really came to prominence in 2002, with the second Lord Of The Rings film, in which Serkis — in a breakout performance — turned Gollum into a “living” creature that physically interacted with other characters. To mark his latest performance, we’ve donned our Lycra bodysuits and face-mounted cameras to take a look back at a brief history of motion-capture, which started as a tool, but has now evolved into something of an art.

1970s-2001: The Early Years

The idea of mapping animation onto actors is almost as old as animated feature films: Disney’s pioneering 1938 film Snow White & The Seven Dwarfs partly utilized a process called ‘rotoscoping,’ whereby artists drew over live-action film frames. This technique was later used in animated films like Ralph Bakshi’s Lord of the Rings (1978) and the 1985 music-video for a-ha’s “Take on Me.” Computers pushed the process further: the Rotoshop system used in Richard Linklater’s Waking Life (2001) and A Scanner Darkly (2006) is something of a primitive precursor to motion capture.

Initially developed by bio-mechanical engineers for motion studies, by the mid 1990s, the technology was being used in video games: Highlander: The Last Of The Macleods and Soul Blade both utilized motion-capture to bring greater realism to their on-screen avatars. It was only a matter of time before fimmakers caught on, although the first movie to be made exclusively with the technology is justifiably forgotten. 2000’s Sinbad: Beyond The Veil Of The Mists, an Indian-made animation featured the voice (but not movements) of Brendan Fraser, which looked like a very creaky video game cut-scene, and made just $30,000 on its brief theatrical release. The following year, big-budget animation Final Fantasy: The Spirits Within also utilized the technology in part, though also flopped at the box office.

1995-2006: The First Motion-Capture Characters

Popular conception has it that Gollum in Lord of the Rings was the first motion-captured character in live-action film, but that’s actually incorrect. The technology was first put to use to create a ‘digital double’ for Val Kilmer in 1995’s Batman Forever, and James Cameron populated crowd scenes in Titanic with performance-captured figures, replicated on a grand scale. And in the summer of 1999, actor Ahmed Best played Jar-Jar Binks in Star Wars Episode I: The Phantom Menace — he was present on set, but the CGI character, his movements animated based on the capture of Best’s performance on a sound-stage after the fact, was added in his place afterwards.

Jar-Jar was hated by fans, though, and so it’s no surprise that Gollum has the more lasting place in the history books. British stage actor Andy Serkis was hired by Peter Jackson to play emaciated ex-Hobbit Gollum/Smeagol in the Lord of the Rings films, and flew out to New Zealand essentially ignorant about motion-capture. The process wasn’t dissimilar to the one that Best had been through: Serkis would work with the actors on set, another take would be filmed with him just off-screen (during which he’d describe his motions to the other actors). Months later, Serkis would record Gollum’s movements on a special soundstage at Jackson’s visual-effects company WETA Digital, in front of a camera setup called The Volume. The basics of the technology weren’t all that different to how it works today: an actor (usually clad in a tight-fitting unitard) wears a series of sensors around their bodies. Multiple cameras are placed around the performer, which allows a computer to replicate a 3D model of the movements, which can then be exported into animation programs.

image

Rings star Serkis, in motion-capture unitard (left); at right, the final on-screen result.

At the time, only basic movements in his torso, legs and arms could be captured, and the animators had to use footage of Serkis performing scenes as reference to animate the face, but as the technology developed, mo-cap systems enabled actors to perform the scenes in real-time opposite human characters without having to reshoot them months later (this is how the Apes movies are filmed today).

2004-2011: 3D Motion-Capture

After the leaps forward made by George Lucas and Peter Jackson, Forrest Gump director Robert Zemeckis went one better: he decided that his next project, The Polar Express (2004), would be an entirely performance-captured 3D animation. The film saw star Tom Hanks play five characters, including a young boy, and Santa Claus, with facial movements captured alongside his body for the first time. The film was a modest hit, but a number of critics called the film ‘creepy:’ the characters were photorealistic in some ways, but the eyes in particular were seen to be glassy and soulless, dipping into the so-called ‘uncanny valley.’ Nevertheless, Zemeckis continued making performance capture movies for the next several years, directing Beowulf (2007) and A Christmas Carol (2009), and producing Monster House (2011) and Mars Needs Moms (2006). There were increasingly diminishing returns, though, and the latter proved to be one of the biggest money-losers of all time, causing the director to scrap his planned mo-cap remake of Yellow Submarine and return to live-action.

image

Hanks filming The Polar Express.

Others had more success, though. In 2009, James Cameron unveiled the film he’d been working on for close to a decade, with the live-action/CGI hybrid Avatar. Teaming up with Gollum creators WETA, and filmed mostly on their Volume stage, it wasn’t that Cameron’s space adventure was revolutionary as such, but it refined the techniques and artistry, with near-photorealistic characters with deliberately oversized, uncanny-valley-avoiding eyes, creating a world so immersive that it became the biggest-grossing film in history. Jackson and his company also teamed with another legendary filmmaker, Steven Spielberg, for 2011’s The Adventures Of Tintin, which also featured Andy Serkis among the performers.

image

Serkis in this year’s Dawn of the Planet of the Apes.

2006-Present: The Rise Of The Performance-Capture Superstar

In the years since Lord Of The Rings, performance-captured characters in live-action movies had come on leaps and bounds. 2005 saw Serkis and Jackson reteam for King Kong, which introduced facial capture, and in the following year, a major leap forward was made in 2006’s Pirates of the Caribbean: Dead Man’s Chest: rather than being filmed separately in a studio, Bill Nighy performed on set as squid-faced sea-captain Davy Jones, with the fully CGI-character mapped exactly to his movements. But it was 2011’s Rise Of The Planet Of The Apes that really saw the next sea change, and it was about a person rather than the technology. Andy Serkis had always been seen as a pioneer in the form, but playing Caesar in the reboot was something quite different: he was the lead character in a live-action movie, and with the sensors more impressive than ever, Serkis won rave reviews for his turn as the super-intelligent chimpanzee, even getting some Oscar buzz (with support from human co-star James Franco), and hitting the talk-show circuit in support of the movie. (He also reprised the role that made his name in the first of Peter Jackson’s Hobbit movies, with on-set real-time mo-cap and an improved muscle model making Gollum even more life-like than before).

That’s only become more true with Dawn Of The Planet Of The Apes (which also leaps forward by enabling dozens of mo-cap actors to perform in real life, outdoor daylight locations, which would have been unthinkable only a few years ago): Serkis is the only major cast member to return from the first film, and has carried much of the publicity for the film on his shoulders. He’s also moved behind the camera: in 2011, he founded The Imaginarium Studios, a production company that consults and provides performance-capture work. He’s lined up directing work that takes advantage of his experience, on a passion project version of Animal Farm, and Warner Bros’ new take on The Jungle Book, and with his company, is consulting on the two biggest movies currently in production, Avengers: Age Of Ultron, and Star Wars: Episode VII (with Serkis also taking performance-capture roles in both). Even if you don’t know his face, there’s little doubt that he’s become the very first motion-capture superstar.


*This post has been corrected since its original publication.


Ape photos: Twentieth Century Fox Film Corporation

Polar Express photo: Warner Bros./Everett

Lord of the Rings photo: New Line/Everett