Animating the faces of fear and courage in Call of Duty: WWII

I remember the days when Call of Duty games didn’t show the face of a soldier who was talking, mainly because the facial animations were so unrealistic. Not anymore. With Call of Duty: WWII, the new game in the blockbuster series that Activision revealed this week, you can see the fear, anger, determination, and courage in the faces of the soldiers.

The technology to render the raw emotions of the soldiers of the Second World War have finally reached the point where the faces and bodies of the soldiers look realistic. And that was the job of Christopher Stone and his team of animators at Sledgehammer Games, the Foster City, California-based studio that is making Activision’s next blockbuster first-person shooter. Stone had to study the faces of fear.

“We want you to feel like you’re a vulnerable 17-year-old kid that was grabbed and given a few weeks of training with an M1 Garand and put on a boat,” Stone said. “We want you to feel that vulnerability, that fear, nervousness, anxiety. We want you to feel angry. We go through a lot of effort to make sure that we put you in scenes and tell a story that’s grounded in reality.”

Stone said that when you land on the beach in Normandy, you have to deal with artillery and machine gun fire.

“You get to the beach and you’re running and putting your head in the sand to try to keep from getting shot,” he said. “That’s a different experience from what the player is used to in Call of Duty. You don’t even have a gun.”

Animating that kind of sequence isn’t easy, and it meant that Stone and the team had to do a lot of research, going to the real locations and consulting their historian Marty Morgan. After 2.5 years, the Sledgehammer team is emerging from that work and is finally ready to talk about it. What they want to create is an experience, not just a power fantasy.

Here’s an edited transcript of our interview.

Above: Christopher Stone, animation director at Sledgehammer Games for Call of Duty: WWII.

Image Credit: Dean Takahashi

GamesBeat: What distinguishes an animation director from other art staff?

Christopher Stone: As animation director, I supervise and direct anything that moves in the game. That includes directing the talent on stage for all the performance scenes, directing all the mocap for the AI and follower behaviors, and on top of that directing my team of animators, integrating and doing the cinematography and the look of all the shots. Kind of a director/cinematographer would be the best way to categorize me.

GamesBeat: What’s the work been like this time around?

Stone: It was great. This game is a more challenging one, I think, than some of the others we’ve done, for different reasons. When I look back at the projects we’ve done here – Modern 3, Advanced Warfare, and now WWII – you couldn’t find three Call of Duty games that were more different from each other. Each of them has its own challenge.

Advanced Warfare was all about figuring out future tech and how to make you this super-soldier, a superhero. This is kind of the complete opposite side of the coin. We want you to feel like you’re a vulnerable 17-year-old kid that was grabbed and given a few weeks of training with an M1 Garand and put on a boat. We want you to feel that vulnerability, that fear, nervousness, anxiety. We want you to feel angry. We go through a lot of effort to make sure that we put you in scenes and tell a story that’s grounded in reality.

That’s the other challenge with this project. We’re not just making it up. It’s based on reality. Again, using Advanced Warfare as a point of reference, if we have a level in San Francisco and we want to have a big crazy moment there, sure, we can blow up the Golden Gate Bridge. You can slide down the bridge and do all this other stuff. With Call of Duty you always need to bring a big experience to the player. The challenge here is, what are those big experiences that really happened?

To me it’s been educational, going back and looking at WWII. There were a lot of events that I wouldn’t have believed if I didn’t know them to be true. Some of them are amazing. Some of them are horrible. Our job is to try to figure out which of those events are events we want to put the player in, for him to really feel to the emotion we’re trying to drive.

Normandy is obviously a big one, storming the beach. They didn’t know what to expect. Those guys all thought that beach would be clear by the time they got there. That’s a lot of emotion being driven in that scene – fear, anxiety. You get to the beach and you’re running and putting your head in the sand to try to keep from getting shot. That’s a different experience from what the player is used to in Call of Duty. You don’t even have a gun.

We’re trying to take chances with things like that, with the idea that by making you vulnerable, making you part of the squad, you can feel as close as we possibly can, in an interactive experience, like you know what it was like to be there.

Above: You are part of a squad in the 1st Infantry Division in Call of Duty: WWII.

Image Credit: Activision/Sledgehammer

GamesBeat: How did you approach it when it comes to creative exhaustion? At a certain point, people got tired of WWII games and the industry moved on to modern settings. Now there’s a chance to make it feel fresh again. What goes into that?

Stone: It’s funny. You think about WWII—myself, I’ve been making games for 20 years. Clearly I remember there being other WWII games. I remember experiencing them and thinking they were amazing at the time. But one of the things that was really shocking to me, and maybe a little humbling—I guess I’m realizing how old I am now. I go back and I start looking at how long it’s been since there was another WWII entertainment franchise. Saving Private Ryan is almost 20 years old now. There’s a whole generation of game players that don’t know this content now, that don’t know WWII.

That’s been very heavy on our minds as we develop this game, staying true to the experience and being respectful of what the war was. Millions of people lost their lives. When you think about how that story is told, and you think about how that story hasn’t been told now for a good period of time, a whole generation of people just don’t know it. Even the guys that fought in that war, they’re in their 90s now. They’re not going to be here forever to tell those stories. From our point of view, this is a chance to help bring that information and pass it down.

The studio, I think, more than any studio I’ve ever known, prides itself on investing itself in whatever content it’s working on. We’ve done so much research, looking at every documentary and every pamphlet and very movie and every field guide for every weapon. Everything to understand what the WWII experience is. We worked with some amazing historians. When we need information, we can ask them. They weren’t there, but they’re able to give us the information we need about how things were done.

That’s been really important, maintaining that authenticity and not having you feel like you’re just a super-soldier who happens to be in WWII. That’s not what we want. In fact that’s been another real challenge in this project. We realized, coming into it, that everything we’ve learned and done for Call of Duty in our recent projects, we can just throw that out the window. It has nothing to do with this anymore. Guys don’t fight the same way. They don’t use the same tactics. There’s no stacking up on the door and breaching and flash-banging a room and shooting a dozen guys. That’s fun stuff in a game where it’s suitable, but it’s not what this game is.

This is a game about the squad, the team. You’re fighting for teammates, relying on them and having them rely on you, growing through this experience. It’s a big change.

GamesBeat: How much do you have to allow for the player to move around? The open-world side of things versus the directed side. I remember that in the first Medal of Honor game, the first time through D-Day was pretty impressive, but when you replayed it, there was only one way to go through it. You play it a few times until you get it right and that’s that. This many years later, though, you don’t have to design it that way.

Stone: Yeah, totally. The open-world nature of developing a game—it has its good and bad points. We definitely try to design spaces to give you more than one option as far as how you want to play it. Normandy beach is very big. You can certainly go in different directions if you want to.

That said, in the same way a soldier would hit that beach and know that he needs to go right there and get to that sea wall, because that’s the only place he’s not going to die—if you start running down the beach, you’re probably not going to live long. That’s purposeful. We want you to feel something true to what the experience really was. If you’re in the middle of a battle and all of a sudden you’re wandering off to some other place without your squad, you’re probably not going to live long. It’s deliberate, the way we do it.

Certainly we give you freedom. The other thing that’s important when you’re trying to author an experience like this is making sure you know—if you know the best-case scenario, the way most people are going to go, you want to tailor the experience for that as best as you can. But certainly there are many ways to skin a cat in the way we lay out the levels in the game.

We’re bringing a lot of new things to this game. It’s not just a non-linear quality. There are certainly elements that are linear, and some that are less so. But to me, one of the biggest things we’ve pushed in this is the emotional drive, the experience of our character with his team. One thing that’s been great for me when directing the talent on the stage is that this is the first product I’ve worked on where, when we’re running the scenes and I call “cut” at the end—there are so many scenes where I sit down and I’m looking at what I just shot, and I have to remind myself that it’s real sometimes. These were real experiences. Some of them are just so emotionally twisting.

Above: Vehicles make a lot of noise in Call of Duty: WWII.

Image Credit: Activision/Sledgehammer

It’s been important to us to maintain a high quality level. You guys saw Normandy and Hurtgen, right? One thing, just talking about reality and trying to keep things as real as possible—when we decided we would do the Normandy boat ride, in my mind I’m thinking to myself, “Well, I can put a guy in a boat, but how do I get that simulated in a way that feels legitimate?” I came up with this design for a moving stage, basically. We have a small stage that we’ve built up on platforms of inner tubes. Then we have our boat operators that come take their seat at the front.

When I’m directing the scene, I’m looking at multiple things. I’m looking at the base cinematography and framing here. Obviously this is going to adjust slightly as we get into the game. I’ll start manipulating it and moving characters around to frame things the way I want it. But this is a rough approximation of what that’s going to be. I’m looking at the physical performances of all these guys to make sure I’m getting what I need. I’m listening to the line deliveries, making sure they’re delivered properly. On top of that, I’m using an iPad to keep an eye on all the facial performances from the actors while they deliver their lines.

GamesBeat: Sure looks real.

This capture tech is amazing. We’re getting all this stuff at once. It has a lot of challenges. You think about getting that much data, that many different moving pieces on that many characters—we’re doing 20 or 30 takes of scenes sometimes, just so we can get one shot that has all those elements perfectly working together. If one line gets dropped here, or one physical action isn’t right, or one facial performance doesn’t deliver enough, we need to reshoot the whole thing.

We put these sweaty boat operators—it was really funny. This was probably take 20-something. Not only was I directing all these guys in the boat, but I’m also screaming directions at these guys. “MORE WAVES! MORE WAVES!” I’m telling them what to do and they’re like, “I’m trying!” By this point they had it dialed in, where I need them to start amping it up and building the waves as we’re getting closer to the beach, really trying to build up the energy with that action as they’re realizing the beaches aren’t cleared.

Doing this stuff is super complicated. You compare it to doing a film, where you can do 30 takes of something, but then you can go back and use little bits of different takes and edit them together. That’s not how this works, because all those pieces of data are really linear and perfectly one-to-one with each other. I can’t grab parts of different takes here and there and still have them work together.

With all that facial stuff—let me show you this here. It’s always fun to see the prepro and how things evolve for a shot. This was our prepro on that shot, which is pretty amusing, especially if you listen to the horrible audio deliveries from the people in the studio. But I use this to walk the talent through what the shot is, helping them understand what the goal is. The thing that’s interesting is that some of the guys in our cast are pretty young, in their 20s. They actually had, at the time, very little understanding of what WWII was all about. For them, in every shot we’re doing, I’m trying to be as elaborate as I can be when I explain to them what the situation was. What are the stakes here? What’s your emotion here, when you’re doing this? They understand that it’s not just about what’s happening here. There’s a global scale to the action.

The other great thing we’ve been able to do, what’s really helped deliver this sort of emotion, is the investment we made in all our facial tech, that capture tech. You can kind of see it in one of these videos, the cameras here hanging in front of each guy’s face. It’s a stereoscopic HMC system, head-mounted camera.

If you go back and look at Advanced Warfare, when we started using an earlier version of this tech, we had a single camera that would capture facial performance on an actor. Spacey would have several hundred shapes we captured from him that became his digital likeness. Now we’ve taken these two cameras and split them. We’re not just getting a one-dimensional view of their expressions. We’re seeing the depth as the expression changes in 3D, which helps us convey a lot of emotion that we wouldn’t have been able to before. Not that I would ever use this, but a great example—if I wanted to have Josh swell his cheeks or whatever to blow out a candle, I could get that information now. These are all shapes that come straight from him. If he opens his mouth, inflates his cheeks, whatever he’s doing, we’ll get an exact re-creation of that in the game.

Above: The graphics have been taken up a notch in Call of Duty: WWII.

Image Credit: Activision/Sledgehammer

GamesBeat: One thing that used to govern the limitations of games was how many moving objects you could have on the screen at once. Now, does that matter as much?

Stone: It still matters. I wish we could do anything we want. [Laughs] We’re still running the game at 60 frames per second. We’re not ever dipping below that. We’re putting a ton of guys on screen, which you saw at Normandy. We have hundreds of guys up there. We can certainly push it a lot farther. But we still have limitations. The good thing is, we’ve been making games for a long time. There’s a lot that we’re starting to understand, having worked in this generations of consoles for a bit. We understand how to get the most out of it. We’re taking all that knowledge and dumping it into this game in a major way. Whether it’s the VFX budget or the lighting, how many lights we can use, how many characters are on screen, we’re taking advantage of everything we can. We have to, for an experience on this level.

GamesBeat: What does or doesn’t look lifelike to you now, with this level of fidelity?

Stone: The toughest thing right now—the capture tech and the faces, we’ve got that. The ability to transfer expression and emotion visually, we can do that. The toughest thing to get right now is the way the skin reacts to light. That’s a big one. It’s hard to tell, but if you look at skin, skin absorbs and diffuses light. Getting that to feel legitimate is a tough one.

GamesBeat: When it’s moving, especially?

Stone: Movement isn’t as much of a problem. Let me use this as an example. In the facial capture we’re getting—let’s go to a part where he’s really emoting. The marks on his face help us understand what’s happening. We run this through a proprietary software that will take this and generate it back to the expressions that were captured. Now, obviously this is Josh’s face, and this is a different character. I’m just using it as an example. But this will end up being one-to-one with exactly what we see.

Once we’ve run it through this program that generates those expressions, then we hand it over to the animation team here and they’ll take the HMC footage and put it side-by-side. They go through and fix any little nuances. Is his eye closing just right? Is the corner of his mouth pinching just right? Are his lips sticking together a bit as they open? Those are all things we go in and finesse to get that last 15 percent of realism.

The difficult stuff really comes down to the way the skin acts: the way it reacts to light, the pore definition, how facial hair looks, how facial hair interacts with the skin when it moves, how wrinkles form, stretching and compression of the skin, how skin color changes with blood flow and volume. Those are things we’re adding now that significantly push the boundaries.

Above: Flamethrower in Call of Duty: WWII

Image Credit: Activision/Sledgehammer

GamesBeat: I saw that scene where they’re riding on the tanks and one guy is talking to another. They did that a lot in Call of Duty 3, but you never saw anybody’s face. Just the helmet moving a bit to indicate that someone’s talking.

Stone: That probably made it a lot easier to do. [Laughs] Maybe we should have tried that. But no, that’s been great. Bringing the WWII experience to the next-gen level, that’s what’s really cool about this now. We can really do it justice. We can do the performances. We can do the soldiers justice, do their experience justice. Before, you played Call of Duty the game. To me this is more as if you’re experiencing it rather than just playing it. You’re in it.

I want the player who’s sat down with this game—I want to see them feeling the anxiety and the nervousness when they’re running up the beach. I want to see them feel the fear of running through the forest when those trees are exploding around them and they’re seeing guys impaled by three-foot splinters. I want to see that in the face of the guys playing our game. And when they’re done playing I want them to be able to set down the controller and take it all in, feeling like they know just a little bit of what it might have been like to be there.

That’s completely different from the experience I would have expected from Advanced Warfare, where the whole goal was for the player to feel this power – being able to jump over buildings and all that. It’s a completely different reaction. We want that, a different reaction to a different game.

GamesBeat: It’s not as much of a power fantasy anymore. More of an experience.

Stone: Right. You hit the nail on the head there. That’s what we want. We want the player to feel something.

V

entureBeat's PC Gaming channel is presented by the Intel^® Game Dev program.