Tech companies offer a glimpse of AI-powered characters in games

Glimpses of the future often emerge at the Game Developers Conference in San Francisco. It’s where indie games generate a buzz or the kernel of a big idea is planted.

This year, artificial intelligence took centre stage as Nvidia showed off jaw-dropping capabilities of its chipsets and software companies such as Convai and Ubisoft revealed intriguing gameplay opportunities.

AI has been the source of hype and fear over the past year. It’s been at the centre of labour conflict in Hollywood while in the tech industry companies see it as a way to improve efficiency. In the primordial tech ooze of video games, AI is set up to have a transformative impact on one part of the industry.

Enter Convay

Convai, a company that creates AI developer tools, offered a firsthand look through three projects. One was “Second Life,” where representatives showed how AI bots could act as guides for novice players. In a massively multiplayer online game, the non-playable characters act almost like a concierge with more agency.

Players can ask them about locations and they can teleport them to new locales. They can generate surfboards on command so they can hang ten on a virtual beach. What’s notable is that the AI can not only answer questions but it knows the environment around it. The bot even has long-term memory so that it can recall places that players visited.

This raises privacy issues, but Convai said each bot has its own speaker profile and there’s no mixing of memories. Developers can also limit the memory by time so that it will forget.

“Stormgate” is an upcoming real-time strategy game from Frost Giant Studio, and in that demo, a character named Dr. Fletcher offered me a sitrep on what’s going on in the battlefield. This demo wouldn’t be in the game, but I could ask the NPC about the world and she would offer hints into the lore.

Project Neural Nexus impresses

By far the most impressive demo was “Project Neural Nexus,” it takes place in a cyberpunk world and throws players into a situation where a woman named Seraph has to guide players to the roof. That would be a normal scene in a video game that transitions to a bigger fight but wandering through the world, Convai officials urged me to talk with the inhabitants.

I chatted with them as I walked to the objective, learning more about the world, but I wanted to have a little more fun with it, so I asked one character if I could borrow some money, and they responded without a hiccup berating me. Later in the demo, Seraph handed me a gun and I asked what kind of weapon it was (It’s hard to tell with futuristic sci-fi weapons.) and she told me it was a submachine gun. It showed the characters also know about objects in the world as well as the general setting. Curious about her long hair, I asked where she got it cut and she told me she didn’t have time to have her hair done fighting the enemy.

The most interesting conversation I had was with Mae “Shadow” Liang. This masked character was bragging about how she wasn’t bound by any law and that as an assassin she could do whatever she wanted. I brought up the Geneva Conventions and asked if they applied to her trade, thinking she wouldn’t know anything about it. To my surprise, she addressed it and gave a shockingly interesting answer.

The other fascinating interaction involved a robot named WENDY. I spoke it with and the robot was aggressively hostile toward me. Taking offense, I criticized it and the robot shot back at me for one more memorable conversations. It was strange how an NPC could elicit such a strong reaction from a simple conversation. It just deepened the believability.

Nvida’s demo

I had a similar experience at the Nvidia demo called “Covert Protocol,” which featured Inworld AI technology. In this situation, I had to gain access to the room of NexaLife’s CTO. I went in knowing nothing and I had to talk my way to gaining information.

I started with the doorman Tae, who I learned was also an aspiring comedian. I asked him to tell a joke and it was a bit tepid, but he did guide me to the reception Sophia. With Sophia, I straight up tried to ask her for the CTO’s room number but she rejected me citing the Wavre Hotels’ policy that protected clients’ privacy. I asked her where I could get a drink and she directed me to the bar right across from her. Again, these AI-driven NPCs are aware of the items in the room as well as what’s around them.

I ran into Diego Martinez, who was also a Nexalife exec but he was under strain having to write a keynote speech. I asked him about his colleague’s room number but he wasn’t having any of it. He came off as standoffish, so I lied to him and told him I was the police and needed that information.

Interestingly, he didn’t budge and was even more sarcastic and gruff. The demo ended in failure, but it had a different feel from the Convai one. It felt more focused and I had to peel away at the right responses. I couldn’t go back and talk to him again because like in the Convai demo, Diego would remember me and what I said.

AI in a game developers’ hands

Lastly, I experienced what AI technology could do in the hands of a video game company. Ubisoft offered a demo featuring their Neo NPCs. It takes place in another future dystopic world and players meet with a member of the resistance named Bloom.

Virginie Mosser, the Ubisoft narrative director, called the experience conversational gameplay. Unlike the previous demos, this one was more guided as players have objectives they have to accomplish to build trust with the Neo NPC.

Players have to look for openings in the back-and-forth and use those to find out more about the world and Bloom, my fellow resistance fighter. When I asked him, why he wanted to fight the megacorporations, he mentioned that he had a son, so I asked about his name. He said the child was named Evrand. That effort increased the trust level between us at the same time a robot beside us commented on my interviewing prowess.

The chat between me and the NPC showed how players will have to figure out ways to artfully bring up subjects and investigate the the character in front of them. I learned that Bloom was nonviolent and was a generally positive person. If I went beyond the boundaries and asked a random question, the AI would steer the conversation to be on mission in a subtle way.

During one of the more surreal answers, Mosser said the Neo NPC was hallucinating. Sometimes, when a question pierces the knowledge base, they come up with vague answers that sometimes sound normal.

Planning a heist

The other part of the demo led me to Iron. She’s a female leader of the resistance and asks for my opinion on a mission to break into a villa and steal some files. Around the front yard, there are several possibilities. A tree nearby could allow access to a window and a grappling hook was one of the tools. I brought up the tree first but she had hesitations about it because she didn’t know if a branch would hold up both of us. She took to the grappling hook idea.

Looking over other tools available, I saw a note where Bloom was supposed to praise or mention what he thought of you based on your prior talk, according to Mosser. It’s another generative twist in the gameplay that shows how players impact the characters.

The next section dealt with the guard, and the first thing I asked was if we knew anything about him. She said he had prior military experience so I knew maybe a tranquilizer dart may not be a good idea. I suggested sleeping gas and she liked that idea. The last section was the surveillance camera and I brought up the idea of an EMP but she said it would cause too much of a ruckus. It also wasn’t on the list of tools. The most obvious answer was a smoke bomb but I also saw a diagram, where we could cut the electricity. I went with that.

She then asked what should we do if things went awry and the guards found us. I responded, saying, “I’m going to curl up in a ball and cry.” Iron actually got my sarcasm but wasn’t amused by my joke.

Unfortunately, the AI demo took what I said literally and it showed you the result of the attempted heist in the third part. We failed. That joke had serious repercussions as Bloom gave a run down in the after-action report. He said that my bawling did provide enough distraction for us to escape, but we didn’t get the files. Again, the technology isn’t perfect it showed some fascinating gameplay opportunities and unexpected responses.

Final thoughts

It seems as though AI in games is still a bit of a ways off, but there are moments where it could be used in subtle gameplay ways. That Bloom note is an example of generative AI creating a summary of your actions. That could be helpful if you’re returning to a game after a few months and need to know what you did previously. For some of the more ambitious elements such as the Neo NPCs, that’s going to be much more impactful when it comes to story-driven games.

It fundamentally changes how video games are written. Although branching dialogue doesn’t need to be written, the characters that game studios create have to be more nuanced and have a deeper background. They need to have stories and motivations behind their actions and respones. Writers will need to come up with whether they grew up with pets, who their parents were, what matters to them and how the perceive the world. It’s up to the writers to put the ghost in the machine.

In order to bring characters to life, studios need people who can create compelling characters, and if they don’t, they will appear off to players who are bound to ask them all sorts of questions. Mosser said they’ll hallucinate answers that may not jibe with the fiction.

That in turn will require deeper lore from which the AI bots need to draw from. Games have bibles worth of stories that never reach the masses, but having AI bring that up through conversation would be a more immersive way to expose players to that narrative than through walls of texts or random pickups.

As for the acting, there’s a long way to go before AI can capture the nuance and subtlety of human performances. Many of the character’s performance felt flat compared to human voice acting. It’s notable though that the characters that stood out in my mind were the confrontational ones. Diego Martinez and WENDY the robot left more of an impression than the polite and nice NPCs.

Although having a game full of Neo NPCs seems a little daunting or far in the future with amount of storytelling involved, it could be good and doable in spurts. It would be a way to bring more narrative depth in the down moments of a campaign. It would create more compelling moments that match the realism that players see in the graphics.