Ubisoft debuts GenAI-powered Neo NPCs and gameplay prototype

Ubisoft has unveiled Neo NPCs, its first player-facing generative AI powered characters, at Game Developers Conference. The prototype allows players to interact with NPCs with distinct personalities and memories that allows conversations to drive gameplay. These Neo NPCs are enhanced with new neural functions allowing them to respond to players with unscripted dialogue, contextual awareness and real-time emotion and animation. This turns conversations into their own form of gameplay that can include information gathering and collaborative decision-making.

A team at Ubisoft’s Paris Studio led this R&D initiative with support from its production technology department. Unlike traditional NPCs, the team needed to program every detail — personalities, backstories, agendas and emotions — for these interactive Neo NPCs. As a result, every conversation is unique and responsive to players’ voice inputs and previous interactions while still maintaining continuity.

“My creative work has become much deeper since I get to work on their (Neo NPCs) psychology. I have to give them a soul and do so with a certain finesse that we have never before allocated to NPC creation,” said Virginie Mosser, narrative director on the project. “As a writer, seeing my characters come to life and actually converse with me for the first time has been one of the most fulfilling and touching moments of my career.”

How Neo NPCs drive emergent gameplay

Ubisoft showcased its Neo NPCs at a private event at GDC. The three-part demo highlighted how players’ voice-to-chat inputs could drive unique and emergent gameplay through conversations.

The first part allowed players to introduce themselves to a Neo NPC that adapts emotionally and react in real time based on player intentions. This can unlock new story elements and lore as players build relationships with this new generation of smart NPCs. This can impact future gameplay as NPCs remember information players relayed to them. For example, the same Neo NPC can leave a note that incorporates these player inputs. Eventually, Ubisoft plans to use this to help generate quests on the fly.

The second part demonstrated Neo NPCs’ context awareness where they can react dynamically with unscripted responses to in-game events. Ultimately, this contextual awareness will allow quest providers, training partners and companions to react to players’ actions, creating more immersive worlds for players to explore and shape.

Finally, the third section allowed players to plan a heist with a Neo NPC. Players can collaborate with these advanced NPCs to suggest alternative approaches, no matter how absurd. Neo NPCs will share insights and ask questions in a collaborative process with players.

“It could be the start of a fantastic paradigm shift,” said Xavier Manzanares, director and producer of the project. “For the first time, the game world actually listens to and dynamically responds to the players. Social interactions and skills become part of the gameplay. Smarter NPCs like our NEO NPCs have the potential to become a breakthrough addition to the traditional NPCs we see in games today. They provide the ability to create even more immersive worlds and emergent stories.”

Ubisoft collaborates with Inworld AI and Nvidia

While Ubisoft spearheaded the project, it collaborated with Inworld AI and Nvidia to bring Neo NPCs to life. Ubisoft’s narrative team leveraged Inworld AI’s character engine and LLM technology to build complete backgrounds, knowledge base and conversation style Neo NPCs. Meanwhile, Nvidia Audio2Face powers these characters’ real-time facial animations.

“AI is a powerful tool that can greatly enhance players’ experiences by making games more dynamic, immersive and unique,” said John Spitzer, VP of developer and performance technologies at Nvidia. “We are thrilled to partner with Ubisoft on this project to leverage Nvidia ACE in the advancement of digital characters.”

Together, these companies are building a tools to support Ubisoft’s narrative-driven gameplay. The company’s expansive worlds in franchises like Assassin’s Creed, The Division or Far Cry could benefit from this conversation-driven gameplay. While the technology looks promising, the team is still developing the project without a specific game in mind for now.

“Generative AI brings unprecedented creative opportunities for our teams and players,” said Guillemette Picard, SVP of production technology at Ubisoft. “It is when they find the value it has for them that we start to see its true potential to transform the way games are made and played.”