Researchers Are Training AI to Survive In This MMO

Neural MMO is a new massively multiplayer online game, but humans aren’t invited—only artificial intelligence can play.

In the game, AI agents spawn into an open world and need to gather resources like food and water to survive. Along the way, they’ll encounter rival agents which they can avoid or fight for dominance. It’s a harsh world, designed by its creators to prompt the AI agents to develop strategies that satisfy a task that is both open-ended and highly complex: survival over a lifetime.

OpenAI researchers Joseph Suarez, Yilun Du, Phillip Isola, and Igor Mordatch designed Neural MMO and released its code via GitHub on Monday.

Video games are a great way to train AI because the results are easy to interpret and the goals are (usually) simple—most games have scores, or win conditions. But games also present a problem for training AI to thrive in the real world, which doesn’t have a scoreboard to constantly let you know you’re on the right track. This is why, in Neural MMO, the agents in the game optimize for survival over a long time horizon.

To find a game that was both open ended and complex and give AI a new challenge, the Neural MMO team turned to massively multiplayer online games such as World of Warcraft.

“We consider [Massive Multiplayer Online Role Playing Games] the best proxy for the real world among human games: they are complete macrocosms featuring thousands of agents per persistent world, diverse skilling systems, global economies, and ad-hoc high stakes single and team based conflict,” the team wrote on GitHub.

The world of Neural MMO resembles a crude RPG map. The AI agents spawn on the outskirts of a world made up of tiles representing stone, grass, lava, forests, water, and scrub. They have “health” and the ability to attack at close range for massive damage, at a distance for less damage, and with a freeze spell that stops opponents but does little damage. The agents have to feed themselves and drink water in world with limitations. The water sources provide an infinite amount of water, but the food resources are scarce and respawn slowly.

The game divides AI agents into “species” by having them share parameters. Agents of the same species won’t attack each other. The researchers found that species filled environmental niches, surviving alongside their own and avoiding combat when possible. Interestingly, the researchers note that more of the game map was explored overall when more agents were in competition, trying to avoid conflict.

“Agents learn to explore only because the presence of other agents provides a natural incentive for doing so,” the authors wrote.

Neural MMO is far from finished. According to the researchers, they want others to build on the open source game and essentially release expansions for their MMO—new developers could tweak the rate certain tiles spawn to create harsher environments, make the combat more complex, or give certain species different attacks.

This is all with the goal of creating more complex AI down the line.

“Life on Earth can be viewed as a massive multiagent competition,” the researchers said. “The cheetah evolves an aerodynamic profile in order to catch the gazelle, the gazelle develops springy legs to run even faster: species have evolved ever new capabilities in order to outcompete their adversaries.”

Get six of our favorite Motherboard stories every day by signing up for our newsletter.