MIT Uses Video Games to Create the Smartest, Scariest Computer Ever

Remember how much of a pain SimCity was? Now imagine trying to play it in Russian. Researchers at MIT have created machine-learning systems that are able to analyze instruction manuals to help them beat video games. By correlating terms they come across in gameplay with those found in the manuals, these MIT programs can teach themselves human language in order to win.

The concept started with work by graduate student S.R.K. Branavan, working in the lab of Regina Barzilay, an associate professor in the computer science department at MIT. Branavan’s group created a script that installed software in Windows by referencing instructions from Microsoft’s website. Since then, they’ve taken it further, creating a program to play the turn-based strategy game, Civilization. The program has been a success too, winning 79 percent of the time with the help of the manual as compared to a near-random 46 percent win rate on its own.

“Games are used as a test bed for artificial-intelligence techniques simply because of their complexity,” Branavan said in an MIT press release. “Every action that you take in the game doesn’t have a predetermined outcome, because the game or the opponent can randomly react to what you do. So you need a technique that can handle very complex scenarios that react in potentially random ways.”

A computer playing games is nothing new, whether you’re battling the machine in Mortal Kombatw_CYrrk_ or chess. But in those instances the computer is always reactionary, at least in a sense. They’re programmed to play based on an algorithm devised by a human, and no matter how cleverly that algorithm is designed, the computer is stuck with those rules. If you’ve ever been bored enough to have the computer play itself in, say, a hockey game, the play is generally sloppy and the outcome is mostly a coin-flip, just like the MIT program only winning half its Civilization games when it couldn’t search for help.

The winning program was more successful because it was actually able to learn language. Rather than click haphazardly around the Civilization screen, the program looked up terms it found throughout the game in the manual, and kept a record of what happened when it clicked on it. Also, when looking up a term, the program analyzed the passage around it, creating theories that were either given weight or rejected based on their outcome. As time went on, the program was able to decipher more and more. In essence, it taught itself the language, and then used its increasing understanding to build increasingly refined game strategies.

While creating superpowered video game opponents may be one spin-off of the research, the real mind-blowing takeaway is that the MIT researchers have created a computer program that can act without relying on a set script. By being able to analyze the data found while interacting with its environment, this program is able to teach itself. This has a huge application in the artificial intelligence field, and in fact Barzilay’s lab is already working with robots.

Which is scary when you think about it: a program designed to dominate a game based on the entirety of human history may now find itself a body? I think that’s your apocalypse recipe right there, folks.