Tech by VICE

Bless the Man Who Trained AI to Identify Pokémon Type

Who needs a Pokédex?

by Jordan Pearson
Mar 16 2017, 3:26pm

Computers are slowly degrading humanity's special-ness, beating us at some of our most challenging games like Go. Hell, even videogames and poker have been mastered by complex algorithms. It was only a matter of time before they came for that most cherished human ability: correctly identifying Pokémon .

Yes, machines now have the ability to study and train until they can look at a Machop and tell you with some certainty that it's a fighting-type Pokémon. This isn't an easy task, since there are hundreds of Pokémon and 18 different standard types. The secret to this ability is a technology known as neural networks.

Neural networks are computer programs made up of "layers" of connected nodes that run semi-random computations on input data like images, and rearrange themselves until they "learn" how to recognize the objects in them. This is called "training," and once the process is completed, the network should be able to recognize unfamiliar objects that are similar to the ones it's been trained on already.

Read More: When AI Goes Wrong, We Won't Be Able to Ask It Why

Brazil-based independent researcher Henrique Soares thought that seeing how this technology did in a game played by millions of children would be an interesting test of its image recognition abilities, and further his own skills with the tech. And it's fun.

"I find that reading books, and even solving some classical problems [...] are awesome tools to learn something," Soares wrote me in an email. "But only by applying them to a new problem (even if it may be a silly one) am I really able to turn this knowledge into a real world skill." Fair enough! So, how'd he do it, and how'd it do?

The first step for Soares was assembling a database of Pokémon game sprites for the network to train on. He gathered thousands of character sprites from the first five generations of games, covering hundreds of creatures. After visually optimizing the sprites and annotating them by type, Soares earmarked 20 percent of them for training purposes, and the other 80 percent for testing.

Correctly identified Pokémon. Screengrab: Journal of Geek Studies/Henrique Soares

The computer could recognize a Pokémon's type just by looking at it with 39 percent accuracy overall. So, better than a random guess, but not by a whole lot. If you look a bit closer, however, you'll see that the AI was highly accurate with some individual Pokémon—upwards of 90 percent certainty—but with others, it had a bit more trouble. The algorithm was also better at recognizing some types than others. The network correctly recognized dark-type Pokémon like Meowth 92 percent of the time, but ghost-types just 17 percent of the time.

The low overall accuracy could be attributed to the rather small training set of a few thousand game sprites—the best neural networks train on millions of examples.

You might be wondering what the network was "looking" for when to decide a Pokémon's type. It's possible that the network was looking for things like colour, since, for example, many water-type Pokémon are coloured blue. But this is where the really interesting thing about neural networks comes into play: We don't really know how they work in fine-grain detail. Thanks to some newer visualization tools, Soares was able to peer into the network's layers to see which sprites "activate" individual neurons.

First layer activation of the neural network by a Squirtle sprite. Screengrab: Journal of Geek Studies/Henrique Soares

"We have a kernel that is clearly firing up over red regions of the sprite, which leads us to think that the network is learning to discern colors, [and] associating it to the types," Soares explained over email. "But as we go deeper in the network and further from the original image those interpretations become harder and harder to make from simple inspection."

If you would like to assist Soares in his work, you can check out the project's GitHub page. You know, if you're the type.

Subscribe to pluspluspodcast , Motherboard's new show about the people and machines that are building our future.