We Tried MIT's New AI Software that They Say Can Identify Food in Photos

Over the weekend, Tesla CEO, futurist inventor, and noted artificial-intelligence-hater Elon Musk warned a bipartisan slew of US governors at the National Governors Association in Rhode Island that the threat posed by the continued unregulated development of AI was in fact "the scariest problem" facing humanity. "AI is a fundamental existential risk for human civilization, and I don't think people fully appreciate that," stated Musk.

Do you think Musk—the man who once likened the development of AI to the summoning of demons—was referring to a future in which all the world's culinary recipes were analyzed by a cabal of AI foodie overlords, eliminating any need for cookbook authors, recipe testers, or food media as we know it?

Well, that future might be here. Researchers from MIT just announced they have trained an artificial intelligence system to successfully discern a food item's ingredients and recipe simply by analyzing a photo of said food. The team has fed the AI more than one million existing recipes and 800,000 images of food in the hopes that it will be able to predict a dish's components and recipe simply by looking at a photo.

The bone-chilling nom de guerre of humanity's soon-to-be gastronomic imperator? Pic2Recipe.

To build their database, researchers from MIT's Computer Science and Artificial Intelligence Laboratory and Qatar Computing Research Institute used photographs of food and recipes from the existing "Food-101" dataset, created in Switzerland in 2014, and a database from the City University in Hong Kong, which contained mostly Chinese cuisine. But to expand the requisite data, the researchers turned to websites like Allrecipes and Food.com to create their database called "Recipe1M," which contains over a million recipes. The researchers then used this data to train a neural network to figure out the ingredients and recipes from food images.

Show it a cookie, they say, and Pic2Recipe can now identify ingredients like flour, eggs, and butter. (It might miss the cinnamon, though.) Pic2Recipe is pretty good at cookies and muffins, but still has a hard time with other foods, like sushi and smoothies. But that may be just a matter of time.

The researchers are hopeful that this is only the beginning, and scholars agree. "You can imagine people using this to photograph their meal at a restaurant and know what's needed to cook it at home later," Christoph Trattner, an assistant professor at MODUL University Vienna who did not work on the paper, said. "The team's approach works at a similar level to human judgment, which is remarkable."

Remarkable? Or scary as hell? We're pretty sure we know what Musk would think. In the meantime, you can try out a demo of the AI online by uploading your own images into the database. MUNCHIES decided to put the demo to the test and see just how far along Pic2Recipe is in enslaving mankind with its boundless command over the world's apple turnover recipes.

Turns out, there might just be a few wrinkles that need to be ironed out before Pic2Recipe successfully assumes the life of Ina Garten and treats us all like Jefferies, forced into bloody death matches in the hopes of obtaining a recipe for beef bourguignon.

We decided to start off simple, with some basic, plain-ass French fries. Sadly, Pic2Recipe was unable to find a single recipe. On our second try, using the same French fry photo, we ended up being offered a delightful recipe for apple puff pastry.

Figuring that maybe Pic2Recipe wanted us to cook something a bit more substantial, we tried out a photo of a bowl of pho. Only one result showed up, and while it was for a noodle dish, you probably aren't going to want to learn how to make spaghetti carbonara while searching for a pho recipe.

Hoping to see what would happen when you ask Pic2Recipe for a recipe it almost certainly doesn't have, we decided to try out a Big Mac next. Once again, Pic2Recipe failed pretty miserably. Our Big Mac search netted a recipe for a roast beef sandwich, a barbecue pork sandwich, and some chocolate gelato sandwiches (what?).

Guess the robot takeover is still a few years off. Here's to hoping Pic2Recipe's culinary education proceeds a little more smoothly in the future.