Insects With Tiny Glasses Could Help Robots See in 3D

Neuroscientists are giving praying mantises 3D glasses, in research that could help give robots 3D vision. There are many things about this idea that I love, first and foremost that A) the little dudes look rad in their chill shades, and B) it’s actually a worthwhile use of 3D glasses, albeit teensy tiny insect ones.

3D almost always seems to be more of a gimmick than a functional addition, but there are serious reasons behind the Newcastle University team’s project to kit out praying mantises with mini 3D specs in a mini insect cinema (h/t Huffington Post). Project leader Dr Jenny Read from the university’s Institute of Neuroscience spoke to me from Japan, where she was visiting another praying mantis lab, and told me that these particular insects are of great interest to our understanding of stereo vision—i.e., the ability to see in 3D.

“They’re the only invertebrate that we know have 3D vision,” she said, recalling work by neurobiologist Samuel Rossel in the 80s, in which mantises were receptive to the visual effect created by putting prisms in front of their eyes to bend their line of sight and make an object look closer than it really is.

Read hopes to find out more about how this 3D mechanism works, which might also give insight into the human equivalent. “The great attraction of an insect is it’s a much simpler system,” she explained. As the number of neurons in a mantis brain are many orders of magnitude fewer than in a human brain, the mechanisms that allow them to see in 3D should be easier to understand.

To study the insects’ vision, her team are giving them 3D glasses much the same as the kind you’d get in the cinema; they’re actually cutting out the lenses of regular 3D specs into tiny mantis-sized ones, and attaching them to their heads using beeswax. Then the mantises get their own cinema viewing. In one example, they’re shown a dot moving around on the screen at different apparent depths, mimicking a bug, and the researchers observe their “strike” reactions. The idea is that the mantis strikes when it thinks the digital prey is close enough.

They’ll also try out a camouflaged “bug” to see if the mantises can discern it from the background. That’s a key feature of your human 3D vision—you can see things standing out that you wouldn’t be able to make out with one eye—but we don’t know if the insects are capable of the same. “It seems like it would be sensible for them to be able to because they have to hunt camouflaged prey, but nobody knows whether or not they can,” said Read. Knowing that would help understand how their “stereo algorithm” works—the computations going on in their brain that put the images seen by each eye together.

The potential applications of that knowledge is where robotics fits in. There are clear reasons robots can benefit from 3D vision, as they need to be able to sense depth to do many useful tasks. Read pointed out that the Curiosity Rover, for instance, has 3D cameras to help it understand the Martian landscape.

The main question to answer is whether mantis 3D vision works essentially the same as human 3D vision or not. If it does, that tells us interesting things about evolution, as we know that insects and humans evolved 3D vision separately. “We know that insect 3D vision has evolved completely independently from human 3D vision,” she said. “If that’s happened over millions of years of evolution, that must be a really good solution, so any machine 3D vision that we create for robots and so on should work the same way, because it’s clearly by far the best.”

If they find that mantises actually have a different way of seeing in 3D, that’s also pertinent to robots, because it’s likely to be a lot simpler than what goes on in our brains and therefore offer an easier and cheaper solution that’s not very difficult computationally.

“Most machine vision at the moment is based on and inspired by human vision,” Read said. “But insects offer a completely different way of approaching that.”