Scientists Created a ‘Star Trek’-Style Holodeck for Animals

To test how animals react to new environments, scientists created a holodeck—a virtual reality room where images can be projected on the walls—straight out of Star Trek.

According to Andrew Straw, professor at the University of Freiburg and head of Straw Labs, scientists have long wanted to test how animals perceive their environment and respond to it. But there were some drawbacks to the existing testing methods—if you put animals out in a real-world environment, researchers would have less control over certain variables, like what the animal is looking at. Meanwhile, restrained virtual reality systems disrupt some of an animal's senses—for example, if the animal's eyes detect movement, but their inner ear doesn't (because the movement is simulated), the experiment doesn't work as well.

GIF: https://strawlab.org/freemovr

This led Straw and his team to design a holodeck, a room where images are projected onto the walls and floor, that would allow them to test how animals like fish, flies, and mice (three species that are often used in neurobiology research) react to their environment. Their proof-of-concept, a new tool to study spatial cognition, is described in a new study published Monday in Nature Methods, and Straw said the gaming industry is to thank for the tech they used.

In one experiment, they had a fish swim through a tank where Straw and his team projected aliens from Space Invaders. These animated creatures were moving using the Boids algorithm that creates swarm movements. Straw said the Space Invaders were chosen to push the boundaries of what their system could do by showing something highly artificial.

"And honestly, partly because it's fun," Straw said.

Another experiment included a fish that would follow the movements of a virtual fish that was swimming around the tank. Straw said co-author Iain Couzin predicted this was possible back in 2005, as long as the virtual fish would pause and make sure that they were being followed by the real fish, that believed the simulation was real.

"It allowed us to confirm theoretical prediction from 12 years ago that otherwise would have been very hard to do with other kinds of technology," Straw said.

The researchers also used it to test how mice react to tall heights—they prefer being around shallower depths—and how flies change direction based on the perceived obstacles in front of them.

Straw said that the holodeck from Star Trek was always an inspiration for him, and that there's more appreciation for this kind of tech in research today.

"Ever since I first saw it, I just thought, I want one," Straw said.

The team used gaming graphics cards, projectors that specialize in gaming, due to their low latency, and commercial machine vision cameras to project their images in the rooms and properly track the movements and direction of visibility of the animals.

GIF: https://strawlab.org/freemovr

"I am very grateful to the computer games industry. Our entire research program would basically not be possible without gamers having created a market for most of the equipment that we use," Straw said.

This type of holodeck wouldn't work on humans, since our vision is stereoscopic: We use information from both our eyes to determine depth more accurately, as opposed to monoscopic vision in these types of animals. But Straw said that the holodeck could be used to test spatial awareness in animals.

Last June, scientists found that certain neurons in a fly's brain light up depending on what direction they're going, in a way that Straw compares to a compass, and the holodeck could help with further tests on how that works.

And hopefully it's a step in the right direction to finally be able to play Velocity one day.

Get six of our favorite Motherboard stories every day by signing up for our newsletter.