Scientists Are Building VR Worlds for Mice and Monkeys

Researchers now have the capability to build virtual reality worlds that can be inhabited by both lab animals (like rats and monkeys) and humans, allowing for a sort of cross-species brain research that's never really been possible before, as different species are run through virtual environments in video games while scientists study brain activity.

Neuroscientists often work with lab rats and other animal models to learn about the brain, but their findings aren't always applicable to humans, explained Julio Martinez-Trujillo, who's based at Western University's renowned Brain and Mind Institute in London, Ontario. For one thing, humans can't be placed in a maze like lab rats are: you'd need a space the size of an entire building, and monitoring their brain activity just wouldn't be practical.

"A challenge in neuroscience is how to bring animal research closer to human research," Martinez-Trujillo told Motherboard. "We decided to try to solve it by using tools that are available in the video game world."

In a new study, published in Journal of Neuroscience Methods, a team of neuroscientists—including Martinez-Trujillo, Roberto Gulli and Guillaume Doucet, who are both also affiliated with McGill University—describe a new virtual reality "toolbox" they've designed, which can be used to build video games and virtual worlds for brain research in humans and animals.

Video games and virtual reality have already been used in a limited number of lab animal studies, said Doucet, who has a background in 3D animation. But in those cases, "the VR was specifically made for mice," and so the scientists' findings didn't really translate to other species. Most neuroscience experiments use MATLAB or Python, he continued. "These are not made for 3D animation or VR experiments, so I created a way to take a video game engine, and have scripts that could interface with already existing experiments," he said.

In other words, they build a bridge that will allow video game frameworks to create 360-degree, VR environments for detailed studies in animals and humans.

A monkey playing a video game. Neural activity from the hippocampus is recorded as the subject navigates through a VR world. The black dot indicates the monkey's eye position on the screen. Credit: Cognitive Neurophysiology Laboratory/Western University

Doucet and his team used a video game development framework called Unreal Engine 3, which is used in plenty of big-budget, commercial video games. That's what "handles the virtual environment," Doucet said, and makes sure it looks realistic: that a shadow lands in the right place when an object is moved, for example.

"We created a translator, or an interface, between [our] experimental programs, and the video game language that works in the virtual world," explained Gulli.

While the toolbox hasn't yet been used for studies in animals and humans, the team has done some work with monkeys, which is still unpublished, Gulli said. "We used the VR toolbox to create an environment in which monkeys had to navigate through a virtual world," he explained, "and create memories within it."

As these monkeys explored the maze, they had to look for cues to help them figure out what to do: for example, the colours of the walls might change to tell a monkey to turn left or right.

Monkeys aren't wearing VR helmets, Gulli said, "although it is programmed in such a way that they could put on [3D] glasses." Instead, they're sitting in front of a computer screen, and navigating with a (reinforced, monkey-proof) joystick. "It's pretty intuitive to them," he said. "They can learn to use these joysticks on the timescale of minutes." According to Martinez-Trujillo, they're hoping to implement a human version of the study soon.

Neuroscientists are still learning how humans and other species respond to virtual reality, but research done in the virtual world does seem to tell us a lot about how we respond to the real world, too. Now, we'll be able to know what happens in the brains of lab rats and other species in these environments—and how it compares to humans.

UPDATE: This piece has been updated to reflect the fact that Rob Gulli and Guillaume Doucet are both affiliated with McGill University, as well as Western.