How Haptics Make Holograms You Can Touch

Hiroyuki Shinoda pioneered haptic holograms years ago, and now he thinks the world is nearly ready for this tech.

May 11 2015, 5:00pm

Image: Shinoda Lab, University of Tokyo

A small gecko splashes through water in a bowl and skitters across a researcher's arm. The imprints and effects it creates seem so realistic you'd be forgiven for thinking it was alive. But this is no real gecko. It's a visually projected holographic gecko that haptically conjures the illusion that the real thing is running over you.

Haptics is a fun form of interaction which creates the impression of tactility. The tech works when small devices called actuators—providing a mechanical motion to an electrical stimulus— apply forces to the skin in order to conjure the sensation of touch. In the gecko's case, a "visuo-tactile projector"—consisting of 91 ultrasound transducers above the researcher's arm—emits ultrasound radiation pressure onto the user's skin, making them feel like the gecko is crawling over them.

The field of haptics is making waves in both gaming and real worlds, as creators aim to deploy it to make virtual and holographic objects more tangible.

While scientists have been working on haptics in their labs for some time, we've only recently seen the tech start to take off. According to haptics pioneer Hiroyuki Shinoda, who developed the gecko setup among other early haptic interface examples, it's been impossible to design practical haptic devices at reasonable costs until fairly recently. He explained that while scientists worked with haptics in labs ten years ago, limited funding and investment slowed down the impact and applications of this technology.

Shinoda is a professor at the University of Tokyo, and started working on 3D haptic holograms in 2008. As well as the 3D holographic gecko, his creations include 3D holographic raindrops that fall onto a human hand, floating tactile touch screens, and haptic switches intended to replace their physical counterparts.

To create these works, Shinoda pulled more than a decade's worth of research together to "enhance the reality of a hologram." To conjure the falling raindrops effect, Shinoda combined Wii Remote sensors—a controller for Nintendo's Wii console with motion sensing capability—to track a user's hands, with an ultrasound radiation pressure system, which conjures the sensation of pressure on bare skin.

Over email, Shinoda told me that he wanted to use ultrasound radiation pressure to create tactile simulations way back in the 1990s when he was working on his master's thesis. Back then, Shinoda was convinced that haptics would have far-reaching applications in communication, entertainment, and healthcare, seeing as haptic perception is included in our five senses and thus integral to human activity.

But when Shinoda launched his touchable 3D holographics in 2008, the technology was still in its infancy. "Some even doubted the effectiveness of this technology and we needed several years to polish it, and for everyone to agree on its usefulness," he told me. But now the haptics field is really getting going, he noted.

To give a couple recent examples, just last month Rice University's student engineers developed "The Hands Omni", a prototype VR glove that could make gaming more visceral by allowing people to "touch" objects within their simulated worlds. And last December, researchers at Bristol University's Interaction and Graphics group concocted some invisible yet touchable 3D holograms using focused ultrasound.

It's not just for fun. In terms of practical applications, Shinoda is still working on floating, tactile 3D holographic panels, which he hopes will be used in hospitals. Physical switches, touched by scores of people each day, can be vectors of bacteria and disease. However, Shinoda explained that if these could be replaced by tactile 3D holographic ones, people would still retain the impression of touching something, but without spreading or picking up any harmful bacteria. "The merit of these devices is that they are free from dirt and infection. An aerial touch interface is even erasable when unnecessary," he said.

Shinoda has also worked on manipulating what he dubbed an "elastic" 3D holographic image, which users can use their fingers to squeeze and change shape. He wants to develop this further. "We are polishing this technology so that it can be used as a practical 3D design interface where the designer can handle 3D virtual objects." Once the cost of the tech is lowered, any designer would be able to mould, amend and preview their designs through a tactile 3D hologram display before actually making the real thing.

Imagine reaching out and touching the virtual reality world that you see through your Oculus Rift. Or feeling yourself turn off a holographic switch. If you could both touch and see the physical objects around you in your virtual world, or touch the holographic ones in the real world, wouldn't that add an extra dimension to experiences in both real and simulated worlds? So much so, you might even forget that what you were experiencing was just a mirage.