Late last year, Amsterdam-based creative coder and motion graphics designer Sander Sneek dropped Soundscape, a synthesizer that lives in virtual reality. Even now, it’s one of the exemplary efforts towards placing sound on equal footing with visuals in a virtual reality world. But Sneek isn’t resting on his laurels. He’s taking two new pieces of VR gear, the Oculus Rift CV1 and the Leap Motion Orion hand-tracker, and integrating them into Soundscape so that people can use their hands to play the virtual synthesizer.
“It is still a work in progress,” Sneek tells The Creators Project. “Soundscape started as a tech demo last year. I initially started the demo with the Leap Motion, which is capable of tracking your hands. I believe truly immersive VR starts with your hands.”
“Leap Motion just released their new Orion Beta SDK, which is a massive step forward for technology's capabilities,” he adds. “Lower latency, longer range, better and faster hand recognition, vastly improved robustness to cluttered backgrounds and ambient light, and more. Combining the Leap Motion with the new Oculus Rift CV1 creates an amazing music VR experience.”
Judging from Sneek’s footage, Soundscape’s hand-tracking already looks impressive. The overall hand gestures are smooth, and the finger tactility affords users the ability to manipulate the monolithic sequencer’s synth notes and drumbeat, then go about tweaking filter cutoff and oscillator waveforms.
“Manipulating the synthesizers and filter effects via hand-tracking is a real joy,” says Sneek. “Playing around with the cutoff and resonance effects just by moving your index fingers keeps me (and my kids) really amazed. This is the purpose of Soundscape—play with music in a joyful way.”
“The tracking is pretty much spot on and therefore gives you a real sense of presence, of actually being in this gigantic sci-fi environment,” he adds. “The representation of the hands is done as abstract holographic versions, trying to avoid the uncanny valley.”
Click here to keep up to date with Soundscape's progress, and to see more of Sander Sneek’s work.