This story is over 5 years old.

Here's What Happens When Dancing Meets Creative Coders

DanceHackDay generated 18 original dance performances incorporating emerging technology.
December 24, 2016, 5:25am
Hien Huynh dances with HTC Vive headset and controllers to create projection mapped imagery in a WebGL environment programmed by designer Marpi. All photos by the author

What do you get when 70 dancers and creative coders meet for a hackathon? Chaos, yes, but also 18 compelling two-minute performances using both familiar and emerging new media tools, like full-body motion tracking, livestreamed 360-degree video, projection mapping, machine learning, virtual reality, and repurposed biometric data. This celebration of tactile performance and digital art forms the basis for DanceHackDay, an annual project hosted by Kinetech Arts, which was hosted at CounterPulse in San Francisco earlier this month.


“The hack day is really a way of bringing people from different disciplines—tech, art, dance, music—together,” Kinetech Arts founder Weidong Yang tells The Creators Project. Yang, a trained physicist and dancer who also heads virtual reality data visualization company Kineviz, says DanceHackDay is a way to facilitate creativity amidst disorder. “As a scientist, I never achieved a significant discovery in the direction that I set off in,” he says. “All my major discoveries were from the mistakes, from the things that I didn’t expect to happen.”


Stephanie Sherman and Alexander Diaz perform the first act of Heartbeat Across the World to piano music generated by Sachin Deshpande's program which is based on a real-time feed of biometric data

The variety of performances on display was staggering. In one piece, aided by a DIY heart rate monitor, an internet connection, and a program developed by participant Sachin Deshpande, dancers moved to the sound of their own heartbeats and the heartbeats of counterparts in London and Paris. In Heartbeat Across the World, the pulses took the form of lilting piano notes akin to a minimal, freeform instrumental piece, but the biometric data could potentially be shaped into any number of aural or visual accompaniments.

During the hackathon portion of the event, developers Lauren Bedal and Fabien Lemaitre recorded dancer Stephanie Sherman’s movements in Unity. Sherman hugged, swayed, and leaned on another dancer, whose movements were not being tracked, making it appear that Sherman’s avatar was interacting with an invisible partner. “It’s amazing how much emotion is conveyed just through movement, and you don’t even have to see her facial expression,” Bedal marveled.


Developers and dancers collaborating in the main theater room at CounterPulse

For participants like Dustin Freeman, one-half of digital improv group Raktor, DanceHackDay was an opportunity to riff and collaborate on ideas-in-progress. Working with Hallie McConogue and Jasper DeTarr, Freeman enabled dancers to embody a multitude of avatars in real time using Microsoft Kinect. He sees tools like these as having a major influence on the future of live performance and participatory theater.

“Most VR experiences right now are games or they’re film. They’re not participatory,” Freeman says. “So we’re like, let’s bring this to virtual spaces where the people who are interacting don’t have to geographically be in the same location. Right now, we’re interested in local experiences that include digital participants.”


Stephanie Sherman tests out a full body motion tracking suit which renders her movements in Unity, a 3D game development platform used to create VR and AR environments

While event organizers hand-picked a few participating artists, registration was open to the public, and the diversity of performers reflected that inclusive spirit. “It’s all kinds of people doing all kinds of dance. Some people are doing ballet, some people are doing contemporary dance, some people are doing social dance, some people are doing butoh, and some people are coming from Indian dance,” Daiane Lopes da Silva, Artistic Director of Kinetech Arts, says.

Kinetech Arts plans to organize an expanded heartbeat simulcast, which will include a broader scope of international participants from Lisbon, Berlin, Amsterdam, Seattle, Chicago, and New York City during a two-day event in 2017. The organization hosts an open lab every Tuesday night in San Francisco.


New Cinema Hackathon: Dance with Your Face

Dancer Defies Gravity with Stunning 3D Projections

Heart Bot Turns Heartbeats Into Personalized Illustrations