This story is over 5 years old

This Fish's Movements Control The Sound And Light Of A Live DJ Set

Call the fish DJ Don't Tap On The Glass.

by Zach Sokol
19 May 2014, 2:00pm

Watching a fish mindlessly float around a tank is hypnotic enough, but combine its motions with an electronic music system where sound and light respond to its path, and we're talking about whole new levels of aquatic trance. Enter: Dag (fish) is a DJ an installation where a fish's motion is optically tracked and translated into music gestures and visualizations. 

The project was created by Syndrome, an electronic-rock band based in Israel and London that focuses on experimental live performances, concretic sampling, and programmed synthesizing. An early version of the installation debuted in Israel in February, 2013, but Shaltiel Eloul of the band explained they didn't expect the fish part to catch people's attention. "We thought that the fish would only be an enhancement to the experience," she said in an email. "But it turns out that since it was so unusual to see a fish control music, people just stood close to the stage and watched the tank." Thus, the project was reorganized, and now the performance is aqua-centric. 

It works by using Max/MSP/Jitter, where the group developed three programs that control three parts of the installation. The first moderates the image processing that reads the fish's movements and translates it into a position, velocity, and acceleration. The second, handles the music translation and digital signal processing (DSP), while the third controls the visualizations of the LED-screen on the back of the aquarium. So, for example, the notes and volume of the music are triggered according to the fish's acceleration, while the pitch relates to its position in the tank. Furthermore, the fish paints sound triggers on the screen as if the LED screen was a drum machine that responded to the fish's position. 

Eloul explained the fish likes to follow and chase some shapes in the visual screen at the back of the LED, but since the visuals are synchronized with the music, it leads to feedback reactions. The music created by the fish interacts with the visuals and the visuals interact back with the fish. "In other words, the fish can be aware of what he does to the music, but probably not directly from hearing the music."

The team sent this project to scientific journal, Leonardo, where it is currently under review. "we think that this type of application for live interaction is unique in the way we've combined visual interpretation to 'sonification' methods of tracking entities," said Eloul. "It makes it possible to use information of the motion to create advanced and more complex music ideas," as well as engrossing performance pieces. We can't even begin to fathom what a massive set-up at Sea World with a killer whale would look like. The synthesis of animals, interactive audio-visual experiments, and music is certainly something we'd like to see more of. Just don't tap on the glass. 

Images courtesy of Syndrome. Visit their site here for more:


A/V Performance Deliquescent Revisits The Relationship Between Analog And Digital

Ryoichi Kurokawa's Stunning Audiovisual Sculpture

The Phonotube Turns Light Into An Instrument