Muti Randolph Creates Synesthetic Experience in New Show

In neurology, synesthesia is a mental state in which different senses are perceived concomitantly—a blending of the senses. When it happens, sounds can be related to smells, or even colors. Muti Randolph's new work Sinestesia explores that concept with an audio-visual recital in which images are controlled by the sound of a piano.

On the piano is Clara Sverner, Randolph’s mother, who plays a repertoire of compositions from the 19th to 21st centuries. Although it's not at all different from a traditional concert in terms of sound, it's from the sonic aspect that visual projections are generated, creating the spectacle.

A system specially developed for this show draws real-time images from the sound of the piano and key movements. Projections are supported by acoustic and visual features, combining the piano's mechanical technology with the projection’s digital technology.

"A sensor installed in the piano converts notes into a real-time MIDI signal and parameters are processed by customized openFrameworks software. It's the best of two worlds—an acoustic instrument with the original timbre and the crystal clear digital signal to generate graphic images," says Dimitre Lima, who created the notes-to-images conversion system.

For each key, there's a sensor that turns notes and the intensity with which they are played into parameters that decide the form of the visuals, all occurring on the fly. The result is an experience that stimulates multiple senses using the same patterns.