Now You Can Hear What Joaquin Phoenix's Face Sounds Like

The 'Efflorescence' sound machine has the power to turn the colors of films into music.

|
13 April 2016, 3:10pm

Images courtesy the artists

Efflorescence is the process by which salt rises to the surface of a body of water: that body water evaporates, leaving a coat of salt on the now dried wall. It's also the name of a sound machine developed by three students at the Copenhagen Institute of Interaction Design (CIID) that generates music based on the colors of each frame in a movie.

Inspired by filmmakers' use of colors to evoke certain emotions, the Efflorescence team (Daan Weijers, Luca Mustacchi, and Monika Seyfried) decided to explore this capacity through a different medium: sound. As described in the project’s description, each color has is assigned its own sound. The hue or tint of that color determines the base scale of a sound; the saturation measures the pitch; and the brightness establishes the volume at which it’s played.

The Efflorescence sound machine analyzes single frames and processes them until it generates 16 average colors in that frame, each with their own distinct sound. Those 16 individual sounds are then programmed into a 16-note step sequencer which plays through each sound, one by one, to a networked beat.

It’s sort of like scoring a movie by using the colors in the film itself. The sense of sight is simultaneously perceived through sound. Thus, Efflorescence creates an artificially constructed of synesthesia. Each color is processed, and then projected onto this leafy, three-dimensional sculpture, creating “an enchanting floral-like pattern,” similar to Steven T. Wong’s geometric kinetic sculpture, Memesis.

If you’ve ever been curious about what synesthesia feels like, this device may satisfy your wandering mind. Check out some stills of Efflorescence in action below:

You can head over to the project’s website to check out more upcoming works by the students at CIID, here.

Related:

A Virtual Reality Experience Gave Me Synesthesia

Experience Beethoven's Synesthesia with Oculus VR |

A Miami Skyscraper Gets Synesthesia

Experience Synesthesia In Animated Form