The Earth constantly moves, yet few people hear or see the results. At the monolithic multimedia installation Earthworks, which debuted last week at Sónar Festival in Barcelona, the art duo semiconductor attempted to immersively replicate the sights and sounds of these geological processes.
Semiconductor’s Ruth Jarman and Joe Gerhardt positioned their gigantic, five-channel, computer generated animation inside an immense vaulted space. In Earthworks, various types of natural sounds generate ever-evolving masses and strata of colors. The visuals, which are very much like waveforms in shape and movement, undulate and ripple in both frenetic and slow rhythms. And like any great monolith, Earthworks overwhelms in the best possible with with its sonic and visual beauty.
At Sónar, Jarman and Gerhardt spoke to The Creators Project about the conceptual process behind Earthworks. In addition to talking about creating from a global network of seismic data, Jarman and Gerhardt spoke about giving sound a powerful physical dimension.
The Creators Project: Earthworks is quite a large multimedia installation at Sónar Festival, composed of a variety of elements. So, how did it originate or come together?
Ruth Jarman (RJ): Well, we were invited to propose an idea for a commision. I think there are three artists invited to propose for SónarPLANTA, which is Sónar and Fundació Sorigué. And the idea is particularly around nature, technology and science, which are three areas we’ve been working in for about 20 years. So the idea was that we would go visit a quarry in La Planta, Spain, and to be inspired in some way by it. The face of the quarry where they’re digging away all of this aggregate to make roads you can really read all of these layers that have been laid down over thousands of years. So we were interested in how scientists read this landscape—the tools and the processes they use to create an understanding of that.
We were also really interested in the analog laboratory modeling at the University of Barcelona. They create these models that look like fish tanks, where they put in these colored layers of sand and apply pressure to replicate different types of tectonic motion with a hand crank. So you get a type of timelapse of how the landscape is formed, and they can do it quite accurately. You end up with all of these colorful layers, so the computer-generated animation we made is visually based on this as a technique, and then we’ve worked with seismic data.
How was the seismic data incorporated?
Joe Gerhardt (JG): We’ve been working with seismic data for quite a few years. We’d always used it to make sound, so the seismic data comes in as a direct waveform from a network all over the world with different types—volcanos, earthquakes, glaciers. This online network basically records every type of earth movement. It made sense to think about the sound the earth makes when it’s moving and being shaped. So we took the seismic recordings and then made them into a soundtrack that then generates and controls the images.
RJ: So it’s kind of a like a timelapse of the Earth forming over thousands of thousands of years. We’re not working with MIDI to allocate different sounds to different numbers. We do a direct translation.
JG: Yeah, it’s not sonification, which is kind of a buzz word where everyone takes data and turns it into sound. This is literally the sound of the earth.
RJ: It sounds very natural when you listen to it. A lot of people think it’s a glacier or an earthquake when they listen to it, but it’s not. It’s digital information that’s been recorded. It’s a sequence of numbers that then become a waveform, and because it’s a waveform, you can translate it into sound.
How does the software turn the data, the numbers, into sound?
JG: The main tool is the thing that scientists use, which is called MATLAB [Matrix Laboratory]. It’s online and you put in the location, instrument, the frequency and the timeframe and then it all comes down as a [data] packet. And the program then takes it and shapes it into a waveform. All that we do is save it off at 48 kHz, which is CD quality and that kind of defines the speed at which we hear it because then it naturally sounds good quality. Most of the sound in the installation is at that speed, mainly because it has all of the dynamic frequency of a beautiful recording. It has weird high-end squeaks and really rumbly physical low ends, and it’s just a perfect natural translation.
When you say that the sound generates the visuals, should that be taken to mean that the visuals are generative in some way?
RJ: To a certain extent.
JG: There are many layers and patterns of noise and these are all interfering with one another, and then the sound is attached to animate these. So the sound is then determining how the visuals interfere with one another.
RJ: We’ve developed a unique tool that will do the things we want it to do, but then it doesn’t move until we put in the sound. So the sound controls the visuals, but obviously we set the parameters to move or looking in a particular way.
What kind of tool are you using to link sound and vision in the piece?
RJ: We work with Houdini software, and then we worked alongside a programmer to make this specific tool. So we knew exactly what we needed it to do. When we work with CG animation, 90% of the project is building the tool and getting it to the point where it does exactly what you need it to do, and it behaves how you want it to behave.
When working with these seismic data sets, were there particular sounds that were more satisfying than others?
JG: I think it’s when they became more physical actually, so when they have a more dynamic range. Something like the earthquake has a lot of bass in it, and these huge four subwoofers [at Sónar] make the sound really physical. For us, we’re interested in the material nature of things, so that includes what sound is so that sound can be felt.
This is an important thought or approach to sound, I think, because sound—at least when it comes to multimedia experiences—is so often relegated to a second-class citizen role. Because there is such an emphasis on the visuals, people tend to forget that sound can have depth and a physical force.
RJ: The sound has always been as important to us as the visuals because a lot of our work is dealing with data, and quite often we’ll turn that data into sound and image so that one cannot exist without the other. The idea is to see and hear them at the same time, so they’ve always been on an equal footing in that way.
JG: Earthworks is a cross between sound art and land art, is how I like to think about it.
Ideally, what sort of effect did you want it to have on audiences at Sónar?
RJ: I think there are always two elements of that in our work because always make work that you can experience and enjoy on many different levels so that you don’t have to know the concepts. It’s conceptually driven, but it’s not just about concept—it’s about experience as well. So you can experience it, but then I think once you know that information, that it is seismic data and it’s about landscape formation, then it enables people to almost step outside themselves.
There are four different sections of data: earthquake, glacial, volcanic and urban data. So we’re thinking about how man is starting to affect the planet. To start thinking about ideas around the anthropocene (the geological age of human activity), really you need to step outside the very day and how we think about time and space. That’s a really big ask of people. Often the work feels quite overpowering and humbling, and some people can find it quite overwhelming and scary because it’s do to do with the power of nature. You begin to understand that nature really is powerful and you can’t control it. So I think we kind of push people toward that.
There’s another side as well. There’s kind of a more nerdy side that is thinking about science and about science as a process and a language, and kind of saying to people that you are allowed to question science. Because a lot of people think that science is giving you answers to things, whereas science is giving you questions.
JG: Science is the question.
Earthworks was on at Sónar Festival through June 19.