How Queasy Games Turned Beck's Music Into A Video Game

Back in August Sound Shapes was released on the PlayStation 3 and PlayStation Vita by Queasy Games, the result of a collaboration between game creator Jonathan Mak (founder of Queasy Games) and musician Shaw-Han Liem (aka I am Robot and Proud). Mak is a game designer who has a deep interest in music and music composition and Liem is a musician who has a keen interest in visuals and interactivity.

Before Sound Shapes they worked together on projected visuals for live bands. They’d convert the music into generated procedural visuals to show the audience another way of visualizing what's happening on stage. From this they went on to discuss the potential of a video game based around visualising, or seeing, music.

Sound Shapes works within the conventions of a platform based environment where players compose a soundtrack by playing through different levels. The game has both a campaign side and a level editor where you can create your own compositions (levels). The idea was to to empower people to do music creation, not in a simulated way like in Guitar Hero, but where the authorship was real.

Mirroring music with the game world seemed like the natural thing to do. “If you showed someone a drum machine or sheet music it has no connection for them, it's abstract.” says Liem. “Instead of musical notes it's coins, instead of a bar it's a screen of a world, instead of a sample playing it's this entity that's playing this loop that you always hear when you see it—so it’s about making these connections between the game ideas and the music ideas. You put people in a mindset where they're not intimidated by it and it's this fun experience to sit there and manipulate things but by doing that they are essentially doing the composition.”

Some of the levels in the game were designed using tracks provided by Beck, namely “Cities”, “Touch The People”, and “Spiral Staircase”. Below, Mak and Liem talk us through how they went about translating Beck’s music into the different visual components of a platform level, turning the sounds into interactive gaming entities.

The finished level for Beck’s “Cities”

How The Ideas Are Born

Jonathan Mak: It's different for every level, a lot of them started with the level first and then we started to put the music in. The Beck stuff, because we had the songs already written it was sort of like the reverse. We sit there and work out what the entities would be, just listening to the song over and over again trying to come up with a concept. A lot the stuff is instinctual, when I hear it I can kind of see things, what these things could be. With "Cities" we started digging into the lyrics of it, breaking down the lyrics thinking what it could be about.

Shaw-Han Liem: I think one thing that was important, in the Beck example, when we receive the music we get the songs but we also get the Pro Tools sessions too. So we have the whole recording session and we have all the component parts of the music and that gives us a lot of flexibility in terms of what we can do with it. But it also allows us to literally deconstruct the song, isolate elements that may not've been super prominent in the mix, so maybe we can attach other things to it. Being able to dig deep into the musical content allowed us to think about the music in terms of all the various pieces that put it together, and then try to attach those to things that might exist in a game world.

The two videos above show how the “Cities” level for Sound Shapes developed

Providing The Gaming Elements

Liem: The general idea of introducing ideas in a way that is digestible is a big part of the design consideration, both the game and the editor—essentially one of the things we're trying to do is to convince people that they might want to compose music. They didn't necessarily buy the video game because they wanted to compose music, but part of what we tried to do is create those connections and introduce those ideas in way that will encourage people to explore that side of it.

In “Cities” the intro to the song had this kick and snare pattern, this shaking percussion and then a bassline that comes in, so basically the main riff of that song is the bassline that comes in and determines the whole structure of it. So we'd take those four things—the kick and snare pattern, something shaking around, and the bassline—and they need to be introduced in this sequence to create that musical idea.

And you say OK, if I was walking through a world that sounded like this, what would I be seeing? So the bassline becomes missiles and the shaker becomes the flickering fire stuff and the kick and snare become the collectable notes. So that's just one or two screens of this level, but that process is sort of extrapolated across the game, common to all levels.

Above: “Touch The People” cast list—before they lay out the level, they come up with the entities.

Above: “Touch The People” breakdown written out, where they start to get a rough idea of how the song needs to be laid out in the game.

“Touch The People”

Keeping The Momentum Moving

Liem: Keeping the momentum of the scrolling screen moving can be both natural and problematic. For instance, there are aspects of the songs that connect very well. We often talk about the level as a kind of animated musical score, the arrangement moves rapidly from left to right—you've got your high pitch at the top and low pitch at the bottom—and you sort of move across the song.

So on that sort of high level sense the structure of a game mirrors this kind of musical structure. But then you get down to the specifics of like, how could this drum beat happen? Once you start getting into trying to recreate specific musical ideas, that's where the challenge comes in.

Mak: For Beck, after we have the ideas for what the notes are and what the different entities are going to sound and look like, then we laid the song out on a blank screen, laid all the notes down and all the entities just to recreate the song in the game. We do that and that's reasonably close but then you have to play through it. And when you play through it you experience the music differently, whole parts don't come in at once, you have to gradually pick them up. Also timing, when there's a rest in the song, how you're going to do breaks and transition—all that’s important and needs to be considered in the design.

“Spiral Staircase”

Dealing With Lyrics And Who Does What

Mak: In “Cities” I felt a little bit scared of making the lyrics super literal, because I thought it might be too obvious. But it actually really worked out that way. Lyrics are really hard to work with from a gaming point of view, down to how you physically move across the screen, so you just have to be careful about that. But it doesn't actually have any connection to the music, what it actually sounds or looks like, so we came at it from the point of view of just making it look like how it sounds.

Liem: With regards to who does what, there was a lot of overlap. But in the general sense my focus was more on the musical elements and the design of the system, the editor side which we tried to think of like a musical instrument. Jon did the design of the game and the visuals but, like I said, there was overlap on both sides.

The Finished Article

Mak: In general for this game it took a long, long time. But like with anything really, I think we could've kept working on those levels for ever, refining them. We get to that point where we make a level and we play through it and you think "Yeah I like that, I'd wanna play it again". But every time you play you notice little things that are annoying, play it and fix it, play it and fix it, play it and fix it.

@stewart23rd