Music and sound in Portal 2

June 20, 2011

Do you remember recommending Portal to your friends, or having it recommended to you? There was this little dance where you had to temper your enthusiasm to avoid overselling what was, ostensibly, a puzzle game. “No, no, I mean, it’s really cool — it’s like, you make these two holes, and you go in one, and then…listen, you just have to play it, okay?” The fact that Portal could be spoiled was, itself, a spoiler.

From that perspective, Portal 2 was at a disadvantage from the start. Valve had already played their plot twist card, their novel mechanic card, and their Jonathan Coulton epilogue card; while continuing along those lines seemed like a safe bet, I wasn’t sure if bouncy gel and the like would be enough to support several more hours’ worth of test chambers. At the time I expressed this doubt by asking if Portal “needed” a sequel, though that framing seems odd in retrospect.

Regardless, my fears were unfounded. Portal 2 doesn’t — and, indeed, couldn’t — recreate the tremendous effect of its predecessor, but it’s still fresh and funny and stunningly executed. For more on that I refer you to Mitch Krpata, Kirk Hamilton, Michael Abbott, and Matthew Burns, all of whom wrote wonderfully thoughtful pieces while I was dicking around for two months.

So instead of rehashing those posts further, I thought I would talk a little about music and sound.

From Geoff Keighley’s must-read feature “The Final Hours of Portal 2:

The first Portal was renowned for its musical ending, and in Portal 2 composer Mike Morasky wanted to up the ante with interactive music that would subtly evolve as players completed a puzzle. Run along orange speed paint and the music speeds up. Successfully jump across a ledge and the music shifts to let you know you’re doing a good job. “The puzzles are thanking you for playing with them,” is how Morasky puts it. “They love you.”

Interactive music isn’t new, of course, but I haven’t seen anything quite like Portal 2’s approach before. Sound and music are more thoroughly interwoven, and the distinction between the two is less meaningful.

Film editor and sound designer Walter Murch has a theory about categorizing sound. Using the spectrum of visible light as an analogy, he imagines an aural spectrum with “encoded” sound at one end and “embodied” sound at the other:

When you think about it, every language is basically a code, with its own particular set of rules. You have to understand those rules in order to break open the husk of language and extract whatever meaning is inside. Just because we usually do this automatically, without realizing it, doesn’t mean it isn’t happening. It happens every time someone speaks to you: the meaning of what they are saying is encoded in the words they use. Sound, in this case, is acting simply as a vehicle with which to deliver the code.

Music, however, is completely different: it is sound experienced directly, without any code intervening between you and it. Naked. Whatever meaning there is in a piece of music is ‘embodied’ in the sound itself. This is why music is sometimes called the Universal Language.

What lies between these outer limits? Just as every audible sound falls somewhere between the lower and upper limits of 20 and 20,000 cycles, so all sounds will be found somewhere on this conceptual spectrum from speech to music.

Most sound effects, for instance, fall mid-way: like ‘sound-centaurs,’ they are half language, half music. Since a sound effect usually refers to something specific – the steam engine of a train, the knocking at a door, the chirping of birds, the firing of a gun – it is not as ‘pure’ a sound as music. But on the other hand, the language of sound effects, if I may call it that, is more universally and immediately understood than any spoken language.

What Portal 2 does, I think, is push sound effects and music towards each other so that they meet somewhere in the orange on Murch’s spectrum.

Let’s listen:

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

(You can hear this without the ambient natural sounds on the soundtrack as “The Future Begins With You,” but I recorded it, and the others below, in game for context.)

This piece has what I would call a primitive electronic sound: simple waveforms are spread into octaves-wide arpeggios, with a sparse arrangement and raw timbre. In this way, the music is more effective as sound qua sound than as a composition: its harsh, antiseptic quality reflects Aperture Science’s ethos, and the erratic buzz of the synthesizers evokes the facility’s disrepair after years of neglect. In other words, it leans towards the violet or “encoded” end of the spectrum even though it’s ostensibly musical.

Many sound effects in Portal 2 exhibit the opposite behavior. Here’s a laser, or “Thermal Discouragement Beam,” connecting with its target (most audible at 0:13):

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

And this is the sound of soaring through the air after bouncing on an “Aerial Faith Plate” (at 0:09):

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

(These are “Triple Laser Phase” and “15 Acres of Broken Glass,” respectively, on the soundtrack.)

Both of these “sound effects” are, I think, better understood as musical events. They slot too perfectly into the already-playing song when they’re triggered, and they don’t make sense as diagetic sounds that Chell would hear anyway. (Soaring through the air doesn’t actually make a noise, after all.) These sounds, then, lean towards the red or “embodied” end of the spectrum.

There are more early examples of this, but let’s move a little further in. Here’s a short sample of a theme that plays just after the big fall, when you begin the 1950s-style test chambers with Cave Johnson:

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

We’re back to featuring arpeggios, as with “The Future Starts With You” above, but the effect here is quite different. This music is much “redder” — the instrumentation and the arrangement allow us to immediately and decisively identify it as musical, and we aren’t caught up in deciphering an encoded message.

As it turns out, it also provides the harmonic basis for the music in the next set of puzzles.

Here’s a snip from the first repulsion gel test chamber. Beginning at 0:13, you can hear a floaty high-pitched arpeggio play each time Chell bounces:

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

And here, in the first propulsion gel test chamber, you can hear a tremolo effect as she sloshes along starting at 0:04:

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Here sound and music are so intertwined that I don’t think they’re meaningfully distinct on Murch’s spectrum; they are only differentiable insofar as some events are triggered by the player and some are not. It’s an unusually unified sound design that prevents the music from fading into the background.

17 comments

Ever since I wrote about how Halo 2 did something kind of similar (thought less pronounced and less ambiguous) I’ve been wondering who would be the first to do something more “extreme” and blur the lines completely. Now I know.

by Ben Abraham on June 20, 2011 at 8:08 pm #

This approach to music and sound effects in games really is the future. And it really can only shine in the context of an interactive medium. In a movie, for example, it would come off as corny, I think. Or maybe just used as an artisitic flourish in an artsy film.

I can’t wait to see more of this.

by Tzvi Friedman on June 21, 2011 at 9:03 am #

I don’t think all games would benefit from this approach, but more variation is certainly welcome. A lot of modern sound design is just as focused on gritty hyperrealism as the art direction is; in that way, Portal 2 is like the aural equivalent of, say, Okami. A breath of fresh air, you know?

by Dan Bruno on June 21, 2011 at 11:15 pm #

Thanks for this, Dan.

I noticed the interactiveness of the music, but I didn’t really grok what was going on until you put it in context of Murch’s spectrum.

I recently listened to the developer commentary on the remake of Monkey Island and Monkey 2, where they talked about the dynamic music system where different instruments faded in and out in different locations. I guess I accidentally slotted Portal 2 as “Oh, it’s like that” and didn’t realize how cool it was until you pointed it out. So again, thanks!

by Pace Smith on June 24, 2011 at 10:55 am #

I loved Portal 2′s music, and this post has helped me understand its awesomeness more fully, so thank you.

by Rachel on June 28, 2011 at 5:37 pm #

Great analysis and blog, I’m now a follower.

by Astounding on June 29, 2011 at 1:55 am #

I had heard about the ‘dynamic music’ in Portal 2 before the release of the game, and couldn’t really get to grips with it. there was that “you’ll understand when you experience it” element that you noted about the original Portal.
I had even heard the soundtrack before playing so discovering that various stems of tracks were encoded into gameplay elements was a brilliant experience. I would like to see valve really expand on this in some of the DLC for the game.

by New now on June 29, 2011 at 4:07 am #

Whenever a game uses player interaction as a control for musical layers, I love it. It’s why I think De Blob was so great, and I suppose on the spectrum that entire game would be entirely red, since painting buildings incited instruments to play their bits over the base track.

Super Mario Galaxy also pulled some of this off. When you fight bowser for the final time, once you knock him on his back, the male choir kicks in to make everything more intense. The music playing atop the rolling balls is directly tied to how fast you choose to go with them. Most of that too would be red, I suppose. I wish more games did it!

by DanK on June 29, 2011 at 5:21 am #

Cool read.
Still, I think you missed a very interesting example, which happens on test chamber #5 of the Wheatley tests. He’s playing classical music over the chamber, and the sound the aerial faith plates makes when you use them is actually second and third voices to the music.
I love how this happens so far into the game, that you are now completely used to having the elements play musical cues, exccept now, the cues change dramatically. It’s an inside joke, and I for one laughed out loud when it happened. I kept jumping from one plate to the other for a long time, listening to the music.

by Simon on June 29, 2011 at 7:57 am #

@Dan Bruno: interesting link, thanks!

Haven’t played Portal 2 yet, now looking forward to it even more :)

by qubodup on June 30, 2011 at 5:43 am #

Interesting read… at first I thought you were describing what would be quite old tech. (e.g. Blending between environments as with Monkey Island)

But having listened closely (I shamefully haven’t passed through the first few chambers of P2 yet) and as an amatuer producer it’s great to hear synchonisation of effects to the rythmn of the music.

I believe I’ll appreciate the soundtrack even more now as I finally play through the Game (which I will do right now!).
I’d like to see how much this makes a difference with 5.1 turned up loud!

Power to the PC Gamers…. |H|H|Khola

by Daniel 'Khola' on June 30, 2011 at 12:00 pm #

Wow! Thanks, everyone. I’m glad you enjoyed the post.

Simon, I totally forgot about that level! I wasn’t trying to be comprehensive, but I wish I had remembered to include it here. Thanks for reminding me of it.

by Dan Bruno on June 30, 2011 at 9:24 pm #

An interesting post, and makes me a lot more interested in Portal 2 than I previously was(still not paying full price though, can’t afford games at the moment).

I need to correct you on a few things though.

For example, some music is obviously designed more for people that understand it. People from a culture that doesn’t understand the electric guitar to whatever might find heavy metal music a lot harder to decipher. I’ve actually found that a lot of music has become more interesting as I’ve been getting into playing instruments and sound design – it seems like it should take the magic out of it, but somehow, it doesn’t, there is still nothing to explain “Why” that sound makes someone feel that way, but you have a fuller appreciation of it.

I’m also unsure how new this is. You may remember titles like Rez, though it’s got quite a different emphasis for this. Taken as soundclips abstract of the game, that style of music is not particularly new. It’s sort of a middle ground between noise/drone/sound design type artists(this would be more “sound design”) and more musical electronic ambient stuff(for example, Brian Eno and Carbon Based Lifeforms come to mind).

I know some artists who’ve been doing this sort of thing. I’d recommend checking them out, maybe even give them a plug as I feel they’re very relevant.

Who comes to mind particularly is E|||| -

http://soundcloud.com/ellll

Check out her Ctrl Alt Delete set in particular.

He doesn’t have much up on soundcloud, but Johnathan Deasy, another local artist, does a lot of this sort of stuff with his various acts(Mersk, Narwhals, etc.).

http://soundcloud.com/jonathandeasy

I think this sort of combination of sound and music goes all the way back to stuff like Throbbing Gristle, probably further. I remember one song that still scares the shit out of me Hamburger Lady, and them explaining how they imagined the guitar being like the vacuum going around her hospital room, etc. it was quite interesting.

I’ve also tried my hand in this sort of thing!

http://soundcloud.com/takeshiandthekid/internals-a-soundscape

It was kind of influenced by Kenji Kawai’s Ghost in the Shell soundtrack(and Wendy Carlos’s Beauty in the Beast, but not sure how relevant that is here, that’s more an exploration of different kinds of scales etc. and recreating “World” music sounds with digital synthesis), also well worth checking out.

Other people probably know a lot more about this than me though, I’m fairly new to it. The two above artists can probably give you a lot more information.

by Witch of Sound & Fury on July 1, 2011 at 10:28 am #

Dan, thanks for another great post. I’ve only played the first couple hours of the game, and so far it’s great—can’t wait to see more.
But as the previous comment notes, Murch’s account of music is troubling. Certainly musical experiences are “embodied” in all kinds of ways, from rhythmic entrainment (e.g., nodding or tapping to the beat) to cross-domain mappings (e.g., “high” and “low” pitches).
But they’re NEVER pure, in the sense Murch describes. They’re mediated at least as much as language, not just at the level of comprehensibility—“This music makes no sense!”—but also in what they seem to express. There’s no shortage of examples of people disagreeing about the “message” of a given piece of music, whether it’s by Bach or Biggie. In fact, I’d even say that disagreement (at some level) is more common than agreement.
[side note: There's an interesting movement in music theory to question the supposed universality of “embodied” experience by considering people with disabilities. Do quadriplegics understand dance music the same way as fully mobile people? When hearing-impaired people partake of musical experience, how does it differ from that of people who hear well?]
My own instinct would be that sound effects are LESS coded than music, since the material processes that cause footsteps, metallic clanks, and explosions are much less culturally specific than musical styles. Does that make any sense, or have I misunderstood Murch and/or you?

by Peter on July 2, 2011 at 5:37 pm #

Hey Peter and Witch,

Thanks for your comments! I’m not sure if I’m on the same page as you guys, but I’ll do my best to address what you’ve said.

I actually think Murch would agree with you that musical experiences are never “pure.” He essentially says as much in this passage, from the same link as above:

[…] there are elements of code that underlie every piece of music. Just think of the difficulty of listening to Chinese Opera (unless you are Chinese!). If it seems strange to you, it is because you do not understand its code, its underlying assumptions. In fact, much of your taste in music is dependent on how many musical languages you have become familiar with, and how difficult those languages are. Rock and Roll has a simple underlying code (and a huge audience); modern European classical music has a complicated underlying code (and a smaller audience).

To the extent that this underlying code is an important element in the music, the ‘color’ of the music will drift toward the cooler (linguistic) end of the spectrum. Schoenberg is cooler than Santana.

That said, I think Murch is trying to look at this from a higher level. His piece is written for an audience of film editors; his spectrum is a tool for understanding varieties of sound in order to avoid overusing any one type. He is less concerned, I think, with the exact degree to which music has meaning encoded within it — rather, he wants music as a whole to exist opposite speech on the spectrum, and is happy leave the details as an exercise to the reader. Which, I guess, is what we’re doing now. :-)

Regarding sound effects, Peter, I think I disagree with your assessment. It’s not their cultural universality that makes them encoded, but the way in which they convey meaning. Murch again:

And sometimes a sound effect can deliver discrete packets of meaning that are almost like words. A door-knock, for instance, might be a “blue” micro-language that says: “Someone’s here!” And certain kinds of footsteps say simply: “Step! Step! Step!”

Compared to music — any music — I think the effect here is far more linguistic.

Hopefully that made some sense!

by Dan Bruno on July 4, 2011 at 3:27 pm #

Awesome post, thanks! I’ve long been a soundtrack fan. I tried explaining this very topic about Portal 2 to my actor/musician/opera-aficionado partner. Eventually I said “it’s as if the music and sound effects are all on the same click track. There’s never any ambient sound effect stepping on each other or the music.”

But your analysis is MUCH better so I sent it to him.

Thanks again!

by SurlyB on July 7, 2011 at 11:16 am #

Hi folks,
Huge congrats is in order for this blog. I think its great that much of the new media is continually including audio excerpts into their writing, instead of long, often inaccurate, descriptions coupled with elaborate graphics and notation.

In regard to Portal 2, i think the actual use of interactive music, although clever and resourceful, was quite superficial. I always tend to ‘judge’ the quality of music based on 1) the music itself and 2) how it functions in the game. Here i think that although there was very cool triggers at certain parameters within a level, these were very detatched and not really contributing to the overall ‘narrative’ of the level. The propulsion gel, for instance, tended to be a novel idea if anything. Yes the music would speed up…, only to stop at the trigger cut off. However, the trigger was continuos, like the laser beam thingy, which builds up as you complete the level, therefore following some form of narrative. But i believe, the idea of interactive music in portal 2, very appealing for future VGM composers.

Moreover, in regard to the whole music as universal language debate, enthusiasts might like to look into the research of Adrian C. North and Hargreaves, which basically investigates musical preferences and their contruction. I would love to stay and elaborate, but im outa here. Long live VGM!

by Darbeel on July 28, 2011 at 2:57 am #