Fidelity vs. quality

June 27, 2008

Back in April, Michael Abbott had this to say on one of my Ocarina of Time music posts:

I’m wondering how you feel about the whole MIDI versus recorded orchestrated music question. It’s a no-brainer to me, but I was surprised to receive a couple of responses on my blog recently in defense of the MIDI approach to music in the Zelda series and the hope that it wouldn’t change. Is this just preciousness about the old games, or is there really something preferable about triggered music samples?

I suspect Michael is right that his readers’ preferences are fueled by nostalgia, but his comment touches on an issue that I’ve been meaning to talk about: the conflation of fidelity with quality.

When Ocarina was released in 1998, recording live orchestra was not an option — there’s simply not enough room on an N64 cartridge for audio data. With the advent of massive digital storage media like DVDs and Blu-Ray discs, though, such things are possible. The common wisdom is that the latter is an improvement. It is, but not in the way you might think.

Specifically, digital audio is an improvement in fidelity over sequenced music. The percussion will sound like a drumset instead of radio static; the strings will have proper vibrato instead of the silly wobbling that synthesizers often spit out; the choir will have real people singing real words. In short, the sound quality will be purer, and the instruments will sound more real.

But there are two things to keep in mind here. First, these improvements have no bearing on the quality of the music itself. As an old professor of mine once observed, “You can play a shitty song on a Steinway, but it’ll still be shitty.”

Second, high-fidelity sound is an aesthetic choice, not an objective improvement. If you’re a fan of popular music, you’ve probably heard someone criticize a pop song’s production values as “slick” or “overproduced” while lauding a lo-fi artist’s amateur recordings as “honest” or “authentic” — making no reference to the actual songs the two artists performed. If you listen to hip-hop, you know that the TR-808 is still sampled extensively even though “better” drum samples are readily available. Even the sound of a distorted electric guitar is really just a sacrifice in fidelity for aesthetic reasons. In short, sound quality comes with its own set of connotations and preconceptions that are independent of the quality of the music itself.

Getting back to games: the Super Mario Galaxy score has a number of newly composed, fully orchestrated pieces, such as the Good Egg Galaxy and Gusty Garden Galaxy themes. These are great pieces, emblematic of the new Mario sound. At the same time, Galaxy also features a number of sequenced songs, such as the Toy Time Galaxy and Sweet Sweet Galaxy themes, which are nostalgic remixes of old Mario tunes. Are the new songs better? Maybe — but if so, it’s not because they feature an orchestra.

11 comments

Interesting post. I especially agree about Galaxy.

Two things immediately spring to mind. First, the chiptune scene. Do you know of it? It’s very, very interesting, and featured in a documentary here: http://www.8bitmovie.com/

Second, I recently stumbled across this trumpet virtual instrument here: http://www.samplemodeling.com/en/products.php
It’s a reminder of just how far digital music has come. Honestly, I can’t really tell the difference between the sampled instrument and the real thing.

by Daniel on June 28, 2008 at 9:30 am #

Thanks for bringing up chiptunes, Daniel. I was remiss in not mentioning them.

The fact that virtual instruments are getting to be nearly indistinguishable from real ones makes it all the more significant that low-fidelity music (like the chiptune stuff) is still around. It’ll be interesting to see what happens to that scene, and similar lo-fi ones, once it becomes trivial to store high-quality musical data in a set of MIDI instructions. My theory, of course, is that they’re not going anywhere, as people will still enjoy the aesthetic.

by Dan Bruno on June 28, 2008 at 10:19 am #

In games, sequenced music has at least two additional advantages over prerecorded tracks: it takes up much less space, and it can be adjusted on the fly.

As storage media have grown bigger over the years, space might seem like less of an issue. But developers still like to pack the disc with as many assets as possible, and prerecorded music takes a lot of space. Thus it remains a luxury good. (I imagine it’s expensive to hire musicians, too, but I don’t know how big a factor that is in most games’ budgets.)

There’s a long history of debate about how closely a game’s soundtrack needs to reflect the moment-to-moment gameplay, but one thing is certain: sequenced soundtracks allow much for flexibility than prerecorded ones. Richard Jacques, formerly of Sega, mentions this in a recent Gamasutra interview (http://www.gamasutra.com/view/feature/3695/staying_in_tune_richard_jacques_.php?page=6, third paragraph of his first answer): when you enter the last lap of a race in Mario Kart and the music speeds up, that’s much easier to do with sequences than with full tracks. Likewise for transposition or crossfading lots of layers — to say nothing of algorithmic soundtracks like Spore!

by Peter on June 29, 2008 at 8:18 pm #

Good points, Peter.

I actually wonder whether recorded audio in games is compressed or not. My gut tells me that games with extensive popular music soundtracks (like the recent Tony Hawk games) use MP3s, while original scored soundtracks (like Galaxy) are uncompressed. Not sure if that’s the case though.

Anyhow, while it’s still something to think about right now, storage space is quickly becoming a non-issue. As it is, a dual-layer DVD holds about 8.5 GB, and a dual-layer Blu-ray disc holds 50 GB; in another generation of media, storing uncompressed audio will be trivial.

The flexibility point is an important one, and I think the virtual instruments Daniel linked to above will be critical for dynamic music in the future. Of course, the best of both worlds is enough storage space and horsepower to store high-quality audio samples and run full-fledged audio software in the background; then we’d be able to manipulate effects and rearrange music on the fly with actual recordings. I’m not sure when it’ll be practical to have all that running along side a video game, though. :)

by Dan Bruno on June 29, 2008 at 11:44 pm #

Nice post – I think that most people fail to distinguish between fidelity and aesthetic quality. A fully orchestrated, vivid, epic, soundtrack is probably going to sound out of place in a minimalistic side-scroller, and a bleepy-bloopy PC-Speaker melody is going to sound out of place in Mass Effect.

Re: Peter’s comment. I tried to cover some aspects of LucasArt’s phenomenal iMUSE (interactive music engine) system in an article I wrote a few months ago. I fully believe that we need a modern day equivalent, based on multi-track recording and on-the-fly remixing, because there just isn’t anything comparable out there today. Whether it “works” as an expressive mode is another discussion, but for the games that LucasArts made, it not only worked, but transformed their games.

Link:
http://www.artfulgamer.com/2008/01/31/musical-genius-lucasarts-and-imuse/

by Chris on June 30, 2008 at 12:44 am #

Thanks for coming by, Chris!

I loved the article on iMUSE. I think a similar system is inevitable at some point, even if it’s only used for simple tasks like improving transitions. (I’m amazed at how many recent games use crossfades or even hard cuts between musical themes when the technology obviously exists for more seamless connections.) Until such a system becomes ubiquitous, though, I don’t think it’s unreasonable to see recorded audio as a “two steps forward, one step back” phenomenon. Certainly we’ve lost a bit of flexibility.

On a side note, I hadn’t subscribed to Man Bytes Blog at that point, and since music in games is something of a hobby horse, I just went and read the Blogs of the Round Table posts from that month. So thanks for that little diversion as well. :-)

by Dan Bruno on June 30, 2008 at 1:53 am #

Chris: That’s an interesting point about associations between musical style and game genre. However, some games use apparently “mismatched” music to great effect. Nifflas’s minimalistic platform games (http://nifflas.ni2.se/) have beautifully textured music that couldn’t have existed in a game until about 10 years ago, paired with graphics that could plausibly come from 1992.

The iMUSE article was great — thanks for writing it! — but you may be overstating the uniqueness of iMUSE. I’m no Microsoft fan, but from what I understand, the DirectMusic system has similar capability. There are lots of other solutions as well; for example, here’s an article about the relatively new WWise Interactive Music Module: http://www.music4games.net/Features_Display.aspx?id=127

Dan: I’ll bet money that storage space will always be an issue, for the simple reason that people find ways to use it. I remember when high-density floppies held ten games each, but then only a few years later I bought Beneath a Steel Sky as a stack of (I think) 15 discs! The jump from floppy to CD was maybe the last point at which anyone could really believe that storage space would never again be an issue — it increased by a factor of about 500. But it only took a few more years for Phantasmagoria to ship on seven CDs.

Finally, as far as I know nobody uses uncompressed audio anymore. Every console has built-in DSP to decode mp3 or something like it, and even on PC the processing penalty of decompression is generally considered insignificant compared to the storage penalty of raw WAVs. It’s interesting that you distinguish between Tony Hawk and Mario Galaxy, though! Is it possible that the Tony Hawk soundtrack seems more likely to be mp3 because we’re more used to dealing with that sort of music in mp3 format?

by Peter on June 30, 2008 at 2:25 am #

Thanks for the info on compression. You’re probably right that developers will find a way to fill larger media, but they’ll at least be able to stop considering prerecorded music a luxury. A couple hundred megabytes of MP3s is a drop in the bucket.

I’m not sure why I assumed Tony Hawk soundtracks would be in MP3s while Galaxy wouldn’t. I imagine those games as having more music, but I guess that’s not really true. Maybe it’s a cultural thing, as you suggest.

by Dan Bruno on June 30, 2008 at 9:27 pm #

While we now have the ability and appreciation for high fidelity orchestral recordings in video games, I really feel it depends on the game. I much prefer the video game synth sounds I fell in love with for games such as Zelda and Mario- and think that they should stick with those aesthetics to help maintain some of the magic and style. The games look still quite cartoony and the music should follow.

Just as when you move to a game such as God of War you have these highly detailed larger than life landscapes and visuals, with the live music to match it.

by Thom on August 1, 2009 at 5:01 pm #

Just one thing I wanted to say, midi doesn’t have sounds, midi is a programming language that deals with the length of the note, the pitch, and other various parameters. (mod wheel, pitch bend, etc.)

You can use midi to trigger sounds, but when referring to “midi” sounds most people are actually talking about General Midi (or just GM) Sounds. You can create realistic orchestral renditions of a midi file if you have the right software. (EWQL Symphonic Orchestra Edition Sound Banks, for example)

So it isn’t midi VS Orchestra, its GM tones/cheap synthy sounds trying to emulate real instruments VS High quality orchestral samples, or a real orchestra.

And actually, it’s interesting that some modern (also expensive) synthesizers are actually getting really good at producing mock-ups of real instruments, and I’m not talking about sampling, I’m talking about actual synthesis! This realistic method of synthesis is called Physical Modeling, and PianoTeq is a good program that uses it.

The only thing that really is stopping PM synthesis from being just as realistic as live acoustic instruments is CPU and RAM limitations on the modern computer.

by Nate on April 4, 2010 at 12:24 pm #

Eh, what’s really stopping PM synthesis is R&D.

PM varies a lot per instrument but:

- RAM requirements are very low (~4kb per channel for wind instruments, more for strings, but still a LOT less than samples).
- CPU requirements vary a lot from instrument to instrument and they’re definitely higher than other synthesis types, but modern systems are ridiculously overpowered so they could probably handle it. (CPU usage is somewhere between an EQ and a reverb plugin per channel, though you could probably cut down usage a lot with a smart FFT based algo).

The real problem is the amount of research involved: it took Yamaha a team of like 100 engineers and a few years to make the VL-1, and it didn’t sell much commercially. Some instruments just don’t have any good PM versions yet – violins in particular, but a lot of plucked strings and percussion instruments have rather weak emulations too. Plus, considering that you’d need a whole bunch of those plugins to cover most instruments, how would you license that into your game engine?

by Hubert Lamontagne on April 21, 2011 at 2:55 am #