Technical Sound Design - Illustrations

Illustrating the art of technical sound design can be a tricky thing. Fundamentally the role I play is usually as a supporting actor for the lead role of content creator. My job is to make other peoples sounds sound good, so to speak. How that get's done from project-to-project can be a bit of a mystery. A little bit of pipeline, some animation tagging, ambient placement in a level editor, text file hackery, code spelunking, maybe some audio middleware wrangling? I've always tried to expose the inner workings of the process: from my first title rocking FMOD on Conan, unleashing mad physics on StarWars: The Force Unleashed, complicated vehicle simulations of The Saboteur, and now Uncharted 3.

Game audio demo's for technical sound design are inherently difficult to create. How do you express in a video the  process of finding and unifying the implementation across over 300 doors in a game like Dead Space 3? Or what about the look of the producers face when one morning, after weeks of preparation, you flip the switch on the impact implementation for 150 physics objects in Uncharted 3? Pushing thousands of lines of dialog through the pipeline on Marvel Heroes was no small task, but how to show the work?

If you're lucky enough to have survived the development of a game you at least have the pixels on screen and the sound from the speakers to tell the tale, but in the case of technical sound design that is usually only half the story. So it's always a challenge when I try to put together something that represents the work that I've done. Thankfully I've gotten to work with some amazing sound designers (on some amazing projects!) whose work helps carry some of the explanatory text that seems like a necessary evil in these long-form video exposés. Hopefully the side-effect of creating these is that people can gain a greater understanding of the work that goes into making great sounds sound great in-game. It should be clear (by now) that sound for games is nothing like film. Nowhere is this more true than in the role of technical sound design.

Uncharted 3 - Technical Sound Design: Physics Set Pieces from lost lab on Vimeo.

Uncharted 3 - Technical Sound Design: Procedural Animation from lost lab on Vimeo.

Uncharted 3 - Technical Sound Design: Physics Impacts from lost lab on Vimeo.

Uncharted 3 - Technical Sound Design: Animation Set Pieces from lost lab on Vimeo.

Star Wars - The Force Unleashed:
The Saboteur: Conan:

Wwise [HDR] Audio Adventure

A funny thing happened last year...

Audiokinetic approached me to help them craft a sort-of manual for their Wwise Game Audio Middleware. I started by revising some of their already existing documentation. Audiokinetic has worked hard to position their resources as both easy to understand and with a deep well of knowledge. As a continuation of that practice, Simon Ashby contacted me with the seed of an idea: his intention was to create a tutorial for Wwise which encompassed the creation of a fictitious adventure game project. Through a combination of step-by-step instruction and linear storytelling, the development of a comprehensive Wwise project could be created by someone with little-to-no experience, while also providing semblance to the process from start-to-finish.

This was the beginning of the Wwise Project Adventure.

Released in 2012, the "Wwise Project Adventure: A Handbook for Creating Interactive Audio Using Wwise" now comes as an option during the installation of Wwise. In an interview with DesigningSound.org last September, some more of the process of creating the document and associated project was discussed. This was an incredible opportunity for me to solidify some of the practices I had been using on other games and try to bring methodologies from my experience to the community. As I've previously mentioned, good resources in my formative years were hard to come by. It's a great experience to be asked to share knowledge and be given a platform to attempt to increase the Technical Sound Design knowledge share.

So when the time came for Wwise to unleash their vision for HDR Audio in the latest 2013 version of Wwise, they asked me to write a chapter weaving their new feature into the established story line and project. Building on the established work of DICE, HDR Audio in Wwise is just one of six different mixing techniques, including: set-volume mixing, state-based (snapshot) mixing, auto ducking, RTPC (parameter controlled), sidechaining, and high dynamic range mixing. What amazes me about the possibilities in game audio continues to become enabled and able to be realized using Wwise. I can be said that not every game needs all of the tricks available, but to be without the potential, in a situation that could benefit from any one of them is beginning to seem like an oversight.

Audiokinetic's Xavier Buffoni (who I had the pleasure of meeting and hanging out with this spring during the Londaon Audio for Games Conference) wrote up an excellent technical overview of "Finding your way with High Dynamic Range Audio in Wwise" last month. He chronicles the some of the thought process behind bringing HDR to Wwise and clearly expresses the under-the-hood mechanics that make HDR Audio in Wwise a comprehensive addition to the toolset.

Here's a more storied version of HDR Audio that hit the cutting room floor during the writing of the Wwise HDR Audio chapter:

From high atop the mountain: the threshold engages the HDR audio system as the first magic blast resonates across the battlefield. The volume of the magic blast has been authored as the loudest sound in the project, which swiftly engages the window top and removes footstep sounds from the mix. The concussion of the blast is a force of danger regardless of your position during battle. At that moment, nature, footsteps, and the sounds of fallen soldiers are removed from your purview while you focus on the magnitude of sound.

As the window top follows the magic blasts envelope, the cold sound of steel-on-steel begins to cascade across the ravaged field of warriors. Authored below the relative volume of a magic blast, each impact rings out with a sharp attack that continues peaking the window top. The ratio property directly relates to the attenuation of sounds below threshold. This leaves ambience out of the mix amidst the battle cries rise up towards another assault.

The last warriors stand alone. Locked in battle, the final sickening smack signals the triumphant blow that will put an end to the war. Once the window top has adjusted to the amplitude of the impact, the release mode and release time return the HDR window to rest. There is now a return of ambience and sounds of the wounded scattered across the battlefield. Their voices heard quietly and clearly, as if they whisper to the wind seeking sweet relief.

The battle is never fought all at once, but through a series of small decisions that eventually align themselves toward a common goal. Each relativistic change in amplitude between a sound or group of sounds serves to prepare for the final battle for the perfect dynamic mix. With tools and weapons equipped at the ready, there is no fear from the day when the project ships.

One of the things that continues to impress me about the way Audiokinetic chooses to implement new features within Wwise is that they not only integrate the feature but they also give you a way to audition and measure the effectiveness of it. Nowhere is this clearer than in the newly added "Voice Monitor" which allows for viewing the affect of the HDR Audio properties across the entire HDR System.

Rev. Dr. Bradley Meyer goes into some depth with his experience mixing the latest inFAMOUS Second Son game using "Wwise HDR Best Practices". You can catch a first-listen to Wwise HDR (among other interactive mixing techniques) in action through this recent gameplay footage, I'm especially fond of the detail in distance attenuation for weapons and explosions.



Putting the capability to create interactive and dynamic audio into the hands of many, today's audio middleware is allowing for the rapid advancement of techniques and fidelity in game audio. The availability of tools that can be used to design sounds that fit gameplay is a recipe for a bright future in what's to come. I can't wait to hear what everyone has in store for the future!

Game Audio Aspirations

I recently wrote these suggestions out for an aspiring game audio professional. I thought it might ring true for others and so I'm reprinting it here with permission.

"First of all, thanks for sharing your story. We've all got one, keep cultivating yours and try out different plot lines. Keep trying to envision the next chapter. I think it's so cool that you've been in pursuit of a dream for so long and through trials and tribulations continue to find yourself coming back to the path laid before you, ready to move forward. Sometimes knowing what you want is the hardest part and from there it just takes doing. One of my recent epiphanies...and it's not like I made this up...is that sometimes in order to know, you have to do. So my first suggestion is that you do and keep doing what it is your heart tells you to do. Being honest with yourself about what your heart needs lays fertile ground for the cultivation of dreams. This is at the root of it: dreams can come true. It takes work, time, focus, and magic. There are too many stories of dreams coming true for it to be a hoax, and I've got more than a few myself.


Next up, what should you do? It's easy: do what you love. If you focus on that then you can't go wrong. Don't know exactly what it is that you love to do? Do everything instead, until you figure out the things you are naturally drawn to vs. the things that suck your soul dry. We all know these things inherently, but are willing to overlook some of them depending on the paycheck/ circumstance/ reward.


Fuck.That.Shit (thanks to @joecavers for tweeting that every couple of months)


Follow your bliss towards the thing that makes you feel good. Even if the results of that are terrible, don't worry you can't help but get better at the things you love to do. Because you'll keep doing them, not because of x y z, but because it feels good. When you feel good it bleeds into everything you produce. Need some validation that everyone is terrible when they first start out? Go look up my demo reel from 7 years ago...or an example of anyone's work from their early years. Internalize this acknowledgement and press through the emotions. Keep walking the path.

Maybe you should take a break and ready this timeless tome from Dr. Suess...or this beautiful cartoon of Alan Watts' famous lecture.

Ok, ready? Now for some nitty gritty.


1. Find people to work with. You will learn more from others than you ever will from a book or class...err, I guess classes have people so that's not totally true. Look for opportunities and projects that are willing to give you a shot at lending a hand, then prove to be a handy addition to the team in whatever way possible. Maybe it's by being the person who calls a meeting when shit's fucked up or maybe you wrangle a spreadsheet to help keep assets straight for the animators...whatever it is, do good! Some of the first projects I worked on are still alive in the world. Games I never expected to see the light of day, risen from the ashes of troubled development and shipped. It's heartwarming to think about people I worked with who are now in great positions in the game industry, people who were on the same level as me when I first started out. People who had passion, people who cared for the right reasons, people who loved what they were doing, for the sake of it.


2. Find a community online: twitter #GameAudio, gameaudioforum, G.A.N.G., IASIG...whichever one seems good to you. Better yet, find an IGDA meeting, other gamedev meetup, or audio-centric meeting where you can socialize with people who share a passion for games and audio. Find a way to help with their initiatives, find a way to share the experience of learning so that others might learn from you. Become an expert in the thing you love, share that with people. Dig into resources like GameAudioPodcast, GameAudioRelevance, GameAudio101, GameSoundDesign, DesigningSound, CreatingSound, and while you're consuming all of these tasty morsels of game audio, please keep doing. Find a way to build a feedback loop of doing and sharing what you're doing, whether it's through the community or friends or the internet at large. You will build a body of work just by doing, which will stand as a testament to how far you'll have come when you look back from the future. Dig through some of my old stuff, there's some heinous business out there written in my name...it's who I was and what I knew then. Always try your best to represent yourself clearly and honestly (if you don't know say so) but everyone is entitled to an opinion.

3. Sharpen you tools, whatever they are. Before you can do that, know what it is you're trying to build. This goes right back to the question of what your heart wants. Once you've figured that out (no easy feat!) work on making your process and pipeline a work of art. Nobody could have told me I would be rocking some serious spreadsheets in order to do this technical sound design gig, but I have rocked my fair share on every project I've ever been on. Maybe that's how my brain works. I'm no wizard but I can pull off some cool tricks that help me pass through the unenviable task of data management unscathed by concatenated strings. The same goes for some Wwise/ FMOD/ Unreal/ etc. bidness. Ask me to design sounds in Pro Tools and I'm the slowest cowboy in the mid-west, probably not even that good at it at the end of the day. However, I can get surgical on a .wav file and throw something together for fun. Sound for cinematics ten hours a day does not complete me. The beautiful thing is, it's likely that cinematic sound is someone else's passion. Someone who loves that as much as I love getting sounds to play back and sound good inside a game engine."


Above all else, keep dreaming...that is the key.

Sound Quote: Siddhartha - Herman Hesse

Escape is a warm pleasure. Submerged in another experience I can bring back insights to apply back in the real world. Regardless of the medium, I continue to be drawn to the way sound is represented. Whether it's movie-going, game sound studies, or just plain ambiance-soaking; when I am able, I try to practice an appreciate for the sound, and the organization and representation of sound, from artists in each of these fields.

Which brings me to the the Sound Quote series here on the Lost Chocolate Blog. I have a habit of reading books with a keen ear towards how writers choose to describe sounds. It's easy to find plot synopsis and story information for books, but assessing how people write about the ephemeral quality of air-vibrations in print continues to fascinate. Here are a few from Siddhartha by Herman Hesse, which center around the main characters relationship with a stream during the latter part of his life.

"The river flowed softly and quietly, it was the dry season, but its voice sounded strange: it laughed! It laughed clearly. The river laughed, it laughed brightly and clearly at the old ferryman. Siddhartha stopped, he bent over the water, in order to hear even better, and he saw his face reflected in the quietly moving waters, and in this reflected face there was something, which reminded him, something he had forgotten, and as he thought about it, he found it: this face resembled another face, which he used to know and love and also fear.

"You've heard it laugh," he said. "But you haven't heard everything. Let's listen, you'll hear more."

The river sang with a voice of suffering, longingly it sang, longingly, it flowed towards its goal, lamentingly its voice sang.

Siddhartha listened. He was now nothing but a listener, completely concentrated on listening, completely empty, he felt, that he had now finished learning to listen. Often before, he had heard all this, these many voices in the river, today it sounded new. Already, he could no longer tell the many voices apart, not the happy ones from the weeping ones, not the ones of children from those of men, they all belonged together, the lamentation of yearning and the laughter of the knowledgeable one, the scream of rage and the moaning of the dying ones, everything was one, everything was intertwined and connected, entangled a thousand times. And everything together, all voices, all goals, all yearning, all suffering, all pleasure, all that was good and evil, all of this together was the world. All of it together was the flow of events, was the music of life. And when Siddhartha was listening attentively to this river, this song of a thousand voices, when he neither listened to the suffering nor the laughter, when he did not tie his soul to any particular voice and submerged his self into it, but when he heard them all, perceived the whole, the oneness, then the great song of the thousand voices consisted of a single word, which was Om: the perfection."

Herman Hesse is one of the authors I continue to come back to. Whether it's because of his book 'Damien' which comes close in spelling to my name, or because of  the struggling artist theme that seems to run throughout books like Gertrude and Steppenwolf, his writing continues to reveal insight throughout my life.

Game Audio London 2013

Proper Beans & Toast!

Just back from the AES 49th Audio For Games Conference and I'm overwhelmed with feelings from the past week. To say that the trip oversea's was a game-changer for me would be an understatement; It shook me to the very core of my person. Those who know me also know I have a penchant for enthusiastic positivity...it goes without saying that, when I say it I mean it. The rush of emotions I've felt were the direct result of so many things: the support and sponsorship of individuals and institutions, the openness of the game audio community, the laser-focused conference track, and the spirit of openly sharing the heart of our interactive lives with each other.

I helped coordinate a bit of the behind-the-scenes work that went on during the conference and acted as an advocate and evangelist throughout the past year. One of the coolest community things that happened during the conference was the #AudioForGames hashtag on twitter. People were able to jump on it early and bring a second layer to the sessions and presentations that helped fuel the discussion and engaged participants in (what I thought was) a higher level of communication. Thanks to everyone who contributed there as part of the community. (It goes without saying that the #GameAudio community overall is one of the best things!)

Aside from the few people I had met in person before, nearly every interaction I had was immediately "easy"; Some were downright psychic in their communication. (Here I go again with espousing the positive.) I say it every year (and often daily) that it is the people in game audio that continue to make this passion for interactive sound so satisfying. Sure, I like pulling off cool tricks and supporting the sound and dynamics in games...but it's the people that make these tiny victories so much better. I always feel like when I speak so positively to the nature of people that I'm setting myself up for the fall, but I've not felt the repercussions of it thankfully.

Add to this that it was the first time that I'd been in the same room with Anton, co-founder of the Game Audio Podcast, in over 7 years. The podcast grew out of our experience during one of the first couple of GDCs I attended as a way to extend the conversations that happened at conferences. We've been at it remotely for three years and I can easily say that being at the conference with him made it a richer experience. We were able to record an episode per day, wrapping up the previous days events and sharing our experiences. We also were lucky enough to rope in some special guests who were gracious enough to take time out of the jam-packed schedule to chat about their experiences.

The fine folks at DesigningSound.org have helped us round up the episodes which can be accessed here:
http://designingsound.org/2013/02/game-audio-podcast-aes-coverage/

Look for a couple more to trickle out in the near future as we continue our coverage of all things game audio.

Speaking of that, GDC is right around the corner! Anton and I will once again find ourselves together to bring you coverage of this years exciting lineup. Over the past years, a team of dedicated podcast editors have generously donated their time to cleaning, editing, and polishing our (my) frantic ramblings. If you're headed out to San Francisco in March be sure to say hello!

My Aural Fixation One Year Anniversary

I remember sitting in Jesse Harlins office at Lucas Arts near the end of working on The Force Unleashed 2, when introspection was at it's highest point. We got to talking about how important we both felt it was to continue contributing to the game audio community in whatever way possible. His contributions to Game Developer had helped shape my window into the world of Game Audio and I had really valued his insight month after month as I was making my way in. Similarily, I had been writing the Audio Implementation Greats series that year and even kept a development blog during my first project...not to mention this blog, which has been going since 2004.

Time passed and about a year ago he called to asked if I'd be interested in taking over the Aural Fixation column. Since then it's been an amazing outlet for my ramblings, rantings, and purge for insistent earworms. What, you don't know what I'm talking about? You definitely find your way to a subscription of Game Developer so you can follow along at home!

So, to celebrate the passing of a years worth of writing I thought I'd recap and point out the (sometimes hidden) musical references that I've been consistently sneaking into each article. It's evolved over time from pure song lyric quotes at the end of an article into a full blown interwoven breadcrumb-trail burind within the headers. All of this because I often wake in the morning with an 80's jukebox in my head, it's a curse!

2012

Dude History? (February)

How I Found the Future by Looking Backward

“...and in the end the love you take is equal to the love you make.” - The Beatles: The End

It’s About People (March)

The Great Game Audio Community

“Big city, bright lights, cool cool people, big city. Everybody I know can be found here.” - Spacemen 3: Big City

Knowing a Thing, or Two (April)

Career Fragmentation at the Turn of the Generation

“Sprinklers that come on at 3 a.m. sound like crowds of people asking you, ‘Are you happy what you’re doing?’” - Grandaddy: The Group Who Couldn’t Say



“Look out honey cause I’m using technology” - Iggy and the Stooges: Search and Destroy

Vroom Vroom (May)

A Study of Sound in Racing Games

“Let the good times roll.” - The Cars: Let the good times roll

Crossing the Two-way Street (June/July)

From Game Audio to Game Engine

Headers - Huey Lewis and the News: This is it



“Crosstown traffic, so hard to get through you.” - Jimi Hendrix: Crosstown Traffic

Audio in Unity by Alexander Brandon (August)

Welcome to Unity Sound Design and Scripting

Pop Will Eat Itself (September)

Could Game Mix in Popular Music Samples?

Headers - Phil Collins: Various



“Everybody’s talking bout the new sound. Funny, but it’s still rock and roll to me.” Billy Joel: It’s Still Rock and Roll to Me

Procedural Music by Yann Seznec (October)

Making Games Into Music, and Music Into Games

It’s Just a Jump to the Left (November)

Dynamic Animation Sound Now

Headers - Rocky Horror Picture Show - Time Warp


“With a bit of the mind flip, you’re into the time slip, and nothing can ever be the same.” - Rocky Horror Picture Show - Time Warp

Death of an Audio Engine (December)

Reinventing the Wheel No Longer

“Some of the want to use you, some of them want to be used by you.” Eurythmics - Sweet Dreams

2013

Hot for Game Audio (January)

The Game Audio Literature Review

Headers - Van Halen: Hot for Teacher


“I think of all the education that I missed, but then my high school was never quite like this.” Van Halen: Hot for Teacher

WHATCHAMACALLIT (February)

Achieving Transparency in Naming

Headers - Carly Rae Jepson: Call me maybe



“I want to tell you something you've known all along, don't leave me hanging on the telephone.” - Blondie: Hangin’ on the telephone

Every Year I Buy the Magic Beans (March)

The Pains of Being Freelance at Heart

Headers - Olivia Newton John & Electric Light Orchestra: Magic


“You have to believe we are magic, nothing can stand in our way. Don’t let your aim ever stray.” - Olivia Newton John & Electric Light Orchestra: Magic

So with that a special thanks to Jess for passing the torch off to me and big thanks to everyone who has read and given feedback on my articles this past year. It's an incredible opportunity to be able to wax philosophical about the state of our industry and I hope that these article channel the community through the discussions I have with you throughout the year. Feel free to drop a line if something doesn't seem quite right, or if you think we're missing something important that's happening.

Also, give a handclap to the 2012 guest contributors: Alexander Brandon from Funky Rustic and Yann Seznec from Lucky Frame.

Steady on,
-lcl

Am I a Rock and Roll Zombie?

I got a sweet USB cassette deck over the holidays and have been dumping down some priceless dead media from the collection. Among the old demos, musical sketches, and freakish monologues are a few mix tapes that defined my adolescent years in such an encapsulated form that I feel they are worth sharing.

This is one of my faves.

I ran my all-in-one CD/Record Player/Cassette deck through a stereo effects processor I had been using on my guitar for years. The goal was to use it to make transitions and effect the different songs in a way that they would all flow together, the reality is that it get's a bit wild at times. Side A kicks off with a lost sample from a Flash Gordon story album which bridges it's re-introduction at the start of Side B.

I'm beginning to think that we're dealing with some kind of sub-conscious hypnotic suggestion.

If that's the case, we'll have to find the thread that links all the victims together, something the all have in common.

Wait a minute, I think I've got it
Something they all have in common
Repetitive, thought controlling message
It's a wild chance, but it's possible

What is it Flash, tell us?!

It's the music, it's the rock and roll music they're listening to!

Side A
Flash Gordon - Excerpt
My Bloody Valentine - To here knows when
The Cure - Like Cockatoos
Saint Etienne - Hobart Paving
Miranda Sex Garden - Cut
Ride - From time to time
Madonna - Angel
The Filed Mice - Couldn't feel safer
Revolver - If we could spend some time
My Bloody Valentine - Can I touch you
Ride - Seagull (Live)

I used to be a normal kid just like you
I used to do the things that normal kids do
I used to have a girlfriend
I used to ride a bike
I used to like the things that normal kids like

Lately I been hurtin I been feelin' low
Body is pressed against the radio
Turnin' it up, just a little louder
Turnin' it up, just a little louder

Oh no, it is so
Tell me doc I gotta know
Oh oh, is it so
Tell me doc I gotta know

Am I a rock and roll zombie?

Side B
Flash Gordon - Rock and Roll Zombie
The Field Mice - If you need someone
Slowdive - Country Rain
The Beatles - Strawberry Fields
The Boo Radleys - Boo Forever
The Cure - Closedown
My Bloody Valentine - Nothing much to lose
Blind Mr. Jones - Dolores
Revolver - Shakesdown
XTC -Mayor of Simpleton
Slowdive - Golden Hair

From where I'm standing today, it still sounds good!