Tuesday, July 09, 2013

Technical Sound Design - Illustrations

Illustrating the art of technical sound design can be a tricky thing. Fundamentally the role I play is usually as a supporting actor for the lead role of content creator. My job is to make other peoples sounds sound good, so to speak. How that get's done from project-to-project can be a bit of a mystery. A little bit of pipeline, some animation tagging, ambient placement in a level editor, text file hackery, code spelunking, maybe some audio middleware wrangling? I've always tried to expose the inner workings of the process: from my first title rocking FMOD on Conan, unleashing mad physics on StarWars: The Force Unleashed, complicated vehicle simulations of The Saboteur, and now Uncharted 3.

Game audio demo's for technical sound design are inherently difficult to create. How do you express in a video the  process of finding and unifying the implementation across over 300 doors in a game like Dead Space 3? Or what about the look of the producers face when one morning, after weeks of preparation, you flip the switch on the impact implementation for 150 physics objects in Uncharted 3? Pushing thousands of lines of dialog through the pipeline on Marvel Heroes was no small task, but how to show the work?

If you're lucky enough to have survived the development of a game you at least have the pixels on screen and the sound from the speakers to tell the tale, but in the case of technical sound design that is usually only half the story. So it's always a challenge when I try to put together something that represents the work that I've done. Thankfully I've gotten to work with some amazing sound designers (on some amazing projects!) whose work helps carry some of the explanatory text that seems like a necessary evil in these long-form video exposés. Hopefully the side-effect of creating these is that people can gain a greater understanding of the work that goes into making great sounds sound great in-game. It should be clear (by now) that sound for games is nothing like film. Nowhere is this more true than in the role of technical sound design.

Uncharted 3 - Technical Sound Design: Physics Set Pieces from lost lab on Vimeo.

Uncharted 3 - Technical Sound Design: Procedural Animation from lost lab on Vimeo.

Uncharted 3 - Technical Sound Design: Physics Impacts from lost lab on Vimeo.

Uncharted 3 - Technical Sound Design: Animation Set Pieces from lost lab on Vimeo.

Star Wars - The Force Unleashed:
The Saboteur: Conan:

Tuesday, July 02, 2013

Wwise [HDR] Audio Adventure

A funny thing happened last year...

Audiokinetic approached me to help them craft a sort-of manual for their Wwise Game Audio Middleware. I started by revising some of their already existing documentation. Audiokinetic has worked hard to position their resources as both easy to understand and with a deep well of knowledge. As a continuation of that practice, Simon Ashby contacted me with the seed of an idea: his intention was to create a tutorial for Wwise which encompassed the creation of a fictitious adventure game project. Through a combination of step-by-step instruction and linear storytelling, the development of a comprehensive Wwise project could be created by someone with little-to-no experience, while also providing semblance to the process from start-to-finish.

This was the beginning of the Wwise Project Adventure.

Released in 2012, the "Wwise Project Adventure: A Handbook for Creating Interactive Audio Using Wwise" now comes as an option during the installation of Wwise. In an interview with DesigningSound.org last September, some more of the process of creating the document and associated project was discussed. This was an incredible opportunity for me to solidify some of the practices I had been using on other games and try to bring methodologies from my experience to the community. As I've previously mentioned, good resources in my formative years were hard to come by. It's a great experience to be asked to share knowledge and be given a platform to attempt to increase the Technical Sound Design knowledge share.

So when the time came for Wwise to unleash their vision for HDR Audio in the latest 2013 version of Wwise, they asked me to write a chapter weaving their new feature into the established story line and project. Building on the established work of DICE, HDR Audio in Wwise is just one of six different mixing techniques, including: set-volume mixing, state-based (snapshot) mixing, auto ducking, RTPC (parameter controlled), sidechaining, and high dynamic range mixing. What amazes me about the possibilities in game audio continues to become enabled and able to be realized using Wwise. I can be said that not every game needs all of the tricks available, but to be without the potential, in a situation that could benefit from any one of them is beginning to seem like an oversight.

Audiokinetic's Xavier Buffoni (who I had the pleasure of meeting and hanging out with this spring during the Londaon Audio for Games Conference) wrote up an excellent technical overview of "Finding your way with High Dynamic Range Audio in Wwise" last month. He chronicles the some of the thought process behind bringing HDR to Wwise and clearly expresses the under-the-hood mechanics that make HDR Audio in Wwise a comprehensive addition to the toolset.

Here's a more storied version of HDR Audio that hit the cutting room floor during the writing of the Wwise HDR Audio chapter:

From high atop the mountain: the threshold engages the HDR audio system as the first magic blast resonates across the battlefield. The volume of the magic blast has been authored as the loudest sound in the project, which swiftly engages the window top and removes footstep sounds from the mix. The concussion of the blast is a force of danger regardless of your position during battle. At that moment, nature, footsteps, and the sounds of fallen soldiers are removed from your purview while you focus on the magnitude of sound.

As the window top follows the magic blasts envelope, the cold sound of steel-on-steel begins to cascade across the ravaged field of warriors. Authored below the relative volume of a magic blast, each impact rings out with a sharp attack that continues peaking the window top. The ratio property directly relates to the attenuation of sounds below threshold. This leaves ambience out of the mix amidst the battle cries rise up towards another assault.

The last warriors stand alone. Locked in battle, the final sickening smack signals the triumphant blow that will put an end to the war. Once the window top has adjusted to the amplitude of the impact, the release mode and release time return the HDR window to rest. There is now a return of ambience and sounds of the wounded scattered across the battlefield. Their voices heard quietly and clearly, as if they whisper to the wind seeking sweet relief.

The battle is never fought all at once, but through a series of small decisions that eventually align themselves toward a common goal. Each relativistic change in amplitude between a sound or group of sounds serves to prepare for the final battle for the perfect dynamic mix. With tools and weapons equipped at the ready, there is no fear from the day when the project ships.

One of the things that continues to impress me about the way Audiokinetic chooses to implement new features within Wwise is that they not only integrate the feature but they also give you a way to audition and measure the effectiveness of it. Nowhere is this clearer than in the newly added "Voice Monitor" which allows for viewing the affect of the HDR Audio properties across the entire HDR System.

Rev. Dr. Bradley Meyer goes into some depth with his experience mixing the latest inFAMOUS Second Son game using "Wwise HDR Best Practices". You can catch a first-listen to Wwise HDR (among other interactive mixing techniques) in action through this recent gameplay footage, I'm especially fond of the detail in distance attenuation for weapons and explosions.

Putting the capability to create interactive and dynamic audio into the hands of many, today's audio middleware is allowing for the rapid advancement of techniques and fidelity in game audio. The availability of tools that can be used to design sounds that fit gameplay is a recipe for a bright future in what's to come. I can't wait to hear what everyone has in store for the future!