Tuesday, May 01, 2012

Racing Game Sound Study


Welcome to the Racing Game Sound Study

These informal game sound studies aim to expose the technical side of game audio by making an assessment of current generation titles. The assessment is then used as a way to better understand the differences in approach, aesthetics, and progression of techniques across a small sample. By turning the focus onto emerging details that arise during the course of the study we are able to identify area’s of significance and interest that help communicate the current state of the art. These finding are then represented in a content-rich report that includes: video’s, article links, and specialized interviews. The goal is to help raise awareness for the technical side of sound design and help in the understanding of what is often not very well represented in current literature.

...and a seperate post rounding up all the embedded links in this post:
Racing Game Sound Study [Links]

Here's an admission: I'm not really that into cars. The last one I owned took it's own life last fall in a billowing smoke cloud of explosive glory and has yet to be replaced. I still drive on occasion when I have to, but I guess that's one of the reasons I like living in the city: there's plenty of potential within walking/ biking distance and I don't mind having an excuse to slow down the flow of time. Don't get me wrong though, the sound of engines speak deeply to my love of noise and penchant for drones - coupled with games it becomes an almost irresistible combination.

Here's another thing: racing games present an interesting challenge. On one hand, they represent a deep simulation that attempts to recreate the base components of a vehicle’s engine and mechanical performance, and on the other hand attempts to give players access to an experience that is often unobtainable. It's a small pool of highly specialized talent that will ever have occasion to strap into a rally or formula one car to experience the thrill of racing. But being able to experience the thrill without the danger is something that games have always provided a safe place to experiment with.

While games generally do a good job of enabling role-play, not all racing games have their roots so firmly planted in the reality of everyday life. I mean, most of us have driven cars. We know what it means to accelerate and break, the feel of inertia when turning a corner and the emotion involved when interacting with other drivers on the road. Racing games do a good job of taking this shared experience and amplifying it by creating a time based reward system that is dependent on your skill as a driver. Additionally, the sound and fury of driving is often abstracted from the realistic simulation going on under-the-hood in order to make a more compelling and fun driving experience.
Study Specifics:

This sound study was taken on an informal basis as a way to expose aspects of vehicle sound design, aesthetic and implementation in comparison across a number of current generation racing games.

  • Recording was done using analog stereo- RCA outputs into a DVD recorder in conjunction with the recorded footage.
  • The dynamic range remains intact for each game's audio capture, while levels have been normalized individually per game for each video to suit the online-video delivery method.
  • Whenever possible, the audio settings in the menu were adjusted to highlight the specific vehicle component highlighted.
  • The intention of the capture and delivery was not to present hi-fidelity examples, but instead to create as clear a representation possible within means.
  • It was also not our intention to compare loudness or dynamic range between titles.
This study doesn't address the difference between a racing "game" and racing "simulation". While simulation data is used in both types of game to present an experience to the driver (player), a racing game's intention is usually centered around a feeling of speed or fun and not necessarily created for the purpose of simulating fully the experience of driving a car. (This would be an exciting future sound study!)
We hope that the informal study is able to show a progression in sound and implementation technique throughout the current console generation and that it may help expose people to some of the intricacies of the genre.

History of Racing Games

From the first pitched sinewave the acceleration of vehicles in games has been intrinsically linked to the feeling of speed. Whether the flag-drop of Atari’s Dragster or the physical gear shifting lever of the arcade cabinet for spy-hunter, attempts to bring together the driving experience and gaming experience has been inexorably linked.



There is no greater indicator of the loss of speed then the squealing sound of loose rubber on the road. Present in some of the earliest racing games, the screeching sound of tires has always meant one thing: you're slowing down and losing precious time. It's a sound that immediately puts you on edge. At some point during the current console generation the sound of the rubber meeting the road transformed into a fully dynamic and multi-layered symphony of squeals.

Fundamentally tied to the tire physics simulation, the mathematics involved with keeping cars on the road are anything but simple. Equally complex are the sound and vibrations coming from the independent grip and slip of all four tires simultaneously. As David explains over at Track Time Audio, there is more to a tire than just watching the wheels go round. If the intention is to accurately represent this discordant noise, in an attempt to allow the player to filter through the noise for information, then our industry is edging ever closer.

Early titles in the current generation, like XBOX 360 launch title Project Gotham Racing 3 and Colin McRae's Dirt, the sound of tire friction seems to a be a relatively simple representation of either squealing or not. I say relatively simple, because the leaps and bound that are to made in the interim years are staggering when viewed through the lens of time. That's not to say they didn't give players the information needed to steady their game and self-correct on their way to the finish line. Multiple loops seem to be triggered each time squealing begins, crossfading on and off, depending on information regarding slippage from the tire physics simulation. This slippage or amount of grip would come to be a significant contributor to increased dynamics of tire sounds, but games released early in the console life cycle seem to have less detail and rely on a single sample to represent all four tires.

The second coming of racing games for the X360/ PS3 begin to exhibit the traits of a tire model that leverages the tires as four independent musicians in the howling rubber quartet. Disjointed pitches, independent positioning, and greater dynamic changes adding to the amount of hum and squeal are all part of the current crop of racing game front runners. The most pronounced examples can be found in the latest iterations of both Dirt 3/Grid and Forza 4. Each of these employing tire sound positioned approximately, with the sounds of friction reflecting the players expectation. As a car drifts around a corner and fish tails quickly to a stop, each tire complains in it's own announcement of velocity from the appropriate location on the vehicle. The physical dynamics of this chorus can be visualized in Forza 4 using the “Telemetry View”, which tracks tire friction per-tire.

Additionally, according to Creative Audio Director Nick Wiswell at Turn 10:
“in the new tyre model, we have hundreds of different sounds and skids for each surface type. In fact, it’s more complex than the engine model overall because we’re tracking lateral and longitudinal forces as well as the amount of load on each tyre – there are different sample sets for loaded/unloaded. As you go into a corner and turn, one tyre’s being forced into the ground, while another’s being slightly lifted off the surface and the sonic characteristics in each case are completely different.”
If those examples don't grab you, then you may miss the subtlety at work in Codemasters F1 series...but if you want to hear the defining moment that racing enthusiasts have been waiting for, you'll want to listen closely and turn up the low end. There is a howling sound that comes from a tire as it approaches it's grip limit - the point at which a tire loses grip and begins to squeal. While also present in the current crop of recent titles, the higher frequency tone of engines in the F1 series make it easier to identify when this hum begins.


A combination of parameters coming from the tire physics being used to control the dynamically change between content sets, as well as modify the sound of samples in response to the changing tire physics has opened up a deeper level of communication for the player. No longer simply an indicator of traction loss, sound can now be used to better anticipate the approach of slippage (and loss of momentum). With that additional information, players can further tune their auditory senses to ensure the maximum speed throughout the race.



Sound has a unique ability to communicate directly to the player in a way that is impossible to represent visually – all of the power and fury of an engine revving, for example, can be transferred directly from the speakers to the player’s brain in a pure expression and feeling of speed. Like the oft-cited survival horror genre, attempting to play any racing game without sound immediately reduces your sense of immersion and removes a critical component from the experience. The sound of the engine, like a purring cat or lulling drone that runs throughout a race, is positioned front and center and giving a constant indication of speed in a way that the visual speedometer can't approach.

Likely some of the most complex and specialized technical sound design going on in games today lives at the heart of most racing games. In order to accurately model the sound of a vehicle engine, in direct response to the interactivity of the simulation, there are many challenges that need to be overcome in order to represent each aspect as realistically as possible. Understanding the fundamental approach to current engine implementation is critical to understanding what content is required, in addition to how that content will be used at runtime. Due to the specialized nature of racing games, most developers require a heavily modified audio pipeline in order to handle the special needs of audio for vehicles. Gone are the days of simple sine waves pitched based on the RPM of a car engine, the standard today relies on a complex and constantly shifting puzzle of loops and synthesis modified parametrically from simulation data. Using everything from FFT analysis and granular re-synthesis to distortion and time based effects, like delay and flange, to create dynamics in response to player interaction.

FMOD Engine Designer
The engine is also there as a reminder, as Mike Caviezel recently said in the racing game-focused Game Audio Podcast #18, that “The Car is the Star”. Whether implemented using a loop based system, granular synthesis, physical modeling, or a hybrid, the three main sound components of the engine include the exhaust system, air induction, and the internal mechanics of the engine. Once the content has been established the resulting sound of each engine component is then tied to the same vehicle simulation used for every other aspect of the game using parameters to drive the change and modification in content. The earliest racing game engine sounds used the simulated vehicle RPM parameter to drive the pitch of a square wave. An increase in RPM meant an increase in the pitch of the engine note and, even at this basic level, communicated to the player their current speed.

It's clear throughout the study that racing game sound has made tremendous leaps, even within the current generation of titles. The true testament of good vehicle engine design is currently measured by how natural this progression of RPM is handled during acceleration and deceleration. Taken from its natural recorded state and modified parametrically using DSP, an engine sound can quickly leave the realm of believability if not accounted for in the technical design. Early titles during this console life cycle using a loop based engine sound methodology are easily identified by the synthetic quality that an engine sound exhibits when being dramatically pitched. It's possible that a look in the rear-view mirror from the future will render today’s forerunners of racing game sound as similarly faulted, but from here the improvements over the past few years have been substantial.

Starting with Xbox 360 launch title Project Gotham Racing 3 (PGR3) the genre was off and running. PGR3 was an alarm call for racing games and kicked off current-gen vehicle simulation in a big way compared to some of the other releases. Even stacked against Codemasters first step into the Dirt series it breathes with a natural sound un-marred by much of the “pitchi-ness” that has plagued many racing game vehicle engines. Before their recent studio closing, Developer Bizarre Creations would go on to a 4th iteration of the PGR series before applying their work on vehicles to the arcade-style power-up game Blur and the cinematic vehicle presentation in James Bond's 007 Blood Stone.

While the Dirt series seems (in retrospect) like a slow start in the engine sound department, it wouldn't take long before Codemasters punched the turbo and rapidly accelerated their engine sounds into first place. Beginning with Dirt 2 and continuing in Dirt 3, their engines attained more detail, more depth, and became less synthetic sounding. Something measurable was starting to happen between each new game, a continued harnessing of parameters, more layers, deeper integration and likely a bit of magic under-the-hood. Regardless of whether this was due to the introduction of granular synthesis in Dirt 2 (as indicated on Simon N. Goodwin's website) or just the continued march of technology, the improvements are marked and important steps towards furthering realism in engine sound simulations. Also, the addition of engine damage states (when an engine has been pushed too hard for too long, or the car has been impacted one too many times) brought another level of realism to the simulation. The Dirt 2 Damage footage has an example of engine damage and the resulting sound (what sounds like a loop of knocking and pinging that's tied to the vehicle RPM).

Being able to take a step back and see the progression of sound over a series of iterations has proved to be a valuable way to asses the progression of technology and innovation within such a narrow genre. This is especially true in the case of developments in the Forza series from Turn 10 Studios. Forza 4 takes a huge step towards addressing long-standing fan opinions that the vehicle sounds in the Forza series are “too clean and clinical (sounding)” by introducing a customized version of the Trash Distortion plug-in from iZotope. Used throughout to add a visceral feel to the engine sounds, there seems to be a direct correlation to the power available to a car and the amount of distortion that get's applied. The extreme results of this new technology can be heard towards the end of the Forza 4 Engine capture. In a talk at AES NYC in 2010 Mike Caviezel showed off an in-game debug that allowed Turn 10 to tweak parameters related to the sound simulation realtime, allowing them to apply unique distortion models depends on the vehicle. He goes on to explain in Game Audio Podcast #18 how the distortion effect was applied to the master bus for a vehicles engine component.

The F1 series from Codemasters presents a bit of a “wild card” in comparison to the other racing games spotlighted. With the engines of a formula one car often being refereed to as “the buzzing of a million mosquito's” or “a pack of wild bee's” it's easy to spot the difference between them and the sound of rally cars. F1 2010 lead audio designer Mark Knight explains the technical implementation behind the engine system:
“We use a fundamentally granular approach, analysing the constant pitch change of a rev sweep, and depending on how many cylinders in the engine work out how many cycles there are – with a cycle being an individual ‘grain’,” explains Codemasters lead audio designer Mark Knight. “If you need to play the sweep faster, you’re cutting out cylinders and if slower, you’re adding them. “Plus, we have a system to deal with timbral shift – a bit like those plugins that can shift a formant without changing the pitch and shift the pitch without changing the format. It’s a mixture of tech all in one package.”
The results of this technology are a smooth sounding engine simulation that capitalizes on the unique qualities of the Formula One engine model.


Ultimately, the dramatic arc of engine sounds during the current generation boils down to the increased realism by minimizing synthetic pitching artifacts, introducing granular techniques, deep implementations of parametric DSP (including distortion), and a greater understanding of audio content needs.

Post Engine Notes:
Another exciting technology that has been recently announced is Crakcase Audio's REV, which attempts to put the power of vehicle simulation into an authoring application.

Crankcase Audio's REV Analysis View
Crankcase Audio's REV Simulator
Also notable is 2XL Game's XLR8 technology that uses a iDevice to augment the sound of your car using the stereo. But is there a danger in turning real life driving into a game?

Additionally, DragFX makes your car sound like a muscle car.

Designing the Sound of a Real Car: An Audi, from Silence to Noise




Similar to the advancements in tire squeals, surface material sound has progressed from single loops to the fully realized four-tire-independent-dynamically-controlled and surface-dependent sound of travel and friction. In the same way that footstep surfaces help connect the player to the world and sell the experience, tire surface materials have diversified to meet the needs of many different race track types. These surfaces need to be represented in many ways across the experience, from the sound of gravel spitting-up beneath the wheel well to the subtle differences between grass and dirt. These details fill in the experience and further fortify the reality of the world and create a richer experience for the player.



The sound of a car tire's on any surface is like the icing on the cake of a tire simulation. Somewhat superfluous on tracks that feature only a single material, but nevertheless important in balancing out other aspects of the experience. Surface sounds support the indication of a less than optimal speed - if you're hearing any significant road noise, it's likely that you've veered off optimal path. While this is not necessarily true for games in the Dirt series (which prominently feature, well...dirt and other main surface materials on a single track) for any racing game centered around hot asphalt, clean lines, and fast cars, the sound of surface materials are an indication of failure. In this way it can be a punishing sound, only heard when precious seconds are being added to your lap time.





The camera in racing games serves to enable the player with their preferred visual accompaniment to the driving experience. The accurate representation of sound in the various perspectives is accomplished through the use of DSP, Emitter Placement, and Mixing. Across the different titles it’s interesting to hear the different techniques being used to help root the player in each visual mode.





Dynamic Reverb, dynamic damage, dynamic environments. The one thing these all have in common is the fact that the dynamic aspect of each has a corresponding need to be represented with sound in order to complete the experience and feeling of being a part of the action. From the sound of your fading engine as you tear through a tunnel to the head on collision with the ballard, the visceral and subliminal sound brings you closer to an experience that few will ever have the chance to experience; and in the case of a high velocity impact, thankfully!

The environmental sound of the world is something that has made great strides. With the increased CPU of current generation consoles, developers have been able to harness the potential for realtime reverb in order to more accurately reflect the sound of confined or reflective spaces. While the affects of reverb reflection may be difficult to detect in an open track race, we were able to find several examples across each of the games. From tunnels to skyskraper-lined city streets, warehouses to bridges, the diversity of reflective surfaces makes it clear that today's developers are moving closer to the representation of realistic environments.

In an explanation of the environmental system of Blur (an iteration of racing game from Bizarre Creations after the Project Gotham series) from "technical audio guru" Nick Bygrave:
“To make the car sound part of the world we have two systems. Firstly, we ray-trace into the world to obtain location and material properties, playing an appropriately delayed and filtered version of the car from the reflected image position. This is then mixed with a frequency domain verb for late reverberation – there are zones on the track tagged for different presets. This creates a simple but effective model of the changing environment, and gives the player useful game-play feedback. “
Simon N Goodwin explains the use – and difference between- reverbs on the PS3 vs. Xbox 360 versions of Dirt 3:
“There are six independent reverb units running in the PS3 version, versus two stereo ones on Xbox360. These are not just for reflections in tunnels or when you get close to trackside objects – they works beautifully for reflections from other vehicles too, and give exciting effects when the car goes out of control”
 

Also interesting to note with regards to not only environmental sounds but specifically playback of sound is Codemasters use of Ambisonics.

In a quote from this interview with Simon N Goodwin:
“The technique we use in our games, called “Ambisonics”, means that we can use all the speakers in order to guide sounds. You can use speakers on one side of the listener to guide sounds into intermediate positions between the speakers on the other side. So all of the speakers are working together in order to give you what’s called a “sound field”. Now we can place a sound convincingly at any point around the listener.”



The user interface in any game serves as a hub around which the wheel of gameplay turns. At best it is an iconic representation of the experience that serves the design choices and aesthetic, both visually and through navigation. As a player, it's your first opportunity to interface with a new experience and in some cases defines what is to come once you transition to play.

The majority of racing games aim to make their menu system as communicative as possible. Anyone working to create good design knows that this is no easy pursuit. The clearest example of a clean aesthetic comes in the form of the menu's in the Forza series (and to a similar extent in Dirt3). The antiseptic hospital-white background coupled with surgically-sharp text melds seamlessly with the upbeat-almost-edgy electronic music from Lance Hayes (DJDM) and serves as a window into what's to come. 

In a recent interview with Track Time Audio, Forza 4's Creative Audio Director Nick Wiswell says:
“UI should be a Zen place, the calm in between the storms of racing. We wanted to really push the aggression on track with the new car audio, so making sure the UI was a breathing spot between races became even more crucial this time. ”
Accentuating this contrast was the goal of creating a “dirtier” engine sound than what was used in previous iterations (see Engine Sounds).

Other examples of the WYSIWYG menu design perform similarly, attempting to imbue their menu's with a feel that attempts to mirror the in-game experience. The Need for Speed: Shift series angles for a more dramatic vibe throughout - the intention seemingly to keep your adrenaline pumping, nerves on edge, in preparation for the next impending race. This runs parallel to their intended over-the-top racing experience and is further reflected in the addition of “Unleashed” to the title of NFS:S2.

In contrast, the down-tempo smooth jazz styling of Gran Tourismo 5 aims for a different audience and experience altogether. After slipping into the menu I immediately felt much older than my years - this felt like an experience aimed at a mid-life-crisis early adopter cross section of PS3 owners who bought in for the shiny piece of kit to be prominently displayed in their living room. Visually it looks like a retro-dated Xbox Live menu scheme and includes the ability to zoom in and out making sure that geezers don't have to squint to make things out. UI sounds are percussively synthetic and vacillates between fanfare, high frequency beeps, and something akin to magic? You can probably tell, this is not a UI that I felt defined “me” as a virtual race car driver.

The folks at Codemasters have been experimenting with something a bit different in the way they present their introduction to the user experience. Beginning with Dirt 2 (and subsequently branching to their, then, newly acquired F1 license) they introduced a sort of storyline that utilized a traveling trailer as the user interface hub around which a series of racing, prizes, and celebrity sightings followed. Entering the trailer gives you access to the world map, races and setup, while the crowd sound outside the trailer is low pass filtered appropriately. Stepping outside releases the filter on crowd noise and, after selecting your vehicle for the race, uses an iconic photographing motif to transition you out of the interface and into the load screen. It would seem that their grand attempt at straddling the line between pure racer and story-driven game in the Dirt series was short lived, as Dirt 3 returns to form with a slick new menu driven interface. The F1 series maintains the same "trailer" approach, adding in the pit/ garage as a place to tune your vehicle and eventually begin the race.


Conclusion

 It's been a wild ride. Thanks to the man-on-the-street perspective of David Nichols from Track Time Audio, my education in racing games has extended outside of the confines of the living room and onto the kart track. Along the way I've had the pleasure to engage professional racing game audio enthusiasts in conversations surrounding their pursuits of passion.It would be great to hear any further insight, please feel free to continue the discussion in the comments.

Damian & David




Posted by lostlab at 7:15 AM
Labels: , , , ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)