Adaptive Music In Video Games

As gamers, we're all familiar with the concept of adaptive music in games. For instance, we encounter a combat scenario, and the music changes to a more intense composition. We're triumphant, and the music registers our success by playing the victory theme.

Switching between two pieces of music is commonplace. But what if we apply the notion of adaptation at the compositional level, dynamically changing the melody, harmony or instrumentation? It would then be possible to sculpt a piece of music to fit the contours of a scene, where the player is responsible for making narrative choices and the timing of events.

From the beginning, making music that is reactive to and synchronised with a player's choice has been at the core of our thinking. But more than this, we strive for a captivating soundtrack, music that immerses you in the wild, mysterious world of Arctic Awakening.

In this blog, I want to share some of what goes into writing, recording and controlling Arctic Awakening's music. Here's a deep dive into how I approach the technical demands of adaptive music in video games with an example from episode 1. Enjoy!

Sections: Form & Function

Different music forms have been used for centuries by composers to establish structure and hone composition skills. For games, though, the form of a piece starts with the scene. The structure of the game scene lays out what the music needs to do and when.

For example, I've identified four sections from the scene Night Frights that set out how the music progresses:

Wake
Alfie's Lantern
Search
Return to Plane

The composition has four sections to match the scene's structure.

Adaptive music using FMOD - Structure in video game music

When these sections are activated depends on the player, so discrepancies in the timing of triggering events are inevitable.

Looping

To accommodate the variation in timing, I split the composition into looping sections.

A good loop is a thing of joy! They're tremendously satisfying to make and very useful when the length of music required is undetermined. Effectively, once the play head enters a loop, the music can play indefinitely until the player triggers the next section.

The critical point in any loop is where the end meets the beginning. During this transition, any natural decay (the time it takes for the recorded signal to fall from 100% to 0%) at the end of the loop is interrupted when returning to the beginning. At this point, a noticeable artefact is audible, and unless the natural decay at the end is allowed to play out, the loop point is obvious.

To remedy the problem, I record all sections separately, allowing the instruments and effects, such as reverb and delay, to fade to silence. Then in FMOD, I use the transition timeline to crossfade the end of the loop with the beginning.

Using a crossfade creates a seamless loop where the reverb tails and natural sustain of instruments at the end of the loop play underneath the beginning as the play head makes its next pass.

Transitions

Whilst looping deals with the issue of undetermined duration, transition regions in FMOD manage the shift between sections.

Getting a smooth transition between the parts requires careful writing. Consider the chords of the first section - they need to work with those of the second. Any harmonic progression must make sense from measure to measure and on different beats within bars.

Whether on individual beats, smaller subdivisions or the beginning of bars, when the music is allowed to transition affects how quickly the music can progress. For instance, if the composition can only transition every 5th beat, then at a tempo of 120 bpm, the maximum time between transition points would be two seconds. Not a lot of time but enough to make synchronisation problematic in certain situations.

Using FMOD's quantisation feature allows me to set the number of bars, beats or smaller subdivisions between transition points with smaller values like eight notes making a more agile transition.

Transition Conditions and quantization in FMOD — Quantization set to one eighth note.

Sometimes the transition between sections can happen slowly with a long smooth crossfade, but the first music transition in Night Frights occurs in sync with Alfie activating his lantern, which happens in the blink of an eye.

Setting the quantise value low makes sense in this situation, but the design of the composition also plays its part here.

Composition

I spend a lot of time experimenting with compositions making many mock-ups before settling on a final idea.

Getting adaptive music in games to work as intended requires lots of testing. Night Frights, for instance, has been through several revisions, and each redraft has gotten me closer to my goals whilst simplifying and refining the writing. I'll break down the sections with illustrations from the score.

Section 1

Night Frights begins with a subtle percussive bed. A low drum ostinato and shakers form the rhythmic texture above which a synth drone and eerie vocals sit.