Audio is the most immersive development tool you have at your disposal. It achieves a level of engagement with users that we can’t replicate with other senses or technologies. And this is especially true when it comes to the exploding industry of Virtual Reality.
Visual Immersion vs. Audio Immersion
The key selling-point of virtual reality is, of course, its unique immersive nature. When we think of VR, we may imagine ourselves wearing an Oculus Rift or an HTC Vive. With our visual senses, we are surrounded on all sides by a complete digital environment, no matter which direction we look. This is certainly a major technological achievement, and there’s no denying that it can be a highly entertaining and compelling way to experience a narrative or interact with a game.
But how immersive is this, really?
In order to visually perceive the environment itself, we have to look around. You could argue that we, in a weird way, have to actively pursue the immersion. And technically speaking, though it feels like we are surrounded, we are only processing sensory data on one directional plane at a time: forward.
Now think, for a moment, about how our auditory senses work in comparison to this. No matter where you’re looking, you hear audio from all sides. You hear the door opening in front of you as well as the monster growling at you from behind. Even when you move between rooms or maps and the audio changes completely, you are still hearing that change develop in 3D space.
The Ambiance of Rain (Example)
A very simple way to outline the instantaneous impact of audio would be to use rain as an example.
- Imagine a virtual environment in which you could see rain falling all around you, but there was no audio.
- Now imagine a virtual environment in which you are just listening to the sound of rain, but your eyes are closed. Which is more immersive?
Which is more successful at making you feel like you are in a different place, entirely? Our ability to hear is the only sense we possess that is truly 360 degrees. And therefore, audio is the fastest route to immersing a user or a player into a virtual environment.
The Immersive Elements of Music
Right off the bat, let’s look at some numbers. It has been scientifically proven that music can have an immediate influence on listeners.
Data from various experiments showed that:
- Music increased endurance in people working out by 15%
- Music reduced blood pressure in over 1500 patients
- Music influenced new feelings and emotions in the brain after only 15 seconds
If sound can instantly immerse us in a physical space, music immerses us into an emotional one. Our brains process a variety of complex data from music and translate it into a specific mood or frame of mind.
Notes and key signatures forge an emotional base for the situation.
- Major notes may encourage feelings of amusement or joy.
- Minor notes may encourage feelings of uncertainty or sadness.
- Major and Minor notes, together, may encourage feelings of beauty or bitter-sweetness.
Rhythm and tempo dictate the energy of the scene or environment.
- Slow and simple may emulate the feeling of rest and reflection.
- Fast and complex may emulate the feel of action and momentum.
Instrument choice can help develop characterization and context.
- Heavy percussion may trigger feelings of primal energy or battle.
- Powerful horns may trigger feelings of strong intent or willpower.
- Human vocals may trigger feelings of humanity or spirituality (something larger than life).
Hell, even a lack of music can be immersive if used with intent in the right context. If we are used to hearing music and don’t anymore, we’re put on alert, a little, and we pay more attention to what’s going on. Silence is golden, after all.
Of course, all these elements combine together to mix and match results in an infinite number of emotionally complex possibilities, and sometimes they can miss their mark.
The Mood of Rain (Example)
Let’s demonstrate this with another simple example of rain.
- Immerse yourself into a world entirely made of rain, and nothing else.
- Now, immerse yourself into this world made of rain, but with a mysteriously ominous twist.
- Or this one, a world made of rain as well as a fantastical element of beauty.
- Or, finally, a world made of rain, where strength and darkness are always at odds.
Break Out the Headphones…
In 2010, Hans Zimmer’s company Remote Control Productions released the Inception app for all Apple devices. The app is a kind of virtual reality experience entirely designed for the ears, alone. The result was a series of “dreams” unlocked by the user that captured the audio of their surrounding environment via microphone and then augmented it with distortions and effects to correspond to the rhythm of a music track from the film’s original score. In essence, the auditory senses would become the sole mechanism that separates you from reality and the virtual experience.
If you’re still not convinced about the immersive potential of audio (and have an iPhone or iPad), I’d highly recommending checking out that free app here.
Audio is the most powerful resource we have at our disposal when it comes to creating immersive experiences. There are very few alternatives that can create such a lasting impact on our brains with the same instantaneous effect as audio. Music and sound are vital components to any gaming experience as they can provide compelling player feedback and unique environmental characterization. But they become even more essential when used in Virtual Reality experiences as they complete the 360 degree immersion and can even be used as trigger cues to direct the player where to look.
Ultimately, audio is such an important part of virtual reality, because sound on its own is a kind of virtual reality in and of itself.