I would like to start this article by sharing with you a particularly illustrative video. The scene is taken from the film ‘The Artist’, directed by Michel Hazanavicius, released in 2011 and the winner of numerous international awards, including five Oscars, three Golden Globes and seven BAFTAs.

The film recounts the advent of sound in the world of cinema – until then, silent.

Have you seen it?

In this scene, the protagonist – a famous silent-film actor – has a nightmare: he dreams that objects, people, the world, suddenly start to produce sounds; everything has a sound, only he is left without a voice. I think this scene shows very well the importance and function of sound in association with images.

Since 1927, the year sound cinema was introduced, sound and image have been inextricably linked: the audiovisual was born. This concept, over the years, has taken on a wider and wider meaning: not only films, therefore, but any experience capable of combining sound and images. Including Virtual Reality.

Although the marriage between sound and images in the world of cinema will soon celebrate its centenary, we certainly cannot say the same for VR, where the new frontier of spatialized audio has been introduced.

The possibilities that VR offers are yet to be discovered, and its limits are being pushed further and further. With the advent of this new media, sound has become perhaps even more important than with other audiovisual channels.

The importance of sound in VR

Imagine you’re in a VR experience with the volume at zero. A character is talking to you, but you can’t hear what he says. Out of the corner of your eye you can hear the frantic movements of some people running away. You turn around, and you’re just in time to see a rather hungry dinosaur in front of you. Game over!

Well, if the purpose of this hypothetical VR experience was to confuse you, it would have succeeded perfectly.

Let’s try to imagine the scene again, but this time, with the volume turned up. You find yourself in front of a character who tells you, in an anxious tone, that a dangerous animal has escaped from the zoo. You distinctly hear the screams of people telling you to take cover; and at the same time, you hear the heavy footsteps of a creature fast approaching. A deep and frightening sound causes you to turn around, and when you find yourself in front of the T-Rex, you may have a better chance of getting to safety.

Mission accomplished: you will be saved and you can progress in the experience. All this has been made possible thanks to sound.

From this small example we can understand some of the functions that are conveyed by sound in VR.

The functions of sound in VR

• It advances the narrative through speech (in our previous case, a character tells us about the escape from the zoo of a dangerous animal);
• It promotes a sense of presence in a place, making us fully immersed in an environment (we know we’re in the middle of a screaming crowd);
• It triggers emotional reactions in the participant (people’s screams will activate a stress reaction and a state of alert in every one of us);
• It allows us to understand the localization of events (we know where the creature is coming from because we can hear the direction of its steps);
• It gives ‘weight’ and ‘body’ to the images. It seems a contradiction, but it is the sound that creates the ‘body’, and not the other way around (the T-Rex becomes huge and heavy because its movements rumble like cannon shots);
• Through the synchrony between images and sound – syncretion – it is possible to create the ‘suspension of disbelief in the participant’, a fundamental condition for the involvement of the audience, who suspends his perception and accepts to believe in the imaginary world in which the experience is set; even though he knows that it is not real, but virtual.

This last point is certainly the most important:

an immersive sound design for the VR must contribute to making the experience not real – but authentic.