Chapter 12 of “Designing Immersive 3D Experiences” focuses on audio for XR, and starts with a listening exercise. I needed to close my eyes for five minutes and listen to all the sounds around me, then draw a chart showing what sounds I heard and their approximate position in my space. I did the exercise at work during my down time, and drew the chart below.
Focusing on the sounds let me pick up on the “background noise” I normally don’t notice, like the air conditioner and hum of the overhead lights, as well as sounds I usually hear, like my chair creaking as I move. This background noise is Ambient Sound, sounds like wind and far away conversations that give a sense of place to where you are. These sounds are used in XR to make the virtual environment or elements feel more “real.” If a beach scene in VR uses the sounds of the waves, wind, and seagulls in the program, the user becomes more immersed in the scene than if no sound was used. If the beach sounds are “placed” around the user, they become Spatial Sound, audio that simulates depth or direction. This way, the user can tell what sound sources are in front of or behind them, and how far away from them sound sources are. The Sony Sound Demo and preshow for the Museum of Science’s Omni theater are good examples of spatial audio. Feedback Sound, or audio sound effects that are synced to certain actions in XR, are another type of audio that help users navigate virtual and mixed reality. A chime can indicate unlocking a treasure chest in a VR game, for example, while a buzzing sound indicates an error or problem in performing a task.
This week’s design challenge is to design my own soundscape for XR, and draw a chart showing what sounds will be used, and where they will be located around the listener.
An extra challenge is to create this soundscape by recording the sounds on the chart, and bringing them into a design program to make my own spatial soundscape. I’m not one to back down from a challenge, so I decided to create an immersive soundscape using WWise audio editing software. Unfortunately this software had a very steep learning curve, and I struggled to move sound files around a virtual space, and then to export the files with my changes intact. After much trial and error, I was finally able to export my edited audio files, and upload my spatial soundscape to YouTube. Feel free to check it out below, and let me know what you think in the comments section. Next time I will tackle the final exercise in the book: creating an AR interaction using the full design project. Look forward to it!
Sources:
“360 Spatial Sound Mapping Demo-Sony.” YouTube, uploaded by Sony Electronics, July 20th, 2021, https://www.youtube.com/watch?v=VGL3x3Cc1io
“Mugar Omni Theater Introduction-Who Put the Bomp?” YouTube, uploaded by 2HHB, February 7th, 2013, https://www.youtube.com/watch?v=MHK2-BVfUzs
Stevens, Renee. Designing Immersive 3D Experiences. Pearson Education Inc., 2022, pp 301-324.