Panel 15: New Immersive Perspectives

Panel 15: New Immersive Perspectives

Dr John T. Murray and Mark C. Marino, Rebecca Rouse and Lissa Holloway-Attaway

In Voices in the Hollow Reach: Spatial Audio Storytelling in VR, Dr John T. Murray and Mark C. Marino argue that although VR is often billed as a visual experience the experience is contextualised by audio distributed in a three dimensional space. Popular platforms like Altspace VR and VR Chat rely on voice to bring their worlds to life. In this paper, they present lessons derived from their initial exploration of VR in an interactive narrative experience entitled The Hollow Reach, an exploration of traumatic loss and recovery, Coming to grips with what is beyond your grasp.

Scene one: a spacefleet starship at red alert investigating a distress call. The first officer reports an enemy ship. The player must navigate this crisis with the help of the crew. But this sci-fi world turns out to be merely an entry point.

As the action-packed world of the starship dissolves along with the sci-fi soundscape, the player character removes their VR headset to find their dreary apartment, drenched in self-pity. The soundscape echoes with isolation: an air conditioner cycles on and off, the refrigerator hums to itself, and the answering machine takes another message. What’s more, the player can’t seem to hold on to anything. Everything slips from their grip.

To bring this project to life, Murray and Marino have relied on the voice talents of Syni Champion and Randy Vi. These voices, they argue, transform the experience from generic set pieces (since the 3-D models and environs were purchased in the asset store) to encounters of emotional depth. While the spatial arrangement of sound helps create physical settings, these vocal performances provide the emotional setting, as the characters call for the player’s attention both in the fantasy space of the ship and from the real world. By using sound in this way, Murray and Marino are trying to “block” the player’s movement on the virtual stage.

In When You Hear the Chime: Amplified Reading from Vinyl to Augmented Reality, Rebecca Rouse and Lissa Holloway-Attaway, examine the role of audio technologies in connection with the codex across a range of time periods and forms, to develop a nuanced understanding of audio, affect, augmented reality and the reading experience. Beginning with the advent of technologies of recorded sound, a genealogy of intermedial storytelling with sound is traced. Examples include mechanical sound synchronization in early cinema, Edison’s wax cylinder doll toy, children’s books accompanied by LPs and later cassettes, and embedded digital sound chips in books. Finally, the authors’ own work is provided as a case study, with the presentation of their AR (augmented reality) artist book, Simmer. 

Simmer (2019) combines traditional pop-up and movable elements with a mobile AR soundtrack. The audio includes both original music by composer Brendan Padgett, as well as dialogue and narration. The audio moves among different semiotic registers and sites for engagement for interactors who must parse the dispersed information that slowly unfolds, like the book. Interactors discover the story in non-linear pieces through the objects (dolls, LPs, miniature books). The audio situates the interactors in different states of proximity to access the story; sometimes narrative voices, confessional and diary-like, invite the interactor to come close. Other times, voices place the interactor outside the story, as an eavesdropper. 

Musical scores, constructed and deconstructed as themes for primary characters are composed according to their placement within the book – on linear accordion folds, or in non-linear dollhouse space, allowing the material setting and structure to contextualise the sound. This presentation examines these affective relationships between audio, image, text, and interactivity to provide a deep and historically informed understanding of the role and potentials of audio and augmented reality in conjunction with unique codex constructions today.