Panel 15: New Immersive Perspectives

Panel 15: New Immersive Perspectives

Dr John T. Murray and Mark C. Marino, Illya Szilak, Rebecca Rouse and Lissa Holloway-Attaway

In Voices in the Hollow Reach: Spatial Audio Storytelling in VR, Dr John T. Murray and Mark C. Marino argue that although VR is often billed as a visual experience the experience is contextualised by audio distributed in a three dimensional space. Popular platforms like Altspace VR and VR Chat rely on voice to bring their worlds to life. In this paper, they present lessons derived from their initial exploration of VR in an interactive narrative experience entitled The Hollow Reach, an exploration of traumatic loss and recovery, Coming to grips with what is beyond your grasp.

Scene one: a spacefleet starship at red alert investigating a distress call. The first officer reports an enemy ship. The player must navigate this crisis with the help of the crew. But this sci-fi world turns out to be merely an entry point.

As the action-packed world of the starship dissolves along with the sci-fi soundscape, the player character removes their VR headset to find their dreary apartment, drenched in self-pity. The soundscape echoes with isolation: an air conditioner cycles on and off, the refrigerator hums to itself, and the answering machine takes another message. What’s more, the player can’t seem to hold on to anything. Everything slips from their grip.

To bring this project to life, Murray and Marino have relied on the voice talents of Syni Champion and Randy Vi. These voices, they argue, transform the experience from generic set pieces (since the 3-D models and environs were purchased in the asset store) to encounters of emotional depth. While the spatial arrangement of sound helps create physical settings, these vocal performances provide the emotional setting, as the characters call for the player’s attention both in the fantasy space of the ship and from the real world. By using sound in this way, Murray and Marino are trying to “block” the player’s movement on the virtual stage.

In the artist presentation VRChat for amplification, exhibition and creation: an artist’s perspective, Illya Szilak describes two multimedia projects created with her longtime artistic partner Cyril Tsiboulski for VRChat; a massive multiplayer, free to play social VR platform, accessible by PC and any VR headset including Quest. Both Queerskins: Home and In My Own Skin function as art exhibition, explorable story world and performance space. Queerskins: Home is used to illustrate two key ideas related to extended content publishing – modular creativity and the use of multiple platforms/media types to build new audiences for existing work. 

Queerskins: Home acts both as an addition to the Queerskins world as well as an accessible segue into that world (currently consisting of a free, online interactive narrative, included in the Electronic Literature Collection vol. 3 and two commercially available VR experiences) for the predominantly young, tech savvy audience that frequent VRChat. In My Own Skin is an interactive photography/fashion project at the intersection of art installation, game, and performance art. It illustrates the potential to extend reach through the VRChat platform by using an artwork for multiple purposes; for wooing potential funders/exhibitors, for performance, and for sales of artwork. 

In In My Own Skin, visitors first choose an avatar to don; these are uniform, androgynous human figures dressed in the same white t-shirt that differ only in the message embossed upon them. These poetic words and phrases both veil and reveal the person contained within. Visitors become defacto performers, models in the space. They can then explore a Potemkin village of edifices which house/”dress” photographs of amateur models wearing garments, “soft machines”, which challenge conventional notions of gender, sexuality and identity. In addition to functioning as a 24/7 art exhibition, the space housed live walk and talks for visitors during CPH:DOX fest. 

In When You Hear the Chime: Amplified Reading from Vinyl to Augmented Reality, Rebecca Rouse and Lissa Holloway-Attaway, examine the role of audio technologies in connection with the codex across a range of time periods and forms, to develop a nuanced understanding of audio, affect, augmented reality and the reading experience. Beginning with the advent of technologies of recorded sound, a genealogy of intermedial storytelling with sound is traced. Examples include mechanical sound synchronization in early cinema, Edison’s wax cylinder doll toy, children’s books accompanied by LPs and later cassettes, and embedded digital sound chips in books. Finally, the authors’ own work is provided as a case study, with the presentation of their AR (augmented reality) artist book, Simmer. 

Simmer (2019) combines traditional pop-up and movable elements with a mobile AR soundtrack. The audio includes both original music by composer Brendan Padgett, as well as dialogue and narration. The audio moves among different semiotic registers and sites for engagement for interactors who must parse the dispersed information that slowly unfolds, like the book. Interactors discover the story in non-linear pieces through the objects (dolls, LPs, miniature books). The audio situates the interactors in different states of proximity to access the story; sometimes narrative voices, confessional and diary-like, invite the interactor to come close. Other times, voices place the interactor outside the story, as an eavesdropper. 

Musical scores, constructed and deconstructed as themes for primary characters are composed according to their placement within the book – on linear accordion folds, or in non-linear dollhouse space, allowing the material setting and structure to contextualise the sound. This presentation examines these affective relationships between audio, image, text, and interactivity to provide a deep and historically informed understanding of the role and potentials of audio and augmented reality in conjunction with unique codex constructions today.