Panel: Immersive Technologies and Liveness

Panel: Immersive Technologies and Liveness

Sunny Teich (Victoria University of Wellington)

Literature Capture

The possible-worlds theory in literary studies suggests that each fictional universe is a possible version of the universe created by a human mind. And through reading, we are transported to this other world by devoting our attention to making sense of it. Explorations into VR as a technology reflect this same desire to be immersed in a different world and often concentrate on developing visual verisimilitude to the real world. While these visually-oriented efforts are admirable and important, literature capture takes a different approach.

Literature capture is an interdisciplinary approach to narrative, combining written prose with a live reading that is motion-captured and applied to a digital actor. The result is an instance of oral storytelling that is experienced in a spare virtual space. By limiting the visual experience to a representation of the reader and non-representational environmental elements (i.e. the events of the oral text are not being visualized), we explore how the existing immersive qualities of reading a literary text can cause a VR experience to become more immersive, and how the immersive qualities of a VR experience can cause reading a literary text to become more immersive. How do these interconnected aspects of both mediums augment and converse with one another? Through the practice of literature capture, we hope to create a new experience of narrative that is both deeply engaging and deeply affective.

My presentation will include an overview of the aims of this approach to VR storytelling and a demonstration of an instance of literature capture I created using a short piece that I wrote, performed and ported onto a digital actor in Unreal Engine.

Aste Amundsen (Fellow of Immersion with the South West Creative Technology Network)

Personalised responsive storytelling for live immersive experiences

Personalised experiences are a holy grail for immersive events, themeparks and experiential marketing. Personalisation and responsive storytelling is currently delivered through digital platforms and games, but with the launch of Disney’s Starwars Hotel, the success of Evermore, and closer-to-home developments like (my company) Computer Aided Theatre’s Live Events Narrative Engine, data-augmenting human-to-human interaction for In-Real-Life encounters is the next horizon in experiential storytelling.

Placing you-the-audience-member at the centre of an unfolding dramatic-arc is an aspiration these mediums share, and immersive entertainment audiences are enchanted by prospect. But how do event producers and storytellers move beyond the current limits of IRL personalisation, at best a character might address you by name or giving you an object, and into something more involving and potentially transformative for the audience member? We would need to know something meaningful about the person, and have the means of effectively employing that knowledge in our interactions with them. Assuming that the means of delivery exist, what things might be relevant to know about someone, in order to improve their experience? What represents meaningful parameters for storytelling personalisation?

Currently the most prevalent means of personalisation rely on profiling frameworks shaped to attain sales. What would a profiling system designed to achieve more satisfying and transformative human-to-human interaction, within immersive experiences, look like? Through my SWCTN Immersion Fellowship I am exploring existing profiling, psychometric and player-profiling frameworks and assessing how these track to the requirements of personalisation for immersion and transportation. I am also comparing contrasting modes of employing profiling in audience experiences from covert to audience-co-creative, and how these lend themselves to different types of storytelling.

Sandra Gattenhof and Nathan Sibthorpe (Queensland University of Technology)

Turning Tables: democratising the role of audient in narrative construction using medial interfaces

Much has been written about democratized writing practices in the field of theatre/performance (Schechner 2006; Lehmann 2006; Cobley 2001). In such a space theorists are not merely interested in the physical act of writing but rather they interrogate the joint construction of meaning. Developing an enabling platform for such embodied participation can some times be a stumbling block for theatre makers who have found it difficult to move beyond token engagement of audience members. This paper will explore a recent Australian performance work titled Crunch Time (2018) created by Brisbane-based company Counterpilot. The live work is a performative dinner party designed to model the processes of democracy. Seated around a projector-mapped dining table, participant diners use interactive tools to vote for every ingredient used in an elaborate meal. Viewed via live-feed video from a nearby kitchen, a guest chef prepares real food in response to demand. Each performance hosted individuals from public positions of leadership – politicians, artistic directors and CEOs who took their turn to cook up requests from the dining room with dangerously unpredictable results. Using the lens of intermediality the paper will explore how the performance developed a “meeting point in-between the performers, the observers, and the confluence of media involved in a performance” (Chapple and Kattenbelt 2006, p.12). This theoretical construct will be coupled with postdramatic theatre with particular reference to how the text was ‘wrought’ by components other than language (Lehmann 2006, p.86), and Eco’s (1989) open work in which in which artists make a decision to leave some elements of the work in unfinished form to multiply possibilities both in their performance and reception. This paper will investigate how Crunch Time engaged medial interfaces to allow audience members to experience democratized narrative construction. Two points of view will be considered – the performance maker and the audient. The paper will be of interest to creative practitioners engaged with digitally-mediated narrative construction and algorithmic software systems.

Enrica Lovaglio Costello (CalPoly and UCSB)

The inclusive forest – Measuring discrimination across cultures through immersive and interactive narratives

One of the most neglected and yet important frontier of extended realities is the exploration of emotions, perceptions, and behaviors by challenging multiple participants to act collaboratively in immersive environments. In the study of prejudice, for example, such investigations are difficult to achieve in lab settings. Virtual reality (VR) has proven to be an effective medium for this purpose. Technological advances have uncovered unprecedented and unexplored opportunities by using real-time feedback collected through the measuring of biometrics (brain, heart, and thermal variations). We propose an art project and a tool to measure racial and gender bias, by collecting biomedical data while participants are immersed in emotive narratives that explore discrimination across cultures.

In VR, participants collaborate and interact inside stories of racial and gender discrimination. Their personal and biomedical data are collected, analyzed and visualized in the form of abstract trees in a digital forest. Participants can contribute additional data by recording their own story of injustice and discrimination or tolerance and acceptance. Following the installation, we will engage in a data visualization phase to organize and analyze the personal, biomedical data and audio recordings. This data will be the basis for objective conclusions about cultural behavioral patterns in regard to discriminatory bias.

The participants of the art-based VR experience first explore a surreal digital forest, then collaborate throughout stories, which demand quick associations and decisions, while we collect their biometric data and time of reactions. This project’s innovation resides in the unique combination of a game-like artistic VR experience, that engages participants in a non-threatening manner, the use of telepresence, the interactive narratives and biometrics in the exploration of emotions, such as compassion and empathy. As the metaphor of the forest implies, all humans are equally plagued by biases and all of us can contribute to lessening its magnitude.