Dr Henrik Schønau Fog - Heriot Watt Digital Storytelling Lab

Investigating Engagement and the Desire to Continue

in Interactive Storytelling Application

- Experiences from a RIDERS exchange visit

at Heriot-Watt University, Edinburgh

 

 

Henrik Schoenau-Fog

Assistant professor at University of Aalborg Copenhagen

Department of Architecture, Design and Media Technology

Section of Medialogy

 

 

In 2010-2011, I developed the "First Person Victim" application (FPV) with colleagues at Aalborg University, Copenhagen. The FPV is an interactive dramatic experience, which is intended to initiate discussions about refugees and victims of war (Schoenau-Fog et al., 2010)[i]. Furthermore, I have developed a player engagement framework based on data from over 200 respondents and interviewees (Schoenau-Fog, 2011a)[ii], and an evaluation method of engagement through the concept of "Continuation Desire" for interactive experiences, games and interactive storytelling applications.

I presented this work at the 2011 International Conference on Interactive Digital Storytelling in Vancouver (ICIDS 2011) as a case study on how to evaluate engagement in interactive storytelling applications like the FPV. During the conference, I discussed with Dr. Sandy Louchart from the Digital Story Lab, Department of Computer Science at Heriot-Watt University (HWU) the possibility to investigate possible correlations between the data from the evaluation method I devised (self report measures of engagement conducted intrusively during run time) and brainwave electroencephalogram (EEG) measures and data from other biofeedback assessment methods. Dr Louchart expressed his interest in hosting a research visit so as to conduct this investigation and explore new ways to measure engagement in interactive storytelling applications. Therefore, I was obviously very excited to be invited to Edinburg in the late summer of 2012 with the aid of the RIDERS Exchange Visit Grant to start a collaboration so as to scientifically investigate the notion and applications of Continuation Desire further. 

What is Continuation Desire?

The concept of Continuation Desire was developed in order to create a straightforward way of investigating player experiences in digital games and was based on online surveys asking hundreds of players what made them want to continue when playing (Schoenau-Fog, 2011a)2.

Continuation Desire can be described as a user's willingness, tenacity, or persistence to continue an experience - this can for example be the desire to keep playing a digital game or to keep exploring an interactive story world.

Continuation Desire is initiated by the motivation to begin an experience, followed by the desire to continue the experience and finally the interest to try an experience again. The process can be described as when a user wants to begin an experience due to the intrinsic user-defined objectives (e.g. How does the story end?) or the extrinsic game or application-defined goals (e.g. Identify the murderer!). These objectives may be reached through activities performed in order to accomplish said objectives, and users may also want to continue due to these activities – for example the activities of experiencing the story, the character development, the experimentation with storylines or simply exploration of the story world. The accomplishments that users might strive for are for example advancement in the narrative or completion of the story, as well as the achievement experienced when a new event unfolds. Finally, emotional affect may also support the desire to continue when the objective is accomplished or while the activities are performed. If the resulting affect is either positive, negative to a limited degree (for example if it is slightly frustrating that it is hard to find the next event), or for example if the user feels the sensation of being absorbed in the story world, he or she might want to continue by trying to accomplish new objectives, and the process can begin again.

Usually games and interactive experiences are investigated through multifaceted concepts such as Flow, Immersion, or Enjoyment. However, the prerequisite of the experience of these concepts could be argued to be a user's willingness to continue the experience. Nevertheless, the concept of presence – i.e. the feeling of being in a virtual world mediated through the senses – can be experienced even if the user does not want to continue. For example a 100% realistic flight simulator might be boring to experience, even if the user feels like he or she is in a plane and piloting it.

Additionally, any game, interactive experience, or Digital Storytelling application would require at least some degree of Continuation Desire experienced by its users in order to be a successful experience.

The other approaches (Flow, Immersion, Presence and Enjoyment) will, however, require the researcher to assess or take into account the nature of the artifact and its content. This becomes a issue when it comes to evaluating Interactive Storytelling as a user experience might (should) be different for each participant. Continuation Desire is therefore also intended to evaluate a user experience independently of the medium or the actual artifact.

In this context, stepping back to a simpler concept such as Continuation Desire and detecting signs of disengagement (as opposed to engagement) opens the door to actually starting to develop an evaluation framework adapted to Interactive Storytelling (amongst all other narrative media). 

Evaluating Continuation Desire through interruptions

 

A simple approach to identify engagement through Continuation Desire would be just to let the user keep playing a game, exploring an interactive story, or interacting with other forms of digital experiences until he or she becomes disengaged and stops the experience. At this point, users could be interviewed about what made them stop and the reasons for disengagement could be identified.

However, it is also valuable to assess engagement through the level of continuation desire before, during and after an experience or activity. This method is intrusive in its design and interrupts the user during the experience. For example, a user is navigating a virtual world, experiencing its narrative and at certain points or events in the experience, the simulation stops and the user is interrupted to be asked how much she or he wants to continue followed by a number of questions regarding the experience. This approach has the advantage that one has to have a certain level of the desire to continue in order to proceed, thus isolating the time period in which the decision not to continue has been made.

The method, where users are interrupted during their experience, has for example been used in the evaluation of the "The First Person Victim" serious game / interactive storytelling application (see Schoenau-Fog, 2011b)[iii]. Results showed that even though the experience was unpleasant (the user plays a victim of war in a 3D environment) up to 40% wanted to try the experience again despite their experience of negative affect. This investigation also acquired data on reasons for wanting to continue or not and the results showed that pupils who tried the experience reported that they wanted to continue due to the experience of the story, the exploration of the environment, experimenting with story outcomes, experiencing the characters and curiosity about what was going to happen. Pupils also showed signs of affective learning by trying participate and respond in order to help other characters in the virtual environment (Schoenau-Fog, 2012)[iv].

Even though the intrusive method can acquire valuable data, the goal is to be able to evaluate an experience without interruptions.

In other words – how can the desire to continue be evaluated non-intrusively?

This was one of the questions, which I was particularly interested in investigating at the RIDERS exchange visit at the Digital Story Lab at Heriot-Watt University.

Investigating possible non-intrusive methods for measuring Continuation Desire non-intrusively

The Digital Story Lab (DSL)[v] is a group of researchers headed by Dr. Louchart and Dr. Judy Robertson, and along with the school of Engineering and Physics at HWU, DSL has developed a framework for synchronizing media outputs such as video, game activity, and haptic feedback with bio-physiological data recording devices. This framework can be used to record in sync a number of different user experiences through an EEG device. By using this framework to test various applications, it is possible to acquire knowledge about how engagement in interactive story experiences are experienced from a physiological perspective.

However, even if the system is able to record all sorts of data, we still need to isolate and  define the parameters, which indicate continuation desire (and disengagement). In order to explore this field further, Dr. Louchart introduced me to a number of colleagues, which we interviewed during my stay at HWU:

Dr. Thusha Rajendran, Reader in Psychology at HWU is interested in developmental psychology and he found the concept of investigating continuation desire during an experience interesting as psychologists usually do not look into the process, they are generally interested in inquiring about the experience before and after, not during the experience. During the interview with Dr. Rajendran we agreed that a possible experimental design could be based on 'within-the-subject' experiments where each test participant try out different conditions of the experience. We also discussed how metrics from games / interactive story applications such as mouse clicks, or distance travelled in the virtual environments could be utilized to acquire non-intrusive and nonbiased information about users' engagement during their experiences. Furthermore, we discussed the potential of using observations by recording video and thus using real time capture of what users are doing in order to support other data sources.

Dr. Judy Robertson, senior lecturer in Computer Science at HWU is interested in game based learning, computational learning, computational pedagogy, and pedagogy driven adaptive systems. With Dr. Robertson we discussed in detail possible experimental design and how the different parameters should be isolated to acquire valuable information about the desire to continue.

Professor Ruth Aylett is from department of Computer Sciences in the School of Maths and Computer Science at HWU. She researches intelligent graphical characters, affective agent models, human-robot interaction, and interactive narrative. With Professor Aylett, we talked about the future of interactive storytelling and discussed how to "hide" evaluations by embedding investigations of engagement within the experiences.

Furthermore, we also had a Skype meeting with Professor of Psychophysiology Stephen Fairclough from School of Natural Sciences and Psychology, Liverpool John Moores University. We agreed that the measurement of engagement is a very valuable data source, which is often overlooked, and that attempting to measure the desire to continue is an approach with potential. We then discussed the multitude of possible ways to acquire brain wave data (EEG) and other bio-feedback data such as bloodpressure and heart rate in order to further isolate parameters which may indicate the desire to continue.

Finally, Dr Louchart and I moreover discussed the concept of falsification and the possibility to use not only causes of continuation desire but also disengagement triggers to predict when users want to stop. We identified several causes of disengagement among not only interactive stories but also other narrative media, such as TV-series, sport events, comic books and films.

After having discussed the concept of Continuation Desire with the above mentioned persons in very different research fields, it became apparent that Continuation Desire is a relevant and valid concept in all the areas we consulted during my stay. Dr. Louchart and I also came to the conclusion that there are enough metrics available so as to explore the evaluation of continuation desire in interactive storytelling even further. 

What are the next steps?

 

We are now working on acquiring a more detailed view of the field both theoretical as well as instrumental with the intention to develop an experimental design focusing on how to evaluate continuation desire in interactive storytelling applications (and other interactive experiences). The objective is to assess such data during run time with a combination of various non-intrusive measures (psycho-physiological / biofeedback measures, in-game metrics and observations) in combination with a self-reported assessment. The investigation will focus on exploring correlations between aspects of engagement and disengagement (e.g. attention, cognitive loads and incentives), which can be identified with the Digital Story Lab system and in self-reported measures of Continuation Desire.

The collaborative research with the staff members I interviewed during my stay at HWU and other partners aims at setting up an experiment or series of experiments in which we investigate various metrics so as to determine whether or not one could quantify the continuation desire towards the automation of narrative-based artifacts (or other cultural experiences).

The plan is thus that our findings during my RIDERS exchange visit will be used to contribute to develop new ways to measure Continuation Desire, which can be used to evaluate Interactive Storytelling experiences in for example serious games, story worlds and live action role-play. Such methods may also assist the RIDERS community in developing novel forms of (adaptive) interactive dramatic experiences, which are driven not only by narrative cause and effect and traditional storytelling structures but also by the users' desire to continue experiencing engaging interactive dramatic environments.

I have been very grateful for receiving the RIDERS Exchange Visit Grant and I would like to thank in particular Dr. Sandy Louchart and Professor Ruth Aylett and their colleagues at the Digital Story Lab and Heriot-Watt University for making my week in Edinburg so inspiring.

Henrik Schoenau-Fog

Contact: hsf@create.aau.dk



[i] Schoenau-Fog, H., Bruni, L. E., Khalil, F. F. & Faizi, J. 2010. First Person Victim: Developing a 3D Interactive Dramatic Experience. In : ICIDS 2010 Lecture Notes in Computer Science.6432, p. 240-24. http://link.springer.com/chapter/10.1007/978-3-642-16638-9_32

[ii] Schoenau-Fog, H. 2011a. The Player Engagement Process – An Exploration of Continuation Desire in Digital Games. In Proceedings of Think Design Play - 5th International DiGRA Conference (Utrecht, Netherlands, September 14-17, 2011)

http://www.digra.org/dl/db/11307.06025.pdf

[iii] Schoenau-Fog, H. 2011b. Hooked! Evaluating Engagement as Continuation Desire in Interactive Narratives. In Proceedings of the Fourth  International Conference on Interactive Digital Storytelling. (Vancouver, Canada, December, 2012). LNCS, 7069, 219–230. Springer, Heidelberg.

http://www.springerlink.com/content/j1044l140159153w/   

[iv] Schoenau-Fog, H. 2012. Teaching Serious Issues through Player Engagement in an Interactive Experiential Learning Scenario. In : Eludamos Journal of Computer Game Culture.6, 1, p. 53-70

http://www.eludamos.org/index.php/eludamos/article/viewArticle/vol6no1-6

 




RIDERS | Heriot Watt University (Riccarton Campus) | Currie | Midlothian | EH14 4AP