\n
\nUsing Games as Psychological Research Tools<\/strong><\/p>\n
Let me give an example from my own work in collaboration with a social psychologist colleague, Dr. Andreas Kastenmueller. \u00a0Andreas and I are interested in a psychological construct called self-activation, i.e. to what extent can the representation of self be influenced by the appearance of avatars during game play. \u00a0We did a study last year (currently unpublished) where we had 4 groups of players play Wii Sports: two groups played an aggressive sport (boxing) and the other two groups played a less aggressive sport (bowling). \u00a0Within each group of boxer and bowlers, half of the players played with an aggressive avatar (Wii avatar with furrowed brow – “angry eyes”) and half with a neutral avatar. \u00a0The only difference between the gaming experience was the facial expression of the avatar, everything else was held constant. \u00a0We ran around 80 people through the design (20 per group, 10 males\/10 females; all 4 groups had approximately equivalent mean ages) and matched gender of the avatar to the gender of the person. \u00a0We recorded heart rate, respiration rate and blood pressure as well as acceleration in three axes. \u00a0Briefly, we found all physiological indicators were significantly higher during the boxing compared to the bowling but no effect for the expression of the avatar. \u00a0However, when we controlled our analysis of physiological data for movement (using acceleration data), the effect of the game type disappeared and the effect of avatar reached statistical significance, i.e. blood pressure increased when people played with the angry avatar. \u00a0In other words, identification with an aggressive avatar increased autonomic activation during the game – why were our participants more physiologically activated when playing with angry avatar? \u00a0In truth, we don’t know – it could be that the aggressive avatar augmented natural competitiveness or that it was more emotionally arousing to play with an expressive avatar or that they produced more testosterone when the avatar was angry. \u00a0This kind of research is exploratory, and as ever, we need another experiment…<\/p>\n
The reason I present this summary is to make a general point; physiological variables are very sensitive measures. \u00a0They respond to subtle psychological variables (appearance of avatar), \u00a0individual differences (between participants), the game context (boxing vs. bowling) and physical activity (movement). \u00a0This sensitivity of physiological variables is large part of why they are useful for psychological research, but this sensitivity is a double-edged sword. \u00a0When your measures can be sensitive to so many things, you need careful experimental control if you want to interpret your data in a way that is robust and unambiguous, which are the qualities that will make your results meaningful to others.<\/p>\n
Evaluation of Player Experience<\/strong><\/p>\n The second context for psychophysiology and gaming is the evaluation of player experience as part of a design cycle. \u00a0The goal here is to inform the process of game design in order to produce better games, and to be more specific, to confirm via play testing that the experience of the gamer conforms with the intentions of the team who were responsible for designing the game. \u00a0Game designers may construct a gaming experience with the objective of inducing different cognitive\/motivational\/emotional experiences and psychophysiology represents one means of confirming that these experiences have been achieved. \u00a0This type of testing can take place at the macro level (was my game scary?) or the micro level (was the part where the nice old lady turned into a zombie brandishing a chainsaw scary?).<\/p>\n This type of testing generally takes place in a commercial context (though there are research projects dedicated to this topic such as FUGA<\/a>). \u00a0Software companies generally indulge in play testing using observational methods in conjunction with post-game interviews, but there are exceptions to this rule such as the TRUE<\/a> method developed by Microsoft Games Studios. \u00a0A number of people working in this field have put forward strong arguments for using psychophysiological methods to evaluate gamer experience – see the talks from Pejman Mirza-Babaei\u00a0and Lennart Nacke from our recent CHI workshop here<\/a> as examples. \u00a0Lennart was also the co-author of a recent methodological paper<\/a> on how physiology could be combined with video and game events to triangulate player experience (i.e. to converge parallel data streams from physiology, observation and game events to understand player experience). \u00a0Vertical Slice<\/a> employ a similar approach known as Biometric Storyboarding (see the CHI link for Pejman’s presentation on this topic).<\/p>\n Using physiology in the context of commercial usability testing is very different to using game software in order to explore psychological hypotheses as we did with the Wii study. \u00a0First of all, the purpose is to gain insight into player experience and then to convey that insight to game designers in a way which informs their practice. \u00a0There are several challenges to be overcome in that last sentence. \u00a0Also, we ought to recognise that this research may be confirmatory rather than exploratory – in other words, it should lead to a clear conclusion about the game experience, not just another experiment. \u00a0In the past\u00a0I worked at a human factors research institute that included both a research branch and a commercial\/consultancy arm, I worked on projects in both sectors and experienced first-hand how the evaluation of technology diverged across academic and commercial sectors. \u00a0A large part of this difference was the need for clarity (how will your test help me sell more products – exactly) and expediency (I need your results now) which combined with financial restrictions to severely limit the number of participants that could be tested or the amount of time that may be devoted to analysis.<\/p>\n However, none of these limiting factors change anything I said earlier about the use of physiology (or biometrics as the game industry seem to call it) to evaluate user experience. \u00a0These data remain volatile, variable and difficult to interpret without a high level of experimental control.<\/p>\n Recently the Escapist published this article<\/a> based on work by Vertical Slice on which is the scariest game for the Xbox 360. \u00a0For me, there was a number of problems with how physiology was used in the context of this study (at least going from the description in the aforementioned article) that encapsulate the problems of player evaluation with commercial products. \u00a0First of all,<\/p>\n “the study, performed across four games (Alan Wake, Dead Space 2, Condemned: Criminal Origins<\/em> and\u00a0Resident Evil 5<\/em>) on six participants between the ages of 20 and 42, attempted to discover exactly which moments of the games were frightening.”<\/p>\n So we have four games to be compared, which are similar in theme but also different in terms of mechanics – this is one big problem for evaluation using bespoke commercial products, the lack of systematic control across different software titles. \u00a0The alternative is to compare different versions of the game world that may be constructed from scratch or to use a SDK to create a systematic variation in the game world. Also the number of participants in this study is very low, especially as the group of six players varied considerably with respect to age and were further divided into casual vs. core gamers. \u00a0What this does is simply increase the level of unsystematic “noise” in the data, although achieving statistical significance is perhaps not a pressing issue for this kind of commercial work, it does make it very hard to find consistency in the data at all, except by considering data from one individual. \u00a0A description of the data collection is presented below from the original article:<\/p>\n “To measure fear, Vertical Slice had each of the participants play about 30 minutes of each game in a counter-balanced order to reduce bias. During play, the participants were asked to think out loud; at the same time, their heart rate, skin surface temperature, Galvanic Skin Response (which measures excitement or frustration) and, in some cases, respiration were measured. After playing, the volunteers were asked to analyze their experience.”<\/p>\n It should be noted that the researchers counterbalanced the presentation of the games to control for order effects. \u00a0I was very surprised to hear that speak-aloud protocol had been used. \u00a0Speaking exerts a profound effect on breathing rate and heart rate – since both breathing and heart rate are physiologically linked. \u00a0In my view, these data would be effectively useless as some people will talk a lot and some will speak very little. \u00a0In that case, I assume that GSR was mainly used for the high\/low scare scale that appears in the figures accompanying the article. Now here I have a major problem due to what I perceive to be a gross simplification of the psycho-physiological inference. \u00a0As noted in the article, GSR is associated with excitement or frustration; according to this<\/a> recent review article on autonomic markers of emotion, GSR has been associated with a range of high activation emotions, including excitement, happiness, anger, frustration and fear. \u00a0GSR basically measures activation of sympathetic nervous system, it is inherently ambiguous with respect to emotional labels (i.e. a one-to-many relationship, see my 2009 paper<\/a> for full description of the complexity of this inference).<\/p>\n This is why experimental control is so important because context is everything for the interpretation of physiological data.<\/p>\n Why Experimental Control is Important for Everyone<\/strong><\/p>\n I don’t wish to bag the guys at Vertical Slice, I think the storyboarding technique they have developed is really interesting and I am reading about their work from a secondary source. \u00a0But I do want to raise a debate about how physiological measures should be used in a commercial game testing situation. \u00a0When I worked in the commercial realm, I was sometimes told by the old hands that a high level of experimental control was unnecessary in the realm of consultancy because this was “quick and dirty” testing. \u00a0I always hated that phrase because the financial stakes are very high in commercial work and researchers need to be as cautious and meticulous as they would be in a pure academic research setting (in fact, more so). \u00a0On the other hand, is it realistic to expect a software company to allow me to test 80+ participants (at a consultancy rate) in order to figure out what kind of avatar they should use in their game? \u00a0 And then to charge them again for a follow-up experiment? \u00a0The applied research on game testing methodology (such as the FUGA project mentioned earlier) charts a middle path between these extremes of academic exploration and commercial confirmation, but no researchers who is using physiological measures can hope to dodge the requirement for real experimental control if they wish to present a confident interpretation of physiological data to their clients. \u00a0The use of scientific tools is not the same as employing a scientific methodology and gross simplification of what physiological measures mean will eventually damage the credibility of this approach in the domain of player evaluation – because people will figure out that interpretation is ambiguous and the lines on the chart are open to multiple interpretation.<\/p>\n What Kind of Evaluation Do Game Designers Want?<\/strong><\/p>\n In the interest of transparency, and to prevent an accusation of criticising Vertical Slice from an academic ivory tower, I’d like to present some “quick and dirty” testing of our own in order to end this post on yet another dilemma and to show that we’re guilty of the same errors. \u00a0Below you can see heart rate data for a single player – this person played the Sony PlayStation game “WipeOut HD” (a futuristic racing game for those unfamiliar with the title) under three conditions of difficulty (defined as increased speed of opponents); venom is the easiest level, phantom is the hardest level. \u00a0We logged heart rate during each time by their position in the race, reasoning that sympathetic activation would be highest when the task was hard and the player is in the highest position in the race.<\/p>\n
![\"\"](\"http:\/\/www.physiologicalcomputing.net\/wordpress\/wp-content\/uploads\/2011\/06\/Player-1-Position-HR-Bin1.jpg\" "\\"Player")
<\/a><\/p>\n