Category Archives: Musings

Thoughts, opinions about physiological computing applications both real and imaginary

BioS-Play 2010 Workshop Experience Report – Part 1 of 2

Admin: Workshop papers can be found here.

Last month I attended the BioS-Play workshop at the Fun and Games 2010 conference over in Leuven, Belgium. I was presenting Physiology as XP – Bodyblogging to Victory, a position paper I co-wrote with Steve in which we extended the body blogging concept to computer games. In part 1 of this 2 part series of posts on BioS-Play I’ll be re-counting my experiences at the conference, as well as providing my thoughts on the likely research direction physiological games will take in the future.

EDIT

The post is rather large so I’ve made a few quick links to provide readers a taster of what’s contained within.

  • EmRoll:  A 2 player co-operative children’s game which uses a mixture of gestures and biological interactions to control Gamboo, a 2 headed monster. What the Xbox 360 Kinetic might offer in the future.
  • Study investigating the effect of sharing physiological information in collocated and networked environments on measures of presence and emotion. Following on from Steve’s Valve post, what measurable benefits might shared physiology actually bring to multiplayer games like Left for Dead.
  • Workshop discussion, covers such issues as: how do we design meaningful physiological interactions and how do we evaluate the efficacy of the user experience of a physioloigcal interface?

The Workshop Theme

BioS-Play was aimed at exploring the use of biological signals (e.g. brain waves) in both a multiplayer and social gaming environment. For full details see the workshop proposal. Over the past decade there has been an up turn in using this class of physiological input in computer games, however the majority of such systems are designed for single player experiences. This is not really surprising, although such signals have been utilised by games since the 70’s,  bio-adaptive interaction was only used in a limited therapeutic capacity. It was not until the late 90’s, a period that saw the emergence of Affective Computing,  that we saw player physiology being used in more interesting ways (e.g. see MIT Media Lab Europe projects on affective feedback).
Continue reading

Valve experimenting with physiological input for games

This recent interview with Gabe Newell of Valve caught our interest because it’s so rare that a game developer talks publicly about the potential of physiological computing to enhance the experience of gamers.  The idea of using live physiological data feeds in order to adapt computer games and enhance game play was first floated by Kiel  in these papers way back in 2003 and 2005.  Like Kiel, in my writings on this topic (Fairclough,   2007; 2008 – see publications here), I focused exclusively on two problems: (1) how to represent the state of the player, and (2) what could the software do with this representation of the player state.  In other words, how can live physiological monitoring of the player state inform real-time software adaptation?  For example, to make the game harder or to increase the music or to offer help (a set of strategies that Kiel summarised in three categories, challenge me/assist me/emote me)- but to make these adjustments in real time in order to enhance game play.

Continue reading

VIP visit at Liverpool John Moores University and issues with BioHarness

Last Tuesday the former chancellor of the university, Cherie Booth visited our place of work in the School of Psychology at Liverpool John Moores University to see the labs in action. During our demo we were running two BioHarness chest-straps and the Enobio wireless EEG while myself (red/white shirt) and a volunteer (blue/white shirt) went head to head on Wii boxing.

Photos of the rest of the tour around the department can be found at LJMU’s Facebook page.
Continue reading

Mobile Monitors and Apps for Physiological Computing

I always harbored two assumptions about the development of physiological computing systems that have only become apparent (to me at least) as technological innovation seems to contradict them.  First of all, I thought nascent forms of physiological computing systems would be developed for desktop system where the user stays in a stationary and more-or-less sedentary position, thus minimising the probability of movement artifacts.  Also, I assumed that physiological computing devices would only ever be achieved as coordinated holistic systems.  In other words, specific sensors linked to a dedicated controller that provides input to adaptive software, all designed as a seamless chain of information flow.

Continue reading

Making Data Meaningful

The problem with collecting in any amount of data is figuring out how to present that data in a manner that is meaningful for its intended audience. For example if you want to assess the physical effort you exert during a run a plot of physiological activity (e.g. heartbeat rate) against time will provide you with a relatively simple visual representation of how your body adapts to physical stress.

Continue reading

Functional vocabulary: an issue for Emotiv and Brain-Computer Interfaces

The Emotiv system is a EEG headset designed for the development of  brain-computer interfaces.  It uses 12 dry electrodes (i.e. no gel necessary), communicates wirelessly with a PC and comes with a range of development software to create applications and interfaces.  If you watch this 10min video from TEDGlobal, you get a good overview of how the system works.

First of all, I haven’t had any hands-on experience with the Emotiv headset and these observations are based upon what I’ve seen and read online.  But the talk at TED prompted a number of technical questions that I’ve been unable to satisfy in absence of working directly with the system.
Continue reading

Calm your spirits Kusagari, Red Steel 3 unlikely to use the Wii Vitality

In April there was a rumour going around that the next Red Steel (the third in the series) might support the Wii Vitality. The gameplay in Red Steel is a mixture of first person shooting and first person sword fighting. In the last Red Steel the combat system felt very similar to that of two-player fighting games like Street Fighter as apart from the basic sword fighting techniques you can perform with the Wii controller (e.g. blocking and striking) you could also pull off a range of special moves with different combinations of gestures and key presses. I’m a big fan of the Red Steel franchise and I believe it would be an interesting series to explore biofeedback based gameplay mechanics as both the mythos and the physical skillset being simulated in Red Steel lends itself well to intrinsically interesting physiological manipulations (e.g. as your playing a swordsman, “zen” powers aren’t too much of stretch for your suspension of disbelief). Below I’ve made a couple of suggestions as to what biofeedback based gameplay mechanics you might find in the next Red Steel game if it uses the Wii Vitality: –

Continue reading

Better living through affective computing

I recently read a paper by Rosalind Picard entitled “emotion research for the people, by the people.”  In this article, Prof. Picard has some fun contrasting engineering and psychological perspectives on the measurement of emotion.  Perhaps I’m being defensive but she seemed to have more fun poking fun at the psychologists than the engineers, but the central impasse that she identified goes something like this: engineers develop sensor apparatus that can deliver a whole range of objective data whilst psychologists have decades of experience with theoretical concepts related to emotion, so why haven’t people really benefited from their union through the field of affective computing.  Prof. Picard correctly identifies a reluctance on the part of the psychologists to define concepts with sufficient precision to aid the work of the engineers.  What I felt was glossed over in the paper was the other side of the problem, namely the willingness of engineers to attach emotional labels to almost any piece of psychophysiological data, usually in the context of badly-designed experiments (apologies to any engineers reading this, but I wanted to add a little balance to the debate).
Continue reading

Post E3: Without Vitality we draw upon Innergy

Well that was a disappointment. In the end Nintendo decided against demonstrating the Wii Vitality at this year’s E3. A representative of the company stated that the Vitality was a no-show because Nintendo did not believe the event was a suitable environment for the product. Disappointing but given their press event was jam packed with a number of AAA games and a new portable it was understandable. However with the Vitality aimed for a late 2010 release it doesn’t give Nintendo much time to create a buzz around a product that frankly has none. In actuality I was surprised that Nintendo didn’t use their recent endorsement deal with the American Heart Association to hype Vitality pre E3. While the product is currently being marketed towards mental health (i.e. stress management), rather than physical health which the AHA endorsement concerns, Nintendo could of easily used the event as part of a broader health platform and so make better use of the publicity the endorsement deal attracted.
Continue reading

Pre-E3: Thoughts on the Wii Vitality

With only hours left until Nintendo’s E3 press event I’ve once again been pondering what we’ll see from the Wii Vitality. At last year’s E3 the device’s announcement didn’t exactly wow the audience. It’s not surprising as Nintendo didn’t provide a demonstration of the device which might have bought gamers into the concept.  Nintendo have remained tight lipped ever since revealing absolutely nothing about what we might expect from the device.  Over the past year it has been suggested the Vitality will be used to monitor the scare factor in games, alter game difficulty, be used as an input for relaxation and exercise games as well as in lie-detection of which I discussed one particular method of implementation here.

Implementing these types of games are indeed possible using physiological measures and you can see versions of these games in the biofeedback and academic game communities for example: –

  1. The Journey to Wild Divine: series of relaxation mini-games controlled using heartbeat rate and skin conductance.
  2. A Fitness Game Reflecting Heart Rate:  boxing game which adapts the gameplay in order to move you towards a target heartbeat rate. Enemy characters require a different physical movement to destroy them, depending on the player’s current heartbeat rate and the goal an appropriate enemy will be selected.
  3. Fairies: target acquisition game which alters the player’s perception of the games difficulty according to the player’s level of arousal as denoted by heartbeat rate.

There are many ways you can use physiological data in gameplay, it’s all a question of how you make the input meaningful for the player (e.g. if the player’s relaxation level is used to switch between character states, those states have to be somewhat representative of the change, so if the player is controlling an avatar with a pyrotechnic ability then a shift from a relaxed state to an agitated state could be used to trigger their fire ability and vice versa, this would be a meaningful relationship).

The problem I have with the Vitality is in their choice of sensor:  a finger based pulse oximeter.  A pulse oximeter uses infra-red to track the changes in the volume of blood in the extremities and from this derive heartbeat rate. If you want to support a wide selection of different play styles (e.g. relaxation, exercise, affective) a finger based sensor would not have been my first choice considering the following issues: –

  1. A finger based sensor limits the player’s freedom of movement. Any physical activity will move the sensor from its recording position and may even possibly become disconnected, both events of which will create data errors. And depending on how responsive the game is to the player’s physiology it can easily lead to erroneous game behaviour.  This will limit how the data can be used in a given context (e.g. a game responsive to emotional physiological responses is not suited to a game involving gestures, this example is more prelevant for the Wii as the system sells itself on physical interaction as the standard input method).  Also the physicality of the sensor attachment to the player’s finger restricts player movement so physical actions may become uncomfortable (e.g. imagine playing Red Steel with a cable attached to your finger). This is not to say games involving physical actions will be taboo using the Vitality (e.g. a calorie counter in an exercise game), it just makes it harder.
  2. With a finger based sensor use of the second Nunchuk is liable to become awkward possibly eliminating it as an input device.

Ideally Nintendo should have gone with either an earlobe based pulse oximeter thereby freeing up the hands (though physical actions still have to be limited as that sensor is not the most secure under intense movement) or ideally a chest-strap*. A chest-strap sensor provides the most secure method of measuring a player’s heartbeat rate as the centre of the body is pretty stable under movement, this is especially true from my perspective given I’ve been wearing one for the last several months collecting data.

At this point these issues are pretty much moot (more like irritations in my noggin I can’t dispell) as I suspect Nintendo will launch the system with a series of relaxation games which the Vitality is clearly geared for**. Or perhaps a lie-detection game as I’ve talked about before.

* The problem inherent in using a chest-strap is in how the player may perceive it invading their personal space.  The chest-strap is an up-and-close personal wearable device, and I imagine given the new wireless heart monitor EA Sports Active 2.0 is using (an armband based heart monitor), there development staff thought so too. The finger attachment does not invade the player’s space so there is no unease in wearing the device.  Having just seen EA’s E3 press conference, the Vitality is already looking obsolete.

**The sensor used by The Journey to Wild Divine has been used in a multitude of different game genres, my favourite being the Half-Life 2 mod Please Don’t Feed the BioZombies. However the sensor is used in conjunction with a mouse and keyboard and this setup doesn’t suffer from player movement to the same degree the Vitality will given the nature of the input device doesn’t require much. Also unlike Vitality a mouse and keyboard is placed on a flat providing support for the hand the sensor is on.