Well there goes by another year I don’t get to spend the holidays playing with the Wii Vitality. I’m beginning to think Nintendo have given up on the whole idea of biofeedback adapted gaming given the lack of noise they’ve made since the platforms original announcement in June 2009. I originally bought my Wii on the premise that Nintendo was going to start the ball rolling on integrating biofeedback interfaces into mainstream games, and to that effect have speculated several times on the type of experiences we might see and how they would work (e.g. action games, lie-detection, relaxation and fitness). However it looks like this device will remain vaporware for the foreseeable future. That’s not to say biofeedback gaming failed to make an entrance in 2010, at E3 2010 Ubisoft demoed Innergy for the PC, a biofeedback relaxation game similar to what the Wii Vitality would of initially been released with, and EA released (and I’m currently playing) Sports Active 2 for consoles, an exercise game which measures (and displays) heart activity to assess the player’s physical performance during set exercises. 2010 also saw a strong portfolio of biofeedback experiences (especially EEG) developed using a range of low-cost sensors which I can only imagine will improve in the following year (e.g. Mattel announced a competitive version of their Mindflex system just recently at CES). So what can we expect from the field of Physiological Computing in 2011.
Kiel’s Predictions for 2011
1. EA Sports Active supports biofeedback adaptive workout packs, and third party games
EA is clearly onto a winner with the sensor they chose to bundle Sports Active 2 with. The device is an arm based pulse oximeter and strikes the perfect balance between measurement accuracy under movement and the player’s level of comfort with the sensor. As previously demonstrated a finger based pulse oximeter is not suitable for gaming experiences that involve significant movement as it induces signal artifacts. In informal tests EA’s sensor compares favourably well with the chest-strap I wear but without being as invasive which would of easily become a marketing problem if they had decided to use one.
Currently the device only supports EA Sports Active 2 and Sports Active NFL Training Camp. Both games (as far as I know) only support biofeedback in the form of a numeric display. The player is shown a visual of their current heartbeat rate and the level of effort they are exerting which is represented as a series of training zones. Using this information the player can vary the level of effort they put into their exercise regime. This is a very tame form of biofeedback, and as such I expect EA to release a workout pack that experiments with biofeedback adapted exercises as they already have a supporting platform and the cost/risk of including a few experimental exercises is minimal. For example, they could make the glut of running exercises more interesting by changing the colour of the gaming environment as the player increases their exertion level, or perhaps they could play the Rocky theme during the boxing exercise when the player enters the best training zone.
As the game needs to motivate players to sustain set levels of exertion, by attaching biofeedback enabled rewards to the desired exertion level the player can be better motivated. For a more comprehensive exercise game I recommend reading “A Fitness Game Reflecting Heart Rate” which describes a boxing game which modifies itself to encourage the player to interact at specified levels of exertion.
As a multi-platform device EA’s technology is ideally placed to be licensed to 3rd parties interested in providing novel biofeedback experiences. While I don’t expect triple A games to be licensed, the app stores on each console would seem appropriate.
2. Psychophysiological testing in game design used as a selling point
If you played last years Silent Hill: Shattered Memories on the Wii, you would of seen the benefit in personalising game experiences to the psychological make up of the player (e.g. when visiting your character’s home it’s coloured according to a previous drawing you made). Interestingly enough Silent Hill encapsulated this calibration process inside the game experience which I believe is key for psychophysiological adaptive game experiences.
I don’t believe were at the point we’ll see any commercial psychophysiological adaptive games (e.g. adapts to player emotions) in the near future. However I do expect videogames to invest more in psychophysiological studies in order to craft the player experience. As the sensor technology becomes cheaper and the academic literature in this area becomes more applied its easier for game companies to pick-up this research and try it out. The benefit of psychophysiological testing is it provides objective measures of various psychological states (e.g. engagement) over the period of a gaming session which can be used for example, to identify which level of fog in a horror game struck the right balance between unease and playability.
3. Extendable data repository web service for self-quantifiers
Last year saw huge gains in the self-quantification movement. There are numerous web services and devices that allow people to collate information about themselves, analyse it and use the result as a motivational tool (e.g. the Direct Life pedometer by Philips, sends motivational e-mails to you based on your level of recorded activity). The problem I see here is that the user’s data is tied to a specific web service and/or device. In my experience, you rarely find a platform that provides the ideal combination of software and hardware for your needs. Therefore having your personal data tied to a specific platform becomes problematic as your now dependant on a single company.
As such I expect there will business opportunities in providing a middleware platform for a user’s self-quantification data. Such a platform would allow the user to upload any number of feeds to their database. Data analysis would be provided by other companies who create a suitable web interface to run on top of the middleware platform for which they pay the provider access for. Subsequently the user can shop around for the type of analysis they want without having their data constrained to a single compnay.
4. Marginal increase in real-time body bloggers
While it has been fun this past year to blog my heart rate in real-time, I don’t see any trends emerging from this beyond a marginal pick-up from a few self-quantifiers, academic researchers and the medical domain. While body blogging in general has seen a significant increase (e.g. EA Sports Active 2’s web interface is a body blogging platform), real-time body blogging has yet to find its niche.
5. Emergence of inituitive physiological interfaces
If you’ve ever played with a neurofeedback toy like the Force Trainer, there’s always that element of doubt that accompanies any game, are you moving the ball thanks to your mastery of your own thoughts or you moving it with that strained face you make every time you stare at the ball. A variety of biofeedback interactions rely on the user’s belief that the mechanism they’ve been told to use to manipulate the physiological signal is actually what’s going on.However this can be problematic, for instance EEG measurments are particularly fraught with problems caused by other physiological events related to the manipulation method creating false positives (e.g. when attempting to concentrate the muscles in the forehead activate causing signal noise which can be interpreted as the desired mental activity). As such when the user figures out the disconnect between their implied actions and the manipulation it can diminish the experience. You can easily see this effect in motion games, for example, in Wii Boxing an inexperienced player may perform a physical punch to trigger an in-game one however an experienced player is likely to figured out that the required range of body motion needed to pull off the move is a small flick of the controller.
To get around this disconnect, there is an emerging trend to study which manipulation techniques provide the most meaningful relationship between the user’s implied action and a physiological change. This trend can only increase in 2011.
Steve’s Predictions for 2011
1. Development of BCI technology for healthy users
I wrote a couple of posts about the EPOC system last year, which made headlines wherever it was demonstrated. It will be really interesting to see how this system evolves over the coming year, particularly in terms of take-up from development community that the company are trying to facilitate. My fear is that the range of response from EPOC doesn’t have sufficient range to create more than a small number of reliable commands, which would make a niche product with novelty value – but I hope I’m wrong and the development community can deliver some applications that make more people want to buy the system and experiment with this kind of technology.
2. Greater emphasis on hybrid BCI
This may be an academic trend but I see two ways in which BCI technology may progress at the research and development phase. We’ve already seen examples of hybrid systems where several different physiological measures may be combined to increase the information transfer rate for the system. There is great potential in this approach but personally I would like to see more academic work on ‘mash-up’ technology where BCI and affective computing/biocybernetic adaptation concepts are combined in one system.
3. Multi-modal representation of the user
This blog is concerned with the use of physiological data as an input to a computing system but this should not imply that the user can only be represented by physiology. The possibilities of multi-modal representations of user state that combine physiology with posture, facial expression and vocal affect remain relatively unexplored. Similarly, the linkage of real-time performance data (e.g. task progress) with physiology has not been exploited in any significant way. Unfortunately academic communities with knowledge on each specific technique tend be split across different disciplines and this is a real barrier to the development of this theme. Hopefully we’ll see more attempts from researchers to develop multimodal combinations to represent the user. This is more of a hope than a prediction.
4. Personal informatics, mood monitoring and health-monitoring
As Kiel’s pointed out above, I also see a rising trend of interest in self-monitoring and self-experimentation as exemplified by the international Quantified Self community. We are beginning to see apps for mobile devices like iphone that are capable of monitoring heart rate activity and storing the data. Interest in these kinds of apps should lead to the development of low-cost, comfortable and discrete sensors. I don’t see body-blogging spreading beyond a small group of hobbyists until these sensors are available to everyone. The current QS community is dominated by self-experimentation using measures that are readily available and easy to capture (e.g. weight, sleep/wake times). I do think we’ll see more examples of real-time physiological monitoring in the home or office for people being treated as out-patients – which will be an ice-breaker for this trend.
5. Psychology research using Ambulatory apparatus
Once upon a time if a psychology researcher wanted to measure physiology, the person had to come into a lab. Not a problem for some, but applied psychologist who want to study how emotions and stress affect the brain and body, have to create artificial scenarios to capture these phenomena. The improved quality and reduction of cost for ambulatory apparatus means that more applied psychophysiology research can take place in the field using realistic scenarios. I’ve noticed more examples of good research in the literature in the last year using ambulatory measurement and predict that this trend will continue. In my opinion, this is important for the development of physiological computing for one simple reason – most of our knowledge of these signals has been gained in the laboratory and more research in the field will improve our understanding of how these signal work in the real world; this in turn will lead to the development of more sophisticated data treatment algorithms that can co-opted into the design of real systems.
So that’s our predictions for 2011, do you think we’re on the money or do you have your own predictions for 2011. Hit the comments below and give us a shout.