In April there was a rumour going around that the next Red Steel (the third in the series) might support the Wii Vitality. The gameplay in Red Steel is a mixture of first person shooting and first person sword fighting. In the last Red Steel the combat system felt very similar to that of two-player fighting games like Street Fighter as apart from the basic sword fighting techniques you can perform with the Wii controller (e.g. blocking and striking) you could also pull off a range of special moves with different combinations of gestures and key presses. I’m a big fan of the Red Steel franchise and I believe it would be an interesting series to explore biofeedback based gameplay mechanics as both the mythos and the physical skillset being simulated in Red Steel lends itself well to intrinsically interesting physiological manipulations (e.g. as your playing a swordsman, “zen” powers aren’t too much of stretch for your suspension of disbelief). Below I’ve made a couple of suggestions as to what biofeedback based gameplay mechanics you might find in the next Red Steel game if it uses the Wii Vitality: –

1. Relax-to-Heal: The player’s level of relaxation controls their character’s healing rate.

If your in battle and your health bar is dangerously low in order to avoid death your going to have break off from combat and hide while you wait for the your character’s health to return. This healing period is not particularly interesting as you’ve had to disengage from the current challenge. To make play more interesting during this time we can make health regeneration dependant on the player’s control of their state of relaxation (i.e. a new type of challenge). So the better the player is at relaxing the faster they can regenerate their health and go back into combat. If the enemy is particularly viscous in pursuing the player or the player has to perform time sensitive tasks this mechanic will add a new dimension to play as they have manage the effort they put into a physiological challenge in order to successfully participate in the games cognitive ones.

2. Focus-Sword: The player relaxes in a combat scenario in order to charge their sword to unleash special attacks.

In competitive biofeedback games (e.g. Simmer Down Sprinter) players are rewarded when they relax. It can be difficult to relax in a competitive environment as the body’s natural instinct is to become more aroused. The same principle applies in single player combat situations, the more stressful the situation the more more aroused you become.  As such we reward the player for relaxing in a hostile environment with new abilities. In Red Steel 2 the player can unleash certain special attacks by charging their sword for a few seconds. In the context of the Red Steel 2 mythos we can add a “zen” bar for the sword which the player can only charge by relaxing during combat. Once the bar is full they can unleash a special attack. As the player must risk diverting their attention to controlling their physiological state while engaging in a cognitive task this physiological challenge has its own inherent risk reward system to entice the player to attempt charging the sword.

Problems with Motion Artifacts

The problem in realising a physiological game with the Vitality is in the sensitivity of the sensor to movement artifacts. As I mentioned in my pre-E3 post I was concerned that Nintendo’s choice in sensor might impact the game experiences we can expect the device to support. A pulse-oximeter uses reflected light to track changes in blood volume; if the beam moves the relative changes in blood volume we’re interested in become corrupt by the change in the sensor ‘s position thereby introducing errors in any derived measures (e.g. heartbeat rate). So the question is, how sensitive is a finger based pulse-oximeter to movement?

Using a TSD123A pulse-oximeter I had a look at what affect finger movement might have on the pulse signal. Figure 1 shows what type of waveform the sensor should detect under normal conditions (i.e. index finger extended, no motion). The waveform shows a periodic increase and decrease in blood volume as the heart beats. Figure 2 shows what happens to this waveform as I move my finger from side to side and Figure 3 shows what happens when I repeatedly extend and curl my finger (both motions performed midway in the time series repeating every second). In both cases the waveform deforms quite severely. Heartbeat rate is derived from the time difference between each peak in the pulse signal and in both cases as I move my finger my heartbeat rate appears to increase by tens of beats which would normally imply I’m either walking or slowly running.

Figure 1 - Pulse wave of index finger at rest.

Figure 2 - Pulse wave of index finger as moved side to side.

Figure 3 - Pulse waveform of index finger as extended out and curled to palm.

As Nintendo has implied their aiming the Wii Vitality at the relaxation market I imagine there will be little expectation of body motion during play and so need to watch for motion artifacts. The artifacts I’ve just demonstrated are also pretty extreme and unlikely to occur during normal play unless the player has to move their sensor finger (e.g. for key presses). If the finger does move, singular blips will be averaged out when the heartbeat rate is derived and usually relaxation games are not set to be very sensitive to changes in the player’s physiology so noise isn’t too much of a concern. I also would expect a relaxation game to tell the player to remain relatively still during play further minimising movement artifacts. The problem comes when the game does require gesture input as with the Red Steel series. In such games there can no expectation by the developer that the player will keep their finger still during play. And when performing more extreme gestures (e.g. pretending to swing a sword around) the arm the finger is attached to is going to take the sensor with it.

Using a BioHarness chest-strap to compare signal quality I decided to have a look at what motion artifacts (if any) a finger based pulse-oximeter might suffer from while performing common Wii actions (e.g. menu manipulation, controller waggle, bowling, tennis, boxing). I’ve selected to use a chest-strap based heart monitor (obtains an ECG signal) as my control as even extreme physical activities (e.g. running, jumping, flailing your arms about) doesn’t dislodge the sensor being as it is located around the chest which remains pretty still (motion wise) during these activities. Its reliability under motion is why you see this type of sensor used in sport watches like Polar. The BioHarness also comes equipped with a series of accelerometers which measure body movement (see acceleration channel in Figures 4-7). Wearing the TSD123A on my left hand’s index finger and the BioHarness across my chest I performed the following actions using a Wii controller with my right hand (i.e. no nunchuk). Each task was performed sitting down as well as standing up.

• Wii menu navigation
• Bowling
• Tennis swing
• Boxing (both hands)

To my surprise the bowling and tennis actions (while sitting down) didn’t deform the pulse signal too much. As shown in Figure 4 and 5 the pulse waveform is still usable as my body’s natural movement during these actions didn’t move my right arm enough to dislodge the sensor. However when performing the tennis action standing up, as shown in Figure 6, the pulse waveform suffers a lot more noise and in parts it becomes too difficult to derive the interval between heartbeats. The same goes for the boxing action as shown in Figure 7.

Figure 4 - ECG, pulse and acceleration waveforms while bowling sitting down.

Figure 5 - ECG, pulse, acceleration waveforms while swinging arm sitting down.

Figure 6 - ECG, pulse, acceleration waveforms while swinging arm standing up.

Figure 7 - ECG, pulse, acceleration waveforms while boxing sitting down.

What does this mean for Red Steel 3 and other Vitality games?

Based on the performance of the pulse-oximeter during select Wii actions its unlikely the Wii Vitality will support games which require sustained  periods of exaggerated motion control and/or nunchuk gestures, and sadly Red Steel is one of those games. If you’ve ever played Red Steel, especially Red Steel 2, the game is a borderline exercise game as the player is required to perform all sorts of exaggerated motions. While these periods of activity only occur for short bursts of time (e.g. enemy encounters) the crux of the interesting biofeedback based gameplay mechanics we could implement will need to occur during these moments (e.g. like the zen sword).  If a bio-mechanic doesn’t operate during these combat sessions (i.e. core gameplay) then I would argue the entire concept of physiological input becomes less interesting for the player*. So unless Red Steel 3 undergoes a radical redesign I don’t imagine we’ll be seeing a Wii Vitality version anytime soon though that doesn’t stop the developers from using EA’s heart monitor which I expect to be more noise resistant being an ECG armband.

In conclusion

In the end the game experiences the Wii Vitality will support are largely going to depend on the following two issues 1) the gestures the developer wants to support (as seen above) and 2) the sensitivity their bio-mechanic needs to be to the player’s physiology. The less responsive a game is the more likely any motion artifacts will be averaged out.

* The biofeedback based gameplay mechanics I’ve talked about in this post are explicit adaptations (e.g. the player is aware their exerting physiological control).  Implicit adaptations where the player isn’t consciously aware of the game change (e.g. arousal controlled AI) occur over longer periods of time and so noise becomes less of problem (e.g. in Red Steel signal becomes reliable after combat and will pick up traces of the previous activity). I have not considered implicit adaptations for Red Steel 3 because if I was trying to sell a gamer on wearing an overt sensor like a pulse-oximeter I’d want them to be able to actively do something with their physiology as its more interesting for what will they’ll probably approach as a gimmick. Implicit adaptations you’d sell the player on once the idea of physiological play has been accepted.