{"id":136,"date":"2009-06-10T15:28:34","date_gmt":"2009-06-10T15:28:34","guid":{"rendered":"http:\/\/www.physiologicalcomputing.net\/wordpress\/?p=136"},"modified":"2021-12-22T20:22:10","modified_gmt":"2021-12-22T20:22:10","slug":"overt-vs-covert-expression","status":"publish","type":"post","link":"https:\/\/www.physiologicalcomputing.net\/?p=136","title":{"rendered":"Overt vs. Covert Expression"},"content":{"rendered":"

This article<\/a> in New Scientist on Project Natal got me thinking about the pros and cons of monitoring overt expression via sophisticated cameras and covert expression of psychological states via psychophysiology.\u00a0 The great thing about the depth-sensing cameras (summarised nicely by one commentator in the article as like having a Wii attached to each foot, hand and your hand) is that: (1) it’s wireless technology, (2) interactions are naturalistic, and (3) it’s potentially robust (provided nobody else walks into the camera view).\u00a0 Also, because it captures overt expression of body position\/posture or changes in facial expression\/voice tone (the second being muted as a phase two development), it measuring those signs and signals that people are usually happy to share their fellow humans – so the feel of the interaction should be as naturalistic as a regular discourse.<\/p>\n

So why bother monitoring psychophysiology in real time to represent the user?\u00a0 Let’s face it – there are big question marks over its reliability, it’s largely unproven in the field and normally involves attaching wires to the person – even if they are wearable.<\/p>\n

But to view a\u00a0 face-off between the two approaches in terms of sensor technology is missing the point.\u00a0 The purpose of depth cameras is to give computer technology a set of eyes and ears to perceive & respond to overt visual or vocal cues from the user.\u00a0 Whilst psychophysiological methods have been developed to capture covert changes that remain invisible to the eye.\u00a0 For example, a camera system may detect a frown in response to an annoying email whereas a facial EMG recording will often detect increased activity from the corrugator or frontalis (i.e. the frown muscles) regardless of any change on the person’s face.<\/p>\n

One approach is geared up to the detection of visible cues whereas the physiological computing approach is concerned with invisible changes in brain activity, muscle tension and autonomic activity.\u00a0 That last sentence makes the physiological approach sound superior, doesn’t it?\u00a0 But the truth is that both approaches do different things, and the question of which one is best depends largely on what kind of system you’re trying to build.\u00a0 For example, if I’m building an application to detect high levels of frustration in response to shoot-em-up gameplay, perhaps overt behavioural cues (facial expression, vocal changes, postural changes) will detect that extreme state.\u00a0 On the other hand, if my system needed to resolve low vs. medium vs. high vs. critical levels of frustration, I’d have more confidence in psychophysiological measures to provide the necessary level of fidelity.<\/p>\n

Of course both approaches aren’t mutually exclusive and it’s easy to imagine naturalistic input control going hand-in-hand with real-time system adaptation based on psychophysiological measures.<\/p>\n

But that’s the next step – Project Natal and similar systems will allow us to interact using naturalistic gestures, and to an extent, to construct a representation of user state based on overt behavioural cues.\u00a0 In hindsight, it’s logical (sort of) that we begin on this road by extending the awareness of a computer system in a way that mimics our own perceptual apparatus.\u00a0 If we supplement that technology by granting the system access to subtle, covert changes in physiology, who knows what technical possibilities will open up?<\/p>\n","protected":false},"excerpt":{"rendered":"

This article in New Scientist on Project Natal got me thinking about the pros and cons of monitoring overt expression via sophisticated cameras and covert expression of psychological states via psychophysiology.\u00a0 The great thing about the depth-sensing cameras (summarised nicely by one commentator in the article as like having a Wii attached to each foot, […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":""},"categories":[5,6],"tags":[8,9,28,61,65],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/pY315-2c","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=\/wp\/v2\/posts\/136"}],"collection":[{"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=136"}],"version-history":[{"count":1,"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=\/wp\/v2\/posts\/136\/revisions"}],"predecessor-version":[{"id":4752,"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=\/wp\/v2\/posts\/136\/revisions\/4752"}],"wp:attachment":[{"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=136"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=136"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.physiologicalcomputing.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=136"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}