The field of Physiological Computing consists of systems that use data from the human nervous system as control input to a technological system. Traditionally these systems have been grouped into two categories, those where physiological data is used as a form of input control and a second where spontaneous changes in physiology are used to monitor the psychological state of the user. The field of Brain-Computer Interfacing (BCI) traditionally conceives of BCIs as a controller for interfaces, a device which allows you to “act on” external devices as a form of input control. However, most BCIs do not provide a reliable and efficient means of input control and are difficult to learn and use relative to other available modes. We propose to change the conceptual use of “BCI as an actor” (input control) into “BCI as an intelligent sensor” (monitor). This shift of emphasis promotes the capacity of BCI to represent spontaneous changes in the state of the user in order to induce intelligent adaptation at the interface. BCIs can be increasingly used as intelligent sensors which “read” passive signals from the nervous system and infer user states to adapt human-computer, human-robot or human-human interaction (HCI, HRI, HHI). This perspective on BCIs challenges researchers to understand how information about the user state should support different types of interaction dynamics, from supporting the goals and needs of the user to conveying state information to other users. What adaptation to which user state constitutes opportune support? How does the feedback of the changing HCI and human-robot interaction affect brain signals? Many research challenges need to be tackled here.
Sensors, such as wireless EEG caps, that provide us with information about the brain activity are becoming available for use outside the medical domain. As in the case of physiological sensors information derived from these sensors can be used – as an information source for interpreting the user’s activity and intentions. For example, a user can use his or her brain activity to issue commands by using motor imagery. But this control-oriented interaction is unreliable and inefficient compared to other available interaction modalities. Moreover a user needs to behave as almost paralyzed (sit completely still) to generate artifact-free brain activity which can be recognized by the Brain-Computer Interface (BCI).
Of course BCI systems are improving in various ways; improved sensors, better recognition techniques, software that is more usable, natural, and context aware, hybridization with physiological sensors and other communication systems. New applications arise at the horizon and are explored, such as motor recovery and entertainment. Testing and validation with target users in home settings is becoming more common. These and other developments are making BCIs increasingly practical for conventional users (persons with severe motor disabilities) as well as non-disabled users. But despite this progress BCIs remain, as a control interface, quite limited in real world settings. BCIs are slow and unreliable, particularly over extended periods with target users. BCIs require expert assistance in many ways; a typical end user today needs help to identify, buy, setup, configure, maintain, repair and upgrade the BCI. User-centered design is underappreciated, with BCIs meeting the goals and abilities of the designer rather than user. Integration in the daily lives of people is just beginning. One of the reasons why this integration is problematic is due to view point of BCI as control device; mainly due to the origin of BCI as a control mechanism for severely physical disabled people.
In this challenge (organised within the framework of the Call for Challenges at ICMI 2012), we propose to change this view point and therefore consider BCI as an intelligent sensor, similar to a microphone or camera, which can be used in multimodal interaction. A typical example is the use of BCI in sonification of brain signals is the exposition Staalhemelcreated by Christoph de Boeck. Staalhemel is an interactive installation with 80 steel segments suspended over the visitor’s head as he walks through the space. Tiny hammers tap rhythmic patterns on the steel plates, activated by the brainwaves of the visitor who wears a portable BCI (EEG scanner). Thus, visitors are directly interacting with their surroundings, in this case a artistic installation.
The main challenges to research and develop BCIs as intelligent sensors include but are not limited to:-
- How could BCIs as intelligent sensors be integrated in multimodal HCI, HRI and HHI applications alongside other modes of input control?
- What constitutes appropriate categories of adaptation (to challenge, to help, to promote positive emotion) in response to physiological data?
- What are the added benefits of this approach with respect to user experience of HCI, HRI and HHI with respect to performance, safety and health?
- How to present the state of the user in the context of HCI or HRI (representation to a machine) compared to HHI (representation to the self or another person)?
- How to design systems that promote trust in the system and protect the privacy of the user?
- What constitutes opportune support for BCI based intelligent sensor? In other words, how can the interface adapt to the user information such that the user feels supported rather than distracted?
- What is the user experience of HCI, HRI and HHI enhanced through BCIs as intelligent sensors?
- What are the ethical, legal and societal implications of such technologies? And how can we address these issues timely?
We solicit papers, demonstrators, videos or design descriptions of possible demonstrators that address the above challenges. Demonstrators and videos should be accompanied by a paper explaining the design. Descriptions of possible demonstrators can be presented through a poster.
Accepted papers will be included in the ICMI conference proceedings, which will be published by ACM as part of their series of International Conference Proceedings. As such the ICMI proceedings will have an ISBN number assigned to it and all papers will have a unique DOI and URL assigned to them. Moreover, all accepted papers will be included in the ACM digital library.
Deadline for submission: June 15, 2012
Notification of acceptance: July 7, 2012
Final paper: August 15, 2102
- Femke Nijboer, University of Twente (Researchgroup HMI),The Netherlands
- Mannes Poel, University of Twente (Researchgroup HMI), The Netherlands
- Anton Nijholt, University of Twente (Researchgroup HMI), The Netherlands
- Egon L. van den Broek, TNO (Technical sciences), The Netherlands
- Stephen Fairclough, Liverpool John Moore University, (Research Group Physiological Computing) United Kingdom