Cognitive and Affective Probing for Neuroergonomics

With increasingly portable and user-friendly electroencephalography (EEG) systems, this brain monitoring technique continues its path towards commercial applications (Mullen et al., 2015; Zander et al., 2017). Research into passive brain-computer interfacing (passive BCI, pBCI; Zander & Kothe, 2011) has focused on EEG to enable human-computer interaction (HCI) to make use of implicit information—information that is not explicitly communicated by a human operator, but is instead be obtained from brain activity (Zander, Brönstrup, Lorenz, & Krol, 2014). For example, a cognitive workload index can be established based on reference recordings of an operator’s brain activity (Gevins & Smith, 2003). This can then be used in the form of a pBCI to automatically adapt automation levels (Parasuraman, Mouloua, & Hilburn, 1999). pBCI-based adaptive automation is an example of a closed-loop system (Krol, Andreessen, & Zander, 2018). Access to an operator’s brain state, however, potentially allows a computer system more flexibility than that. Through cognitive probing, a system can be given the ability to autonomously learn specific pieces of information, independent of pre-defined loops. This paper is part of an effort to discuss the wider implications of cognitive probing in a number of different disciplines. It overlaps with and continues a paper presented earlier (Krol & Zander, in press).


Introduction
With increasingly portable and user-friendly electroencephalography (EEG) systems, this brain monitoring technique continues its path towards commercial applications (Mullen et al., 2015;Zander et al., 2017).Research into passive brain-computer interfacing (passive BCI, pBCI; Zander & Kothe, 2011) has focused on EEG to enable human-computer interaction (HCI) to make use of implicit information-information that is not explicitly communicated by a human operator, but is instead be obtained from brain activity (Zander, Brönstrup, Lorenz, & Krol, 2014).For example, a cognitive workload index can be established based on reference recordings of an operator's brain activity (Gevins & Smith, 2003).This can then be used in the form of a pBCI to automatically adapt automation levels (Parasuraman, Mouloua, & Hilburn, 1999).
pBCI-based adaptive automation is an example of a closed-loop system (Krol, Andreessen, & Zander, 2018).Access to an operator's brain state, however, potentially allows a computer system more flexibility than that.Through cognitive probing, a system can be given the ability to autonomously learn specific pieces of information, independent of pre-defined loops.
This paper is part of an effort to discuss the wider implications of cognitive probing in a number of different disciplines.It overlaps with and continues a paper presented earlier (Krol & Zander, in press).

Cognitive Probing
We proposed earlier (Krol & Zander, in press) to define cognitive probing as utilising cognitive probes.
A cognitive probe is a single autogenous system adaptation that is initiated or co-opted by that system in order to learn from the user's contextual, cognitive brain response to it.
A 'system adaptation' here refers to anything the computer does, be it stimulus presentation, feedback, response to input, etc. Cognitive probing involves measuring the operator's responses to computer actions and contextual factors in such a way that a causal effect can be determined.Importantly, cognitive probes are computer actions that are autonomously controlled by the computer for the purpose of learning such effects.When it is learned what actions cause what cognitive responses, the computer can act purposefully to support the operator.
The proposed definition encompasses a number of different methods already in use.For example, a secondary oddball task to infer workload (Kohlmorgen et al., 2007) is a form of cognitive probing: the computer purposefully plays sounds in order to learn from the operator's response to those sounds.
The framework highlights a number of aspects to cognitive probing.One aspect concerns the probe's intrusiveness.Whereas an oddball paradigm must necessarily interfere with the operator's tasks, probes can also be embedded unobtrusively in HCI.
For example, a computer may learn that whenever certain 'naturally' (Krol & Zander, 2017) occurring notifications pop up, workload levels increase.It could then suppress such notifications when workload levels are already relatively high.
Of course, this particular rule has been suggested before (Kirchner et al., 2016).Cognitive probing however would grant a computer the autonomy to learn such a rule by itself-or, indeed, any other rule.Cognitive probing enables the system to learn each operator's individual cognitive properties based on their responses to any number of probes.

Affective Probing
The proposed definition focuses on cognition, but can easily be extended to include affect.For example, Zander, Krol, Birbaumer, and Gramann (2016) gave a speculative example of a neuroadaptive book.In this book, the events that unfold as part of the written story are co-opted as probes: the system monitors the readers' brain activity, and registers their implicit responses to specific passages.It may thus learn, for example, which character is the reader's favourite.Based on this information, it can adapt itself by re-writing subsequent passages to manipulate the reader's experience of the book.
Affect (mood, emotion) influences one's experience and performance in the workplace (Brief & Weiss, 2002).For example, many potential sources of anger have been identified in the workplace (Schieman, 2010), which, along with many other negative emotions, often lead to counterproductive work behaviour (Spector & Fox, 2005).
Thus, HCI may benefit when the computer autonomously can probe, interpret, and adapt to the operator's affective state.Systems capable of both cognitive and affective probing would be able to construct a more complete model of the operator's preferences.

Discussion
Probes can be 'hidden', and the operators may not be capable of suppressing their responses to them.This makes cognitive probing a highly privacy-sensitive issue.We have argued earlier that strict ethical guidelines must be adhered to, that data ownership must remain with the operator, and that full disclosure and informed consent are necessary (Krol & Zander, in press).
Ethical issues may be even more sensitive in the workplace.Managers can prescribe what hard-and software must be used.Should they be allowed to prescribe systems that utilise cognitive probes, and build models of their users' needs, preferences, and intentions?At the very least, we argue that employees must remain in control of who has access to the information derived from probes.This includes momentary information such as responses to single probes, as well as general models derived form larger numbers of probes.We believe strong regulation must be in place to prevent managers from obtaining unwanted access to these cognitive profiles before any cognitive probes are used in professional settings.
Affective probing means that computers must be able to influence their operator's emotions.In private settings, such manipulation may be desirable-horror movies, for example, are a clear example of adults deliberately choosing to experience negative affect.Similarly, neuroadaptive emotional manipulation may be desirable in some contexts.However, aside from the earliermentioned ethical considerations, we at least suggest the need for additional mechanisms to avoid positive feedback loops leading to extreme emotions.Looking specifically at the workplace, furthermore, we believe an open debate must be held concerning whether or not neuroadaptive emotional manipulation is desirable at all, even when the above-mentioned requirements are met.
Cognitive and affective probing may lead to uniquely personalised, cooperative systems, and thus increased productivity and satisfaction.However, there are a number of ethical issues that must be considered now-before the first such systems are developed.Where people may largely decide for themselves in their private lives, a neuroadaptive workplace requires additional considerations ahead of time.