Abstract
The development of cognitive task battery applications for rehabilitation in telemedicine is a rapidly evolving field, with several tablet or web based programs already helping those suffering from working memory dysfunction or attention deficit disorders. However, there is little physiological evidence supporting a measurably significant change in brain function from using these programs. The present study sought to provide an initial assessment using the portable and wearable neuroimaging modality of functional near-infrared spectroscopy (fNIRS) that can be used in ambulatory and home settings and has the potential to add value in the assessment of clinical patients’ recovery throughout their therapy.
Keywords
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
1 Introduction
Every year, there are over 600,000 cases of new or first strokes and over two million cases of traumatic brain injury in the United States alone [1, 2]. This constantly increasing group of patients require frequent neurorehabilitation therapy but may be unable to see clinicians as much as required. Taking into consideration recent technological developments such as low-cost tablet computers and cloud computing, the field of telemedicine has risen to meet the growing demand for available care at home settings [3]. Telemedicine applications allow for patients to perform clinician-mediated therapy and recover from the comfort of their own home, circumventing the need to travel to a specialized therapy clinic which often proves difficult due to various constraints. Moreover, the availability of telemedicine options has been shown to increase patient compliance [4] as well as be effective in treating apraxia of speech, motor control degradation in the arm, and other complications of stroke and injury [5, 6].
The importance of telemedicine also connects with the rapidly developing field of neuroergonomics. Neuroergonomics is the study of how the brain behaves in natural environments, in so-called ecological situations [7, 8]. Several studies have already proven the validity of this school of thought in areas ranging from air traffic control to interaction with virtual reality [9–11]. Recording the brain using a teletherapy application at home bypasses sterile lab environments and more closely simulates real-world use of the system.
Although computerized methods for cognitive rehabilitation designed for patients to use on demand are being continuously developed [12–14], most evidence of the efficacy of similar treatments is limited to behavioral or standardized psychological tests [15–17]. Moreover, these results are valid only for regimens developed by specialized clinicians, and studies have found that some commercial applications do not necessarily provide comparable results [18, 19]. These outline the need for controlled and independent assessment of the capabilities of cognitive rehabilitation tools and specifically neural correlates of the paradigms in use.
In this pilot study, we aimed to perform an initial assessment of select cognitive tasks in Constant Therapy (©2014 Constant Therapy, constanttherapy.com), an iPad-based cognitive rehabilitation app which is already in active use by clinicians and patients to augment traditional therapy. Constant Therapy provides over sixty tasks targeting various areas of cognition [20]. Two tasks targeting attention and working memory have been selected for the proof of concept experiment demonstrating that past studies related to cortical activation and task expertise are applicable to the tablet-based therapy paradigm [11]. We hypothesize that in line with earlier studies, there will be a quantifiable difference in the brain activation of healthy subjects when performing the same task at different difficulties [21]. This hypothesis was chosen for this pilot study to test if task-induced differences in brain activation could be detected in the Constant Therapy neurorehabilitation tool.
To test this hypothesis, we chose to use the portable and accessible neuroimaging modality of functional near-infrared spectroscopy (fNIRS). fNIRS is a safe, wearable and relatively low cost optical measuring system of neural activity related to cognitive state and workload [11, 22, 23]. Compared to the traditional neuroimaging systems such as electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), fNIRS has a balanced trade-off in spatial and temporal resolution, fast setup, silent operation, minimal restrictions, and a miniaturized, battery operated, and wearable sensor set that can be potentially used in a home setting, making it ideal for use with telemedicine brain monitoring in conjunction with therapy apps [24]. By including these physiological signals into the clinician’s assessment of patient recovery, more effective therapy can be applied, which can therefore improve patient outcomes.
2 Method: Brain Activation Measurements During Tablet Therapy Application Use
2.1 Participants
Ten participants between the ages of 18 and 35 (five females, mean age = 23.2 ± 2.8 years) volunteered for this study. All confirmed that they met the eligibility requirements of being right-handed with vision correctable to 20/20, did not have a history of brain injury or psychological disorder, and were not on medication affecting brain activity. Prior to the study all participants signed consent forms approved by the Institutional Review Board of Drexel University.
2.2 Experimental Procedure
The experiment was performed in a single session lasting for approximately one hour. The sessions were held in a specially prepared subject room in our lab building free from distraction and outside interference (Fig. 1). Subjects were recorded using the fNIRS system as they performed the experimental protocol on a provided iPad 2 placed on a stand. The task application used for this experiment was Constant Therapy, a program that allows for clinicians to work with patients both directly and indirectly, and is notable for its ability to provide smart telemedicine even when subjects are at home. Each task was presented in randomized order and downloaded from the internet on the fly. The standard outcome measures of the program were in behavioral performance, the accuracy and latency of each trial.
For the experiment, participants sat on a chair next to a desk with an iPad placed in a comfortable position before them, and were fitted with the fNIRS headband. Subjects were instructed to minimize head movement and only use their dominant (right) hands to interact with the iPad. The iPad was muted and screen brightness was set to maximum for consistency and clear view, and the iPad stand attached to the cover was set the same way for each subject. During the experiments the tablet screen was mirrored to a separate computer using Reflector 2 (©2016 Squirrels, LLC) and recorded using Microsoft Expression Encoder 4 (©2016 Microsoft Corporation).
The experiment was in two parts, one for each of the two tasks chosen, Symbol Matching and Pattern Recreation, as described below. Each of the tasks was given at two preselected difficulties, easy and hard, for a total of four block types throughout the experiment. In between blocks, subjects rested for 30 s to allow for their cortical oxygenation to return to resting levels, and a two-minute break was given in between the Symbol Matching and Pattern Recreation (Fig. 2). The exact order of blocks and whether Symbol or Pattern was presented first was balanced between subjects. The entire session took approximately 45 min.
2.2.1 Symbol Matching Task
The first task used was called Symbol Matching, which tested the subject’s attention (Fig. 3). In this task, subjects are shown a target symbol on the left part of the screen, and must select every panel exactly matching the target in shape and color, followed by hitting the “Check Answers” button to proceed. Clicking on an incorrect panel or hitting “Check Answers” without selecting every correct choice was considered an error. Subjects completed nine repetitions of this task in each block, and performed four blocks of each difficulty for a total of eight blocks of Symbol Matching. Blocks were presented in a pseudo-random order.
2.2.2 Pattern Recreation Task
The second task used was Pattern Recreation, which tested the subject’s working memory (Fig. 4). This task presents the user with a sequence of panels that flash one at a time, which the user must then recreate from memory. The number of panels in each trial varied from four to six depending on whether it was an easy or hard block. Hitting an incorrect panel or hitting one out of order was considered an error. If at any point the subject could not remember the remainder of the sequence, they were permitted to hit the “Repeat” button as many times as needed, which would then play back any remaining unpressed panels. Like before, subjects completed nine repetitions of each trial per block, and four blocks per difficulty for a total of eight. Blocks were again presented in a pseudo-random order.
2.3 Data Acquisition
We used a continuous wave fNIRS system (fNIR Devices LLC, fnirdevices.com) that was described previously in [11]. COBI Studio [22] software was used to acquire data from the headband and visualize and record light intensity at two wavelengths, 730 and 850 nm, at a rate of 2 Hz. The headband itself (Fig. 5) is a soft, flexible, and lightweight foam pad containing four LED light sources and ten near infrared light-sensitive detectors placed in a grid with a fixed 2.5 cm source detector distance, which combined to form a total of 16 distinct optodes (measurement areas). With the inclusion of ambient channels, 730 nm, and 850 nm, a total of 3 channels per optode, 48 channels data overall were collected. Time synchronization markers were used to align the signal with the start and end times of each block throughout the experiment session.
2.4 Data Analysis
The outcome measures for the behavioral data were the accuracy (percentage correct) and latency (time to completion) for each trial within all blocks. For the neuroimaging data, raw light intensity measurements at 730 nm, 850 nm, and ambient light recorded with COBI Studio were analyzed in fnirSoft software [25]. First, raw light intensity data were low pass filtered with an order 20 Hamming finite impulse response filter with a cutoff frequency of 0.1 Hz to attenuate high frequency noise, respiration, and cardiac cycle effects. Data were also run through a sliding-window motion artifact rejection (SMAR) algorithm in order to eliminate motion artifacts and saturated channels [26]. Relevant blocks for each task were isolated and extracted using the manual markers. As the outcome measure, oxygenation that is oxygenated-hemoglobin (HbO) minus deoxygenated-hemoglobin (HbR) values for each of the sixteen optodes were calculated using the Modified Beer-Lambert Law with respect to a ten second local baseline at the beginning of each block.
For statistical analysis, a repeated measures linear mixed model analysis with Bonferroni correction was performed for both behavioral measures and average oxygenation changes for each block across difficulty conditions (alpha = 0.05).
3 Results
3.1 Behavioral Measures
The behavioral results of symbol matching task are shown in Fig. 6. As expected, both accuracy (F1,451.2 = 9.87, p < 0.002) and latency (F1,438.5 = 920.53, p < 0.001) were significantly different between easy and hard task conditions. Similar results were found for the pattern matching task which are shown in Fig. 7. Again, as expected, both accuracy (F1,524.1 = 132.24, p < 0.001) and latency (F1,557.3 = 422.983, p < 0.001) were significantly different across easy and hard conditions.
3.2 fNIRS Measures
Average oxygenation changes showed the high contrast between task conditions. As expected, the higher level of activity observed for difficult conditions indicated a higher level of effort for those conditions. Significant differences were found for all but one optode between Symbol Matching tasks (Fig. 8, Table 1), but only one optode was found to provide a significant difference in cortex oxygenation while subjects performed Pattern Matching (Fig. 9, Table 1).
4 Discussion
In this study, we aimed to perform an initial proof of concept neuroergonomic evaluation of a clinical teletherapy tool, Constant Therapy, using a wearable and safe fNIRS neuroimaging system. Anterior prefrontal cortex activity during performance of the two cognitive tasks (attention and working memory) yielded high contrast differences in oxygenation changes across the task conditions (easy vs. hard).
We found that healthy subjects performing the two selected tasks of a tablet-based therapy application performed statistically worse (lower accuracy and higher latency) on the hard difficulty as compared to the easy one. Moreover, using functional near-infrared spectroscopy of the prefrontal cortex we found measurable differences in the cortical oxygenation of the subjects performing these tasks, demonstrating that consistent neural correlates exist, and can be potentially applied to a wider population using this system. This agrees with previous studies done using fNIRS on cognitive workload while performing challenging tasks, and demonstrates that higher mental effort is correlated with higher brain activation [10, 11, 24, 27–29].
This pilot study was limited by the number of subjects, as well as the time constraints we allowed for each experimental session. Given longer sessions or multiple recordings taken over several days, we would likely see more significant contrast in both task conditions and the longitudinal plasticity differences. This was exacerbated by our limitation of only using two distinct tasks. We also had to limit the range of difficulties for each task, as it was necessary to keep the test engaging but not too challenging in order not to overload the subjects during repetitive performance [30]. In future studies, more of the wide variety of tasks available could be tested, which will provide a richer information base for use of fNIRS measurements in conjunction with Constant Therapy in the field.
This pilot study provided initial proof that fNIRS-based neuroimaging techniques could be used to assess brain function changes during performance of the tasks and also potentially assess the efficacy of cognitive therapy applications during actual clinical use. Although we controlled as many aspects of the procedure as possible for the pilot study, the program itself was unaltered from the commercially available version, and therefore demonstrates the ecological validity of our results. In addition, this was the first neuroimaging study applied specifically with the Constant Therapy rehabilitation application. The results encourage future studies that can focus on cross sectional and longitudinal designs, including clinical populations, to further assess the utility of fNIRS to add value to a clinician’s ability to track and triage a patient’s recovery using quantitative brain measures. Moreover, if brain measures are integrated into the task design’s automated difficulty adjustment algorithms alongside the current smart analytics, they can improve the efficiency [31] and recovery speed as well as patients’ compliance [32].
In summary, this study demonstrates that fNIRS is a good candidate to enhance design (usability) and/or clinical field use of cognitive rehabilitation systems.
References
Oh, J.Y., Park, Y.T., Jo, E.C., Kim, S.M.: Current status and progress of telemedicine in Korea and other countries. Healthc. Inform. Res. 21, 239–243 (2015)
Riegler, L.J., Neils-Strunjas, J., Boyce, S., Wade, S.L., Scheifele, P.M.: Cognitive intervention results in web-based videophone treatment adherence and improved cognitive scores. Med. Sci. Monit. 19, 269–275 (2013)
Gaggioli, A., Meneghini, A., Morganti, F., Alcaniz, M., Riva, G.: A strategy for computer-assisted mental practice in stroke rehabilitation. Neurorehabil. Neural Repair 20, 503–507 (2006)
Varley, R., Cowell, P.E., Dyson, L., Inglis, L., Roper, A., Whiteside, S.P.: Self-administered computer therapy for apraxia of speech: two-period randomized control trial with crossover. Stroke 47, 822–828 (2016)
Parasuraman, R., Lee, J.D., Kirlik, A.: Neuroergonomics: Brain-Inspired Cognitive Engineering. Oxford University Press
Mehta, R.K., Parasuraman, R.: Neuroergonomics: a review of applications to physical and cognitive work. Front. Hum. Neurosci. 7, 889 (2013)
Derosière, G., Mandrick, K., Dray, G., Ward, T.E., Perrey, S.: NIRS-measured prefrontal cortex activity in neuroergonomics: strengths and weaknesses. Front. Hum. Neurosci. 7, 583 (2013)
Carrieri, M., Petracca, A., Lancia, S., Basso Moro, S., Brigadoi, S., Spezialetti, M., Ferrari, M., Placidi, G., Quaresima, V.: Prefrontal cortex activation upon a demanding virtual hand-controlled task: a new frontier for neuroergonomics. Front. Hum. Neurosci. 10, 53 (2016)
Ayaz, H., Shewokis, P.A., Bunce, S., Izzetoglu, K., Willems, B., Onaral, B.: Optical brain monitoring for operator training and mental workload assessment. NeuroImage 59, 36–47 (2012)
Kiran, S., Roches, C.D., Balachandran, I., Ascenso, E.: Development of an impairment-based individualized treatment workflow using an iPad-based software platform. Semin. Speech Lang. 35, 038–050 (2014)
Des Roches, C.A., Balachandran, I., Ascenso, E.M., Tripodis, Y., Kiran, S.: Effectiveness of an impairment-based individualized rehabilitation program using an iPad-based software platform. Front. Hum. Neurosci. 8 (2015)
Dang, J., Zhang, J., Guo, Z., Lu, W., Cai, J., Shi, Z., Zhang, C.: A pilot study of iPad-assisted cognitive training for schizophrenia. Arch. Psychiatr. Nurs. 28, 197–199 (2014)
Westerberg, H., Jacobaeus, H., Hirvikoski, T., Clevberger, P., Östensson, M.L., Bartfai, A., Klingberg, T.: Computerized working memory training after stroke–a pilot study. Brain Inj. 21, 21–29 (2007)
Bogdanova, Y., Yee, M.K., Ho, V.T., Cicerone, K.D.: Computerized cognitive rehabilitation of attention and executive function in acquired brain injury: a systematic review. J. Head Trauma Rehabil. (2015). (Epub ahead of print)
Ayaz, H., Shewokis, P.A., Scull, L., Libon, D.J., Feldman, S., Eppig, J., Onaral, B., Heiman-Patterson, T.: Assessment of prefrontal cortex activity in amyotrophic lateral sclerosis patients with functional near infrared spectroscopy. J. Neurosci. Neuroeng. 3, 41–51 (2014)
Melby-Lervag, M., Hulme, C.: Is working memory training effective? A meta-analytic review. Dev. Psychol. 49, 270–291 (2013)
Smith, S.P., Stibric, M., Smithson, D.: Exploring the effectiveness of commercial and custom-built games for cognitive training. Comput. Hum. Behav. 29, 2388–2393 (2013)
Molteni, E., Contini, D., Caffini, M., Baselli, G., Spinelli, L., Cubeddu, R., Cerutti, S., Bianchi, A.M., Torricelli, A.: Load-dependent brain activation assessed by time-domain functional near-infrared spectroscopy during a working memory task with graded levels of difficulty. J. Biomed. Opt. 17, 056005 (2012)
Ayaz, H., Shewokis, P.A., Curtin, A., Izzetoglu, M., Izzetoglu, K., Onaral, B.: Using mazesuite and functional near infrared spectroscopy to study learning in spatial navigation. J. Vis. Exp. 56, 3443 (2011)
Fishburn, F.A., Norr, M.E., Medvedev, A.V., Vaidya, C.J.: Sensitivity of fNIRS to cognitive state and load. Front. Hum. Neurosci. 8, 76 (2014)
Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P.A., McKendrick, R., Parasuraman, R.: Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development. Front. Hum. Neurosci. 7, 871 (2013)
Ayaz, H.: Functional Near Infrared Spectroscopy Based Brain Computer Interface. Drexel University, Philadelphia, PA (2010)
Ayaz, H., Izzetoglu, M., Shewokis, P.A., Onaral, B.: Sliding-window motion artifact rejection for functional near-infrared spectroscopy. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2010, 6567–6570 (2010)
Ruocco, A.C., Rodrigo, A.H., Lam, J., Di Domenico, S.I., Graves, B., Ayaz, H.: A problem-solving task specialized for functional neuroimaging: validation of the Scarborough adaptation of the Tower of London (S-TOL) using near-infrared spectroscopy. Front. Hum. Neurosci. 8, 185 (2014)
Hasan, A., Ben, W., Scott, B., Patricia, A.S., Kurtulus, I., Sehchang, H., Atul, R.D., Banu, O.: Cognitive workload assessment of air traffic controllers using optical brain imaging sensors. In: Advances in Understanding Human Performance, pp. 21–31. CRC Press (2010)
Gateau, T., Durantin, G., Lancelot, F., Scannella, S., Dehais, F.: Real-time state estimation in a flight simulator using fNIRS. PLoS One 10, e0121279 (2015)
Durantin, G., Gagnon, J.F., Tremblay, S., Dehais, F.: Using near infrared spectroscopy and heart rate variability to detect mental overload. Behav. Brain Res. 259, 16–23 (2014)
McKendrick, R., Ayaz, H., Olmstead, R., Parasuraman, R.: Enhancing dual-task performance with verbal and spatial working memory training: continuous monitoring of cerebral hemodynamics with NIRS. NeuroImage 85(3), 1014–1026 (2014)
Yuksel, B.F., Oleson, K.B., Harrison, L., Peck, E.M., Afergan, D., Chang, R., Jacob, R.J.: Learn piano with BACh: an adaptive learning interface that adjusts task difficulty based on brain state. In: CHI 2016 (2016)
Acknowledgements
The authors would like to thank Dr. Veera Anantha for his help with access to the Constant Therapy tasks.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Mark, J., Onaral, B., Ayaz, H. (2016). Evaluating Neural Correlates of Constant-Therapy Neurorehabilitation Task Battery: An fNIRS Pilot Study. In: Schmorrow, D., Fidopiastis, C. (eds) Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience. AC 2016. Lecture Notes in Computer Science(), vol 9743. Springer, Cham. https://doi.org/10.1007/978-3-319-39955-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-39955-3_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-39954-6
Online ISBN: 978-3-319-39955-3
eBook Packages: Computer ScienceComputer Science (R0)