Figures
Abstract
Background
Cognitive training (CT) is effective at improving cognitive outcomes in children with and without clinical impairment as well as older individuals. Yet whether CT is of any preventative health benefit to working age adults is controversial. Our objective was therefore to investigate the real-world efficacy of CT in the workplace, involving employees from across the working-age spectrum and addressing many of the design issues that have limited trials to date.
Methods and Findings
135 white collar employees of a large Australian public sector organization were randomised to either 16 weeks (20 minutes three times per week) of online CT or an active control (AC) program of equal length and structure. Cognitive, wellbeing and productivity outcome measures were analysed across three timepoints: baseline, immediately after training and 6 months post-training. CT effects on cognitive outcomes were limited, even after planned subgroup analyses of cognitive capacity and age. Unexpectedly, we found that our AC condition, which comprised viewing short documentaries about the natural world, had more impact. Compared to the CT group, 6 months after the end of training, those in the AC group experienced a significant increase in their self-reported Quality of Life (Effect Size g = .34 vs −.15; TIME×GROUP p = .003), decrease in stress levels (g = .22 vs −.19; TIME x GROUP p = .03), and overall improvement in Psychological Wellbeing (g = .32 vs −.06; TIME×GROUP p = .02).
Conclusions
CT does not appear to positively impact cognition or wellbeing amongst white collar office workers; however, short time-out respite activities may have value in the promotion of psychological wellbeing. Given looming challenges to workplace productivity, further work-based interventional research targeting employee mental health is recommended.
Trial Registration
This trial was registered with the Australian New Zealand Clinical Trials Registry: ACTRN12610000604000 (http://www.anzctr.org.au/TrialSearch.aspx).
Citation: Borness C, Proudfoot J, Crawford J, Valenzuela M (2013) Putting Brain Training to the Test in the Workplace: A Randomized, Blinded, Multisite, Active-Controlled Trial. PLoS ONE 8(3): e59982. https://doi.org/10.1371/journal.pone.0059982
Editor: Sonia Brucki, University Of São Paulo, Brazil
Received: November 8, 2012; Accepted: February 20, 2013; Published: March 28, 2013
Copyright: © 2013 Borness et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: While this study was sponsored by The Brain Department Pty Ltd (Sydney, Australia) through their provision of the intervention, CB, the primary researcher, was independently supported by an Australian Postgraduate Award scholarship. This arrangement did not alter the authors’ adherence to all the PLOS ONE policies on sharing data and materials and the funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: While this study was sponsored by The Brain Department Pty Ltd (Sydney, Australia) through their provision of the intervention, Catherine Borness, the primary researcher, was independently supported by an Australian Postgraduate Award scholarship. The Brain Department Pty Ltd had no role in experimental design, data analysis, interpretation, writing or approval of this paper. Michael Valenzuela has also received honoraria from The Brain Department Pty Ltd for running educational seminars for business leaders about cognitive training and the ageing workforce. This arrangement however does not alter the authors’ adherence to all the PLOS ONE policies on sharing data and materials.
Introduction
Demographic ageing of modern and developing nations as well as the rising incidence of mental health disorders represent major threats to workforce productivity in the coming decades [1]. Advanced age is the single greatest risk factor for cognitive decline [2], with each decade after 20 years of age associated with an 8% reduction in memory function [3], 7% reduction in frontal-executive function and 8% reduction in attentional capacity [4]. In addition, the Australian Productivity Commission has found that of the six major health conditions, mental illness predicts the lowest likelihood of workforce participation [5]. Depression in particular is detrimental to job performance [6] and has further negative effects on cognitive ability [7]. Moreover, the interaction of advanced age and depression is particularly potent, increasing the risk of mild cognitive impairment (MCI) and rapid age-related cognitive decline [8]. Taken together, these population-level changes are already placing pressure on workforce productivity.
At the same time, neuroscientific studies of cognition and mental health are having an influence on organizational behaviour and human resource management [9], [10]. Computerized cognitive training (CT), or ‘brain training’, has received much attention in the clinical environment, with evidence of improved symptoms in depression [11], [12], schizophrenia [13], [14], Attention Deficit Hyperactivity Disorder [15], MCI and dementia [16], [17]. Furthermore, long term benefits after the cessation of training have also been reported [14], [18]. However, efficacy of CT in healthy working age individuals is contested. Owen et al (2010) questioned the impact of CT on cognitive ability in healthy adults and the transfer of training to non-trained tasks [19], but this study has been criticised on a number of methodological grounds [20], [21]. While CT has been reported to be vocationally effective, for example, targeted single domain CT improves motor control in surgeons [22] as well as pilots’ flight performance through enhanced attentional control [23], it remains untested in the workplace as a human resource intervention for the prevention or maintenance of cognitive capacity or mental health outcomes.
Our objective was therefore to investigate the real-world efficacy of CT in the workplace, involving employees from across the full working age spectrum and addressing many of the design issues that have limited CT studies to date [24]. We randomized participants to either 16 weeks of online CT or an online AC program of equal length and structure. Outcomes included various cognitive, wellbeing and productivity measures which were collected pre-training, immediately post-training and 6 months post-training. We also planned an a priori subgroup analysis based on a split of baseline subjects into low- and high- cognitive capacity. Our specific aims were to test whether computerized cognitive brain training would, i) increase cognitive abilities vital to effective and efficient workplace performance, ii) augment positive psychological measures of wellbeing and quality of life, and iii) improve objective measures of workplace productivity.
Methods
The protocol for this trial and supporting CONSORT checklist are available as supporting information (see Checklist S1 and Protocol S1).
Ethics Statement
This research was approved by the University of New South Wales’ Human Research Ethics Committee. Written informed consent was obtained from each volunteer. By giving Informed Consent, participants were also agreeing that they did not meet any of the exclusion criteria.
Research Design
This study was a 1∶1 randomized, active controlled, single-blind, multi-centre intervention trial with longitudinal follow-up 6 months post intervention, approximately 10 months since the start of the trial.
Participants
Our sample consisted of full time and part time (working a minimum of 3 days per week) staff from an Australian national public service organization aged between 18 and 65 and employed for a minimum of 6 months by that organization at one of six different office locations around Australia. Any volunteers who were currently using any other form of computer based brain training, were planning to take 3 or more weeks’ leave during the intervention period, or had any clinical diagnoses or treatment for mood disorders or drug dependency, were excluded from the study. While 178 employees initially volunteered for the study, the final intention-to-treat (ITT) sample comprised 135 participants. See Figure 1: Consort Flow Diagram.
Based on power analyses conducted in our pilot study, a sample size of 220 (110 for each condition) was ideal however the final sample did not reach this number. (See Protocol S1 for more information on power analysis). recruited sample (N = 178) did not differ significantly from the ITT (N = 135) group on any demographic variables. In addition, the ITT group (N = 135) and the per protocol completers (PPC) group (N = 88), that is, those participants who completed the full assessment battery on the 3 occasions (baseline, short term and long term follow up) did not differ significantly on any demographic variables, nor on baseline performance outcome measures. See Table 1: Demographic and Baseline Data.
Cognitive Training & Active Control
CT comprised 36 HappyNeuron (Scientific Brain Training, Lyon, France) [25] exercises across the domains of memory, attention, language, executive function and visuospatial abilities delivered online to each worksite using the Spark!™ software system (The Brain Department Pty Ltd, Sydney Australia). During each 20 minute training session, subjects completed a number of exercises from across a range of these domains, and were gradually challenged by exercises of greater cognitive demand tailored to their abilities, facilitated by the program’s in-built algorithms. The AC condition consisted of viewing a series of general interest videos about the natural environment (National Geographic) and answering related multiple choice questions delivered via an online survey. Both interventions ran for 16 weeks with 3 sessions per week (20 minutes per session) and were matched for duration, level of audio and visual stimulus and mode of delivery (online and directly to the participant’s regular work computer).
Outcome Measures
Outcome measures were collected and analysed by an organizational psychologist who remained blind to the training status of participants. All outcomes were measured at Baseline, immediately after the initial 16 week period of training (Short Term Follow Up), and then 6 months after the end of training (Long Term Follow Up). They covered cognitive, psychological wellbeing and productivity outcomes. Cognitive measures were independent of the CT intervention to reduce possible practice effects. Primary and secondary outcome measures are described in Supporting Information S1.
Statistical Methods
Initial analyses were run on PPC followed by ITT using a repeated measures approach. Primary outcomes were considered separately within each group (i.e. cognitive, wellbeing and productivity). Both univariate and multivariate analyses accounting for multiple comparisons were conducted. Further statistical details are provided in Supporting Information S1.
Results
All reported outcomes are based on PPC and replicated through imputation techniques based on ITT analyses. Baseline analysis of our ITT population highlighted that in comparison to the general population, this sample was equally competent across all cognitive measures except the COWAT, where they performed less effectively than their age-matched comparison group (z = −1.23) [26]. This population also had similar levels of subjective psychological wellbeing to the general population, with the exception of a higher level of self-reported Professional Self Esteem (z = +1.65) and a lower level of self-reported Personal Growth (z = −1.14), a subscale of the Scales of Psychological Wellbeing [27].
Compliance and Subjective Feedback
Training compliance rates between the CT and AC groups were equivalent at 81.6% and 82.8% respectively. Survey data post training indicated that participants found both conditions highly engaging. While this study achieved a compliance rate above 80% for both conditions, it also identified two key user issues that impacted upon full compliance: lack of time and workplace distractions.
Effect of Cognitive Training on Cognition
On completion of training.
There were 2 significant post-training TIME x TRAINING GROUP interactions for cognitive measures of divided attention and language. A Hedge’s effect size [28] of g = .29 was found for the CT group on one of the divided attention response time tasks (DIFINDRTC) compared to an effect size of.01 for the AC group (TIME x GROUP: F(1,90) = 4.40; p = .04) immediately post intervention. Those who completed CT became faster at the task, while little or no improvement was observed in the AC group on this measure. However, this was no longer significant after correction for multiple testing [29], [30]. In contrast, although both the CT and AC groups improved in the Language measure (COWAT) over the 6 month period (TIME: F(1,91) = 7.92; p = .006), the AC group achieved a greater effect size (g = .50) than the CT group (g = .28; TIME x GROUP: F(1,91) = 4.41, p = .04). This latter finding may reflect the largely language-based nature of the AC activity in the form of active listening and comprehension practised in each session, whereas the CT intervention loaded on language only 20% of the time, with only one out of five activities containing a language component. This finding also remained significant after Bonferroni correction. Both these effects are illustrated in Figure 2.
Error bars represent SEMs. P-value is for TIME x TRAINING GROUP interaction.
Effect of Cognitive Training on Wellbeing
6-month follow-up.
There were no significant TIME x TRAINING GROUP differences on measures of wellbeing immediately after the completion of training. However, significant TIME x TRAINING GROUP differences were found in two wellbeing variables at long term follow up. Unexpectedly, these benefits occurred in the AC rather than the CT intervention. Those in the AC group experienced a significant increase in their self-reported Quality of Life (g = .34) compared to the CT group (g = −.15; p = .003) and stress levels also declined significantly for the AC group (g = .22) but increased for the CT group (g = −.19; p = .03). However, this effect on stress was no longer significant after Bonferroni adjustment. Overall, Psychological Wellbeing improved for the AC group (g = .32) but not for the CT group (g = −.06; p = .02). These findings are illustrated in Figure 3. The concordance of these findings over a number of wellbeing measures suggests that our AC condition may have had a positive impact on the self-reported wellbeing of this working sample.
Error bars represent SEMs. P-value is for TIME x TRAINING GROUP interaction at the long term follow up point.
Effect of Cognitive Training on Productivity
On completion of training.
There were no significant TIME x TRAINING GROUP differences on various measures of productivity over the short term. A complete participant data set comprised data for each of the nine productivity variables. For employer organisational reasons there were few participants who had all nine pieces of information available at baseline, short term and long term follow up. Using imputation methods to account for the 25% of missing data at short term follow up did not alter our findings.
Discussion
The unqualified use of CT in the work environment amongst healthy adults found no support in this study, even when levels of cognitive capacity and age group were taken into consideration. In terms of wellbeing, significant positive changes were reported over the long term, however, this was observed in the AC rather than the CT intervention. It is possible that this alternative form of mental activity may have a place in the promotion of wellbeing amongst white collar workers, but will require further investigation. Productivity outcomes were difficult to interpret given the large amount of missing data due to inconsistent collection and reporting through the workplace’s organizational systems. A more robust, externally administered and validated tool may be required for future studies intending to evaluate the productivity impact of workplace interventions.
Whilst the impact of CT on cognition may be dose-dependent [31], [32], the ‘dose’ has yet to be determined. So far, the literature suggests that at least four weeks of daily exercises are necessary to enhance cognition in any enduring way in non-aged populations, presuming that the training program is effective at the outset [33], [34]. Lustig et al (2009) conducted an extensive review of studies investigating the effects of various interventions on the cognitive ageing process in healthy older adults [31]. Training dose varied widely, ranging from just one session to 40 hours of training over eight weeks. A meta-analysis of studies of healthy adults suggested that persistent protective benefits required at least a two to three month training program [17]. Our study applied a realistic dose based on the literature (16 hours, 20 minutes three times a week) however it was spread over 16 weeks due to workplace restrictions. This ‘dilution’ of training may be one reason for the limited effects we observed and so the effects of more concentrated and extended doses should be investigated. Other challenges and limitations worthy of note include measurement of cognitive change in a cognitively intact sample, recruitment and retention of participants over a longitudinal study and access to accurate and complete productivity data.
Quite unexpectedly, our AC, which involved viewing an extensive series of short National Geographic documentaries, appeared to have an enduring and positive impact on a number of wellbeing measures. The potential impact of simply taking ‘time-out’ breaks during the work day has recently gained support. In fact, work day breaks have been shown to counter effects of fatigue and actually increase productivity [35]. The nature of any working-day break task also has important implications for the recovery process. Trougakos and Hideg (2009) distinguish between ‘respites’ and ‘chores’ [36]. Respite activities are low effort or preferred by choice, which by their nature allow individuals to restore their personal resources for future work effectiveness. Chores are by contrast non-preferred activities that deplete the individual’s personal resources. Our AC may have been perceived as a respite-type break due to the non-work related content and lack of associated performance pressure, a view supported by qualitative feedback from this group. Further research is required to replicate and understand the nature of this serendipitous finding.
Overall, this trial provides little support for the material benefit of CT to workers in roles of moderate cognitive complexity [37]. On the other hand, new and unexpected evidence was found for the idea that respite-type breaks during work hours can benefit workplace mental wellbeing. For employers, attempting to improve their employee’s cognitive resources may provide limited returns, whereas attention to their mental health and wellbeing could potentially result in improved work performance [7], [8]. Workplace interventions targeting mental health, rather than cognition alone, may help employees and employers achieve a more productive work environment.
Supporting Information
Supporting Information S1.
Supporting Information.
https://doi.org/10.1371/journal.pone.0059982.s003
(DOCX)
Author Contributions
Provided additional guidance on the study design, analysis and content of the paper: JP. Conceived and designed the experiments: MV. Performed the experiments: CB. Analyzed the data: CB JC. Contributed reagents/materials/analysis tools: JC. Wrote the paper: CB MV.
References
- 1.
Beddington J, Cooper CL, Field J, Goswami FA, Jenkins R, et al.. (2008) Mental Capital and Wellbeing: Making the most of ourselves in the 21st century (Executive Summary). London: Government Office for Science. pp. 1–56.
- 2. Brayne C (2007) The elephant in the room - healthy brains in later life, epidemiology and public health. Nature Reviews Neuroscience 8: 233–239.
- 3.
Wechsler D (1997a) Wechsler Memory Scale - Third Edition. Administration and Scoring Manual. San Antonio, Texas.: The Psychological Corporation, Harcourt Brace.
- 4.
Wechsler D, editor (1997b) Wechsler Adult Intelligence Scale - Third Edition. Administration and Scoring Manual. San Antonio, Texas: The Psychological Corporation, Harcourt Brace.
- 5.
Laplagne P, Glover M, Shomos A (2007) Effects of health and education on labour force participation. In: Commission P, editor. Melbourne: Australian Government.
- 6. Lerner D, Henke RM (2008) What does research tell us about depression, job performance, and work productivity? Journal of Occupational and Environmental Medicine 50: 401–410.
- 7. Knegtering H, Eijck M, Huijsman A (1994) Effects of antidepressants on cognitive functioning of elderly patients: A review. Drugs Aging 5: 192–199.
- 8. Hertzog C, Kramer AF, Wilson RS, Lindenberger U (2009) Enrichment effects on adult cognitive development: can the functional capacity of older adults be preserved and enhanced? Psychological Science in the Public Interest 9: 1–65.
- 9.
Gordon E (2008) NeuroLeadership and Integrative Neuroscience: “it’s about validation stupid!”. NeuroLeadership Journal.
- 10.
Becker WJ, Cropanzano R (2010) Organizational neuroscience: The promise and prospects of an emerging discipline. Journal of Organizational Behavior 31.
- 11. Wolinsky FD, Mahncke HW, Weg MWV, Martin R, Unverzagt FW, et al. (2009) The ACTIVE cognitive training interventions and the Onset of and recovery from suspected clinical depression. Journals of Gerontology - Series B Psychological Sciences and Social Sciences 64: 577–585.
- 12. Elgamal S, McKinnon MC, Ramakrishnan K, Joffe RT, MacQueen G (2007) Successful computer-assisted cognitive remediation therapy in patients with unipolar depression: A proof of principle study. Psychological Medicine 37: 1229–1238.
- 13. McGurk SR, Twamley EW, Sitzer DI, McHugo GJ, Mueser KT (2007) A meta-analysis of cognitive remediation in schizophrenia. American Journal of Psychiatry 164: 1791–1802.
- 14. Lindenmayer JP, McGurk SR, Mueser KT, Khan A, Wance D, et al. (2008) A randomized controlled trial of cognitive remediation among inpatients with persistent mental illness. Psychiatric Services 59: 241–247.
- 15. Klingberg T, Fernell E, Olesen PJ, Johnson M, Gustafsson P, et al. (2005) Computerized Training of Working Memory in Children With ADHD-A Randomized, Controlled Trial. Journal of the American Academy of Child and Adolescent Psychiatry 44: 177–186.
- 16. Sitzer DI, Twamley EW, Jeste DV (2006) Cognitive training in Alzheimer’s disease: a meta-analysis of the literature. Acta Psychiatrica Scandinavica 114: 75–90.
- 17. Valenzuela M, Sachdev P (2009) Can Cognitive Exercise Prevent the Onset of Dementia? Systematic Review of Randomized Clinical Trials with Longitudinal Follow-up. [Article]. American Journal of Geriatric Psychiatry March 17: 179–187.
- 18. Willis SL, Tennstedt SL, Marsiske M, Ball K, Elias J, et al. (2006) Long-term Effects of Cognitive Training on Everyday Functional Outcomes in Older Adults. JAMA 296: 2805–2814.
- 19. Owen AM, Hampshire A, Grahn JA, Stenton R, Dajani S, et al. (2010) Putting brain training to the test. Nature 465: 775–778.
- 20. Katsnelson A (2010) No gain from brain training. Nature 464: 1111.
- 21.
Zelinski EM (2010) Scientific Critique of BBC Nature Brain Training Experiment. In: SharpBrains, editor.
- 22. Schlickum M, Hedman L, Enochsson L, Kjellin A, Fellander-Tsai L (2009) Systematic Video Game Training in Surgical Novices Improves Performance in Virtual Reality Endoscopic Surgical Simulators: A Prospective Randomized Study. World Journal of Surgery 33: 2360–2367.
- 23. Gopher D, Well M, Bareket T (1994) Transfer of skill from a computer game trainer to flight. Human Factors: The Journal of the Human Factors and Ergonomics Society 36: 387–405.
- 24.
Gates N, Sachdev P, Fiatarone-Singh M, Valenzuela M (2011) Cognitive and memory training in adults at risk of dementia: A Systematic Review. BMC Geriatrics 11.
- 25.
Croisile B, Wesson-Ashford J, Bender R, Vianin P, Belier S, et al.. (2000) Spark! Brains@Work. Lyon, France: Happy Neuron, A Scientific Brain Training Company.
- 26.
Spreen O, Strauss E, editors (1991) A Compendium of Neuropsychological Tests: Administration, Norms and Commentary: Oxford University Press: NY, Oxford.
- 27. Ryff CD, Keyes CLM (1995) The Structure of Psychological Well-Being Revisited. Journal of Personality and Social Psychology 69: 719–727.
- 28.
Hedges LV, Olkin I (1985) Statistical Methods for Meta-Analysis; Academic Press I, editor. New York: Harcourt Brace Jovanovich.
- 29.
Miller RG (1981) Simultaneous Statistical Inference. New York: Springer-Verlag.
- 30. Simes RJ (1986) An improved Bonferroni procedure for multiple tests of significance. Biometrika 73: 751–754.
- 31. Lustig C, Shah P, Seidler R, Reuter-Lorenz PA (2009) Aging, training, and the brain: a review and future directions. Neuropsychology Review 19: 504–522.
- 32. Jaeggi SM, Buschkuehl M, Jonides J, Perrig WJ (2008) Improving fluid intelligence with training on working memory. PNAS 105: 6829–6833.
- 33. Mahncke HW, Connor BB, Appelman J, Ahsanuddin ON, Hardy JL, et al. (2006) Memory enhancement in healthy older adults using a brain plasticity-based training program: A randomized, controlled study. PNAS 103: 12523–12528.
- 34. Barnes D, Yaffe K, Belfor N, Jagust W, DeCarli C, et al. (2009) Computer-based cognitive training for mild cognitive impairment: results from a pilot randomized, controlled trial. Alzheimer’s Disease and Associated Disorders 23: 205–210.
- 35. Tucker P (2003) The impact of rest breaks upon accident risk, fatigue and performance: A review. Work & Stress 17: 123–137.
- 36. Trougakos JP, Hideg I (2009) Momentary Work Recovery: The Role of Within-Day Work Breaks. Research in Occupational Stress and Well Being 7: 37–84.
- 37.
Jaques E, Cason K (1994) Human Capability: A Study of Individual Potential and its Application. Falls Church, Virginia: Cason Hall & Co.