Introduction
Multiple sclerosis (MS) is an inflammatory disease of the central nervous system that increases the risk for cognitive dysfunction. The effect on cognition is highly prevalent as cognitive impairment has been observed in 40–73% of persons with multiple sclerosis (PwMS) (Chiaravalloti & DeLuca, Reference Chiaravalloti and DeLuca2008; Grzegorski & Losy, Reference Grzegorski and Losy2017). Of further importance, cognitive impairment has been found to negatively impact activities of daily living, particularly those classified as mobility-based and/or physically demanding, and that impact completion of routine household chores (Einarsson et al., Reference Einarsson, Gottberg, Fredrikson, von Koch and Holmqvist2006; Goverover, Reference Goverover, DeLuca and Sandroff2018; Rao et al., Reference Rao, Leo, Ellington, Nauertz, Bernardin and Unverzagt1991). Cognitive impairment has also been associated with decreased social and avocational activities (Rahn et al., Reference Rahn, Slusher and Kaplin2012; Rao et al., Reference Rao, Leo, Ellington, Nauertz, Bernardin and Unverzagt1991), increased psychopathology (Arnett & Smith, in Press), poorer quality of life (Campbell et al., Reference Campbell, Rashid, Cercignani and Langdon2017; Rao et al., Reference Rao, Leo, Ellington, Nauertz, Bernardin and Unverzagt1991), and greater occupational impairment (Cadden & Arnett, Reference Cadden and Arnett2015; Rao et al., Reference Rao, Leo, Ellington, Nauertz, Bernardin and Unverzagt1991; Roessler et al., Reference Roessler, Rumrill and Fitzgerald2004) in PwMS. Taken together, cognitive impairment and related factors likely contribute to the high rate of unemployment in PwMS, which occurs in as many as 80% of adults with MS following diagnosis (Julian et al., Reference Julian, Vella, Vollmer, Hadjimichael and Mohr2008; Roessler & Rumrill, Reference Roessler and Rumrill2003; Strober et al., Reference Strober, Chiaravalloti and DeLuca2018). Consideration for the widespread impact and potential ramifications of cognitive dysfunction shows the importance of routine assessment and monitoring of cognition in PwMS. However, given that comprehensive neuropsychological testing can be resource-intensive, clinicians and researchers will likely benefit from identification of a brief cognitive screener that is accurate and sensitive to MS-related difficulties and can inform the decisions for comprehensive testing.
To this end, the Multiple Sclerosis Neuropsychological Screening Questionnaire (MSNQ) was developed by Benedict and colleagues (Benedict et al., Reference Benedict, Munschauer, Linn, Miller, Murphy, Foley and Jacobs2003a). The MSNQ is a cognitive screener that consists of 15 items related to neurocognitive functioning with higher scores indicating greater cognitive impairment. Importantly, there are both self-report and informant-report versions of the MSNQ. While the MSNQ is designed to fill the need for a brief but sensitive screener for cognitive impairment in PwMS, studies have shown mixed findings with the utility of the MSNQ – particularly when examining the MSNQ-Self (MSNQ-S). For example, Benedict and Zivadinov (Reference Benedict and Zivadinov2006) found that the MSNQ-S and MSNQ-Informant (MSNQ-I) were inversely correlated with neurocognitive performance across several domains (i.e., higher MSNQ scores were associated with worse performance), though the MSNQ-S was also highly correlated (r = 0.56) with depression. Additionally, Nauta et al. (Reference Nauta, Balk, Sonder, Hulst, Uitdehaag, Fasotti and de Jong2019) found that higher MSNQ-S scores were indicative of cognitive impairment, though they noted that this pattern was more pronounced in participants with higher levels of education compared to those with lower levels of education. Another study by Randolph et al. (Reference Randolph, Arnett and Higginson2001) showed that verbal recall, attentional, and executive tasks were significantly correlated with a self-reported metamemory measure (Memory Rating Scale – MRS) in PwMS. Significant other ratings on the MRS were also significantly (p < .05) correlated with verbal recall as well as attentional measures. In contrast, some findings suggest that self-report cognitive measures like the MSNQ-S do not correlate with objective neuropsychological performance and are not sensitive to cognitive impairment in PwMS (Benedict et al., Reference Benedict, Munschauer, Linn, Miller, Murphy, Foley and Jacobs2003a; O’Brien et al., Reference O’Brien, Gaudino-Goering, Shawaryn, Komaroff, Moore and DeLuca2007). Another study that used the Cognitive Failures Questionnaire as a screener for perceptions of cognitive functioning in MS did not find a significant association with objective neurocognitive performance (Middleton et al., Reference Middleton, Denney, Lynch and Parmenter2006). Thus, the clinical utility of self-reported cognitive functioning, such as with the MSNQ-S, is unclear. In comparison, several previous studies have shown that the MSNQ-I is predictive of cognitive impairment and is not correlated with mood disturbance (Benedict et al., Reference Benedict, Munschauer, Linn, Miller, Murphy, Foley and Jacobs2003a; Benedict & Zivadinov, Reference Benedict and Zivadinov2006; O’Brien et al., Reference O’Brien, Gaudino-Goering, Shawaryn, Komaroff, Moore and DeLuca2007).
Notably, research to this point has primarily focused on examining the MSNQ in relation to global cognitive functioning rather than examining specific cognitive domain composites (e.g., processing speed, executive function, and memory). For example, early work examining the sensitivity and specificity of the MSNQ categorized participants as “neurocognitively impaired” if their neuropsychological summary score (which is defined as the mean standard score across a comprehensive neuropsychological battery) fell below the fifth percentile (Benedict et al., Reference Benedict, Munschauer, Linn, Miller, Murphy, Foley and Jacobs2003a). Another study applied similar criteria, though they classified participants as “neurocognitively impaired” if they had one cognitive domain below the fifth percentile and “not impaired” if they did not have any domains below the fifth percentile, and then examined the MSNQ’s ability to classify participants (O’Brien et al., Reference O’Brien, Gaudino-Goering, Shawaryn, Komaroff, Moore and DeLuca2007). While these findings are important and introduce greater understanding of the utility of the MSNQ, they are limited in that they do not provide domain-specific information. Thus, the current study aims to fill this gap by examining the associations between the MSNQ and performance within specific cognitive domains.
Regarding the evaluation of neurocognitive performance, a large majority of research has focused solely on examining mean performance and then making comparisons to a normative group. While this approach has been employed for decades and has demonstrated remarkable clinical utility for a broad range of neurological disorders, some research suggests that meaningful information about an individual’s neurocognitive profile may be missed when strictly making comparisons to normative data (Hilborn et al., Reference Hilborn, Strauss, Hultsch and Hunter2009; Hultsch et al., Reference Hultsch, MacDonald and Dixon2002; Tanner-Eggen et al., Reference Tanner-Eggen, Balzer, Perrig and Gutbrod2015). Relatedly, previous research has suggested that IIV may actually be a better predictor of cognitive outcome than mean differences in a variety of clinical samples (Burton et al., Reference Burton, Strauss, Hultsch, Moll and Hunter2006; Cole et al., Reference Cole, Weinberger and Dickinson2011; Haynes et al., Reference Haynes, Bauermeister and Bunce2017). More recently, Riegler et al. (Reference Riegler, Cadden, Guty, Bruce and Arnett2021) found that patient status (i.e., PwMS vs. Healthy Controls) predicted increased intraindividual variability (IIV) on measures of attention/processing speed and memory, such that PwMS demonstrated greater variability. Taken together, these findings suggest that it is important to evaluate indices of IIV in addition to mean neurocognitive performance scores.
Current study
The current study aims to further clarify the relationship between subjective reports of cognitive function and objective neurocognitive performance. As such, the specific aims are as follows: Aim 1) Examine the relationship between subjective reports of cognitive functioning (MSNQ-S and MSNQ-I) and mean performance on each neurocognitive domain within a comprehensive neuropsychological battery; Aim 2) Examine the relationship between subjective reports of cognitive functioning (MSNQ-S and MSNQ-I) and variability of performance on a comprehensive neuropsychological battery.
Regarding aim 1, previous research has found that MS typically impacts domains of processing speed (Bobholz & Rao, Reference Bobholz and Rao2003; DeLuca et al., Reference DeLuca, Chelune, Tulsky, Lengenfelder and Chiaravalloti2004; Riegler et al., Reference Riegler, Cadden, Guty, Bruce and Arnett2021; van Geest et al., Reference van Geest, Douw, van ’t Klooster, Leurs, Genova, Wylie, Steenwijk, Killestein, Geurts and Hulst2018), executive function (Denney et al., Reference Denney, Sworowski and Lynch2005; Drew et al., Reference Drew, Tippett, Starkey and Isler2008; Lazeron et al., Reference Lazeron, Rombouts, Scheltens, Polman and Barkhof2004), and memory (Brissart et al., Reference Brissart, Morele, Baumann and Debouverie2012; Chiaravalloti & DeLuca, Reference Chiaravalloti and DeLuca2008; Rao, Reference Rao2004; Riegler et al., Reference Riegler, Cadden, Guty, Bruce and Arnett2021). Alternatively, research has demonstrated that general intelligence and simple/focused attention typically remain intact (Chiaravalloti & DeLuca, Reference Chiaravalloti and DeLuca2008; Macniven et al., Reference Macniven, Davis, Ho, Bradshaw, Szabadi and Constantinescu2008; Rao et al., Reference Rao, Leo, Ellington, Nauertz, Bernardin and Unverzagt1991). Considering these findings, as well as previous research with the MSNQ, we predict that the MSNQ-I will inversely correlate with mean performance on indices of processing speed, executive function, and memory but not focused attention. Given that prior research is mixed regarding the association of the self-reported cognitive measures like the MSNQ-S and objective cognitive problems (Benedict et al., Reference Benedict, Munschauer, Linn, Miller, Murphy, Foley and Jacobs2003a; but cf. Benedict & Zivadinov, Reference Benedict and Zivadinov2006; and Middleton et al., Reference Middleton, Denney, Lynch and Parmenter2006; Nauta et al., Reference Nauta, Balk, Sonder, Hulst, Uitdehaag, Fasotti and de Jong2019; O’Brien et al., Reference O’Brien, Gaudino-Goering, Shawaryn, Komaroff, Moore and DeLuca2007; and Randolph et al., Reference Randolph, Arnett and Higginson2001), we examined the MSNQ-S in relation to mean performance on the neurocognitive measures without a set prediction.
Regarding aim 2, previous work has demonstrated that greater variability in performance (i.e., greater dispersion of scores across a battery) indicates greater neurocognitive impairment (Burton et al., Reference Burton, Strauss, Hultsch, Moll and Hunter2006; Cole et al., Reference Cole, Weinberger and Dickinson2011; Haynes et al., Reference Haynes, Bauermeister and Bunce2017; Rabinowitz & Arnett, Reference Rabinowitz and Arnett2013). As such, while IIV has not yet been examined in the manner outlined in the current study, we hypothesize that a similar pattern will emerge when examining IIV as when examining mean performance in relation to the MSNQ. That is, we hypothesize that the MSNQ-I will positively correlate with variability such that higher scores (more reported cognitive dysfunction) will be associated with greater variability in performance. Similar to the MSNQ-S in relation to the variability indices, and mixed findings in the literature, we tested these associations but without offering set predictions.
Methods
Procedure
This study involved an analysis of data collected as part of a project examining cognitive, motor, and emotional factors in MS. Analyses from this project were run on data collected as part of an ongoing longitudinal study on MS. Participants completed a psychosocial interview and a battery of neuropsychological tests and questionnaires during a 1-day visit. Participants gave informed consent according to institutional guidelines and were treated in accordance with the ethical standards of the American Psychological Association and the Helsinki Declaration, and the study was approved by the Institutional Review Board at our institution.
Participants
87 PwMS, 65 female and 22 male, completed a comprehensive neuropsychological battery that included neurocognitive tests and self-report measures of depression and neurocognitive functioning. Participants for this study were all diagnosed with MS by board-certified neurologists using the revised McDonald criteria as described by Polman et al. (Reference Polman, Reingold, Banwell, Clanet, Cohen, Filippi, Fujihara, Havrdova, Hutchinson, Kappos, Lublin, Montalban, O’Connor, Sandberg-Wollheim, Thompson, Waubant, Weinshenker and Wolinsky2011) and were recruited from the greater Central Pennsylvania area. Exclusion criteria included significant history of substance use disorder, nervous system disorder other than MS, sensory impairment that could interfere with testing, developmental history of attention-deficit/hyperactivity disorder (ADHD) or learning disability, significant medical condition other than MS that could interfere with cognitive or motor function, disease relapse or corticosteroid use within four weeks of participation in the study, or physical or neurological impairment that would prohibit testing. MS course types included relapsing-remitting (n = 55 [63.2%]), secondary progressive (n = 26 [29.9%]), and primary progressive (n = 6 [6.9%]).
Measures
Demographic and illness-related information including age, sex, years of education, disease duration, and course type were collected as part of a semi-structured psychosocial interview administered by a doctoral student in clinical psychology. Neurological disability was evaluated using the Expanded Disability Status Scale (EDSS) (Kurtzke, Reference Kurtzke1983). Full-Scale IQ was predicted using the Wechsler Test of Adult Reading (WTAR; Wechsler, Reference Wechsler2001), a 50-item reading test that estimates premorbid cognitive ability. Depression was measured using the Beck Depression Inventory-Fast Screen (BDI-FS) (Beck et al., Reference Beck, Steer and Brown2000). The BDI-FS is a commonly used brief self-report measure of depression in medical populations. Previous work has suggested that the BDI-FS is an appropriate screener for depression in MS since it excludes neurovegetative symptoms that commonly overlap with symptoms of MS (Benedict et al., Reference Benedict, Fishman, McClellan, Bakshi and Weinstock-Guttman2003b; Strober & Arnett, Reference Strober and Arnett2015). It includes seven items that examinees rate based on how they have felt over the past two weeks. Each item has four statements that are assigned a value of 0 through 3, with lower scores indicating lower depression symptomatology. Detailed demographic information can be found in Table 1.
Table 1. Demographic information
Note. EDSS = expanded disability status scale; FSIQ = full-scale IQ.
Subjective cognitive functioning was assessed with the MSNQ (Benedict et al., Reference Benedict, Munschauer, Linn, Miller, Murphy, Foley and Jacobs2003a). We examined both the MSNQ-Self-Report (MSNQ-S) and MSNQ-Informant-Report (MSNQ-I). Objective neurocognitive functioning was assessed with measures based on the Minimal Assessment of Cognitive Function in Multiple Sclerosis, a validated approach to routine neuropsychological assessment of PwMS (Benedict et al., Reference Benedict, Cookfair, Gavett, Gunther, Munschauer, Garg and Weinstock-Guttman2006; Strober et al., Reference Strober, Englert, Munschauer, Weinstock-Guttman, Rao and Benedict2009). The neurocognitive test battery included subscores from the following measures: Digit Span (Weschler, Reference Weschler1997), Oral Symbol Digit Modalities Test (SDMT) (Shum et al., Reference Shum, McFarland and Bain1990), Controlled Oral Word Association Test (COWAT), Animal Naming (Delis et al., Reference Delis, Kaplan and Kramer2001), Paced Auditory Serial Addition Task (PASAT) – 3-Second Trial (Rao & the Cognitive Function Study Group of the National Multiple Sclerosis Society, Reference Rao1990), Digit Symbol Substitution Test (Wechsler, Reference Wechsler1958), Delis-Kaplan Executive Function System (D-KEFS) (Delis et al., Reference Delis, Kaplan and Kramer2001), Brief Visuospatial Memory Test – Revised (BVMT-R) (Benedict, Reference Benedict1997), and the California Verbal Learning Test-II (CVLT-II) (Delis et al., Reference Delis, Kramer, Kaplan and Ober2000).
Data analysis
Scores on the MSNQ were the sum of the 15-item responses on the self-report and informant-report separately, thus resulting in scores for each (MSNQ-S and MSNQ-I). Possible scores range from 0–60, with higher scores indicating a greater degree of reported cognitive impairment. Both the MSNQ-S and MSNQ-I were normally distributed.
Calculation of standard scores and composite scores
Previous research has demonstrated evidence for sex differences in cognitive functioning in PwMS (Beatty & Aupperle, Reference Beatty and Aupperle2002; Donaldson et al., Reference Donaldson, Patel, Shammi and Feinstein2019). Therefore, we first examined potential differences between females and males on neuropsychological tests, which revealed significant sex differences on several of the measures. Consequently, within-sex standard scores were created for all neuropsychological tests, with a mean of 100 and a standard deviation of 15. We used the sample mean and standard deviation for PwMS within our sample, rather than healthy controls. This was predicated upon our interest in examining the utility of the MSNQ specifically within an MS sample instead of making a comparison with healthy controls. Scores were created such that higher scores always indicated better performance.
We next used a multi-step process to create composite standard scores for neurocognitive domains. First, principal component analyses (PCA) were conducted to identify conceptually related neuropsychological test variables. PCA results revealed five distinct components that were conceptualized as follows: focused attention, processing speed, executive function, visual memory, and verbal memory. Focused attention included Digit Span Forward and Digit Span Backward, which loaded at .78 and above. Processing speed included Digit Symbol Substitution Test – Copy Test Condition total number correct and number of items correct per second, COWAT trials 1–3 total score, and Animal Naming total score, all of which loaded above .47. Executive function included the PASAT, D-KEFS Sorting Test total number of correct sorts and correct sorts per second, and the Oral SDMT, all of which loaded above .49. Visual memory included the BVMT-R immediate and delayed recall, which loaded above .80. Verbal memory included the CVLT-II total immediate recall, trial B immediate recall, short delay free recall, short delay cued recall, long delay free recall, and long delay cued recall, all of which loaded above .60. Since all test variables entered into the PCA sufficiently loaded onto one of the five domains, none were removed from further analyses. Following the PCA, the final composite scores were calculated by first creating standard scores from the individual neuropsychological tests, and then by calculating a mean standard score value for each composite. This approach is comparable to previously published work with PwMS (Riegler et al., Reference Riegler, Cadden, Guty, Bruce and Arnett2021) as well as other populations (Guty & Arnett, Reference Guty and Arnett2018; Riegler et al., Reference Riegler, Guty and Arnett2019; Thomas et al., Reference Thomas, Guty, Riegler and Arnett2021).
Calculation of variability scores
Regarding IIV, we selected dispersion as an indicator of IIV as this construct has garnered extensive research in the neuropsychological literature. Dispersion refers to the variability observed across tasks administered in a single test session (Hultsch et al., Reference Hultsch, MacDonald and Dixon2002). For the purpose of the present study, two measures of IIV were calculated to examine dispersion: (1) an average standard deviation score, often referred to as “intraindividual standard deviations” or ISD, which has been used in previous research (Hilborn et al., Reference Hilborn, Strauss, Hultsch and Hunter2009; Morgan et al., Reference Morgan, Woods, Delano-Wood, Bondi and Grant2011; Rabinowitz & Arnett, Reference Rabinowitz and Arnett2013); and (2) a maximum discrepancy (MD), or range, score (Schretlen et al., Reference Schretlen, Munro, Anthony and Pearlson2003). After standardization of the neuropsychological test scores, a global ISD score was calculated for each participant by averaging the standard deviations associated with each of the 18 individual neurocognitive test variables. A higher ISD score is associated with greater dispersion across measures, whereas a lower ISD score is associated with greater uniformity across measures. The global MD score was calculated by subtracting each participant’s lowest test score from their highest score following the standardization of scores. Similar to the ISD score, a higher MD score indicates greater dispersion of scores while a lower MD score indicates greater consistency. This approach is comparable to previous IIV research (Arce Rentería et al., Reference Arce Rentería, Byrd, Coulehan, Miranda, Fuentes, Rosario, Morris and Mindt2020; Merritt et al., Reference Merritt, Clark, Crocker, Sorg, Werhane, Bondi, Schiehser and Delano-Wood2018; Riegler et al., Reference Riegler, Cadden, Guty, Bruce and Arnett2021). Information regarding key variables, including the MSNQ, objective neurocognitive performance, and IIV, is shown in Table 2.
Table 2. Key variables
Note. IIV = intraindividual variability; ISD = intraindividual standard deviation; MD = maximum discrepancy; MSNQ-S = multiple sclerosis neuropsychological screening questionnaire self-report; MSNQ-I = multiple sclerosis neuropsychological screening questionnaire informant report.
a Scores were standardized using means and standard deviations from within our MS sample.
b Scores were calculated by summing the 15 items on the MSNQ-S and MSNQ-I respectively.
Data analyses
To address specific aim 1, five separate linear regression analyses were first conducted with each of the mean objective neurocognitive domain scores as outcome variables and the MSNQ-S and MSNQ-I separately as predictors. In order to address specific aim 2, two separate linear regressions were conducted with each of the IIV measures (ISD and MS) as outcome measures and the MSNQ-S and MSNQ-I separately as predictors. Demographic variables (age, education, EDSS, disease duration and course) were examined as potential covariates via linear regression and were included if they significantly predicted any of the outcome measures listed above. Of note, participants were only included if they had complete information for both the MSNQ-S and MSNQ-I. This decision was predicated upon our interest in evaluating potential discrepancies between self- and informant-report in regard to subjective neurocognitive functioning (n = 4). Outliers were defined as those whose studentized deleted residual was > 4 and were subsequently removed from analysis in the specific model in which their results were significantly skewed. Statistical significance was characterized by p < .05.
Results
MSNQ-S
Mean neurocognitive performance
Regression analyses demonstrated that higher scores on the MSNQ-S (i.e., worse subjective cognitive functioning) predicted worse performance on measures of Executive Function, F(2,82) = 7.26, p = .01, Visual Memory, F(2,81) = 6.81, p = .01, and Verbal Memory, F(1,84) = 6.39, p = .01. The MSNQ-S did not significant predict measures of Focused Attention or Processing Speed (Table 3).
Table 3. MSNQ-S and mean neurocognitive performance results
Note. MSNQ-S = multiple sclerosis neuropsychological screening questionnaire self-report; EDSS = expanded disability status scale.
Intraindividual variability
Regression analyses showed that the MSNQ-S did not significantly predict the ISD or MD (Table 4).
Table 4. MSNQ-S and MSNQ-I and measures of IIV
Note. IIV = intraindividual variability; ISD = intraindividual standard deviation; MSNQ-S = multiple sclerosis neuropsychological screening questionnaire self-report; MSNQ-I = multiple sclerosis neuropsychological screening questionnaire informant report; EDSS = expanded disability status scale.
MSNQ-I
Mean neurocognitive performance
Similar to the MSNQ-S, regression analyses demonstrated that higher scores on the MSNQ-I (i.e., worse perceived cognitive functioning by informants) predicted worse performance on measures of Executive Function, F(2,82) = 9.85, p = .002, and Visual Memory, F(2,81) = 9.85, p = .002. The MSNQ-I did not significantly predict performance on measures of Focused Attention, Processing Speed, or Verbal Memory (Table 5).
Table 5. MSNQ-I and mean neurocognitive performance results
Note. MSNQ-I = Multiple sclerosis neuropsychological screening questionnaire informant report; EDSS = expanded disability status scale.
Intraindividual variability
Regression analyses showed that higher scores on the MSNQ-I significantly predicted greater dispersion of scores on the ISD, F(3,82) = 4.87, p = .03. However, the MSNQ-I did not significantly predict MD scores (Table 4).
Discussion
This study sought to further elucidate potential associations between subjective report and objective performance with the goal of examining the efficacy of available screeners for neurocognitive impairment in PwMS – particularly as it relates to differences between self- and informant-report. Given that comprehensive neurocognitive testing may be costly, time-consuming, or difficult for many people to access, it is important that we select screeners with utility to better inform decisions and referrals for comprehensive testing. It is also important that we understand potential differences between self- and informant-report when selecting screeners, as not every individual will have an informant who is able to speak to their cognitive functioning. As such, the current study aimed to further explore and gain clarity on the relationship between the MSNQ-S, MSNQ-I, and objective neurocognitive performance within specific cognitive domains. Additionally, this study aimed to fill current gaps in the MS literature by examining the relationships between the MSNQ-S and MSNQ-I and IIV in performance across a neurocognitive battery.
Our findings partially supported hypothesis 1 in that the higher scores on the MSNQ-I (i.e., worse perceived cognitive functioning by informants) predicted worse performance on the composites of Executive Function and Verbal Memory; the MSNQ-I did not significant predict performance on measures of Focused Attention. Inconsistent with predictions, we found that the MSNQ-I did not predict performance on measures of Processing Speed and Visual Memory. These findings partially support previous research demonstrating the utility of the MSNQ-I in identifying cognitive impairment. Regarding the MSNQ-S, we were neutral in terms of specific predictions given mixed findings on the literature on the relationship between subjective cognitive reports in PwMS and objective neurocognitive performance. Regarding self-reported cognitive functioning, we found that higher scores on the MSNQ-S (i.e., worse perceived neurocognitive performance) significantly predicted performance on measures of Executive Function, Verbal Memory, and Visual Memory. These findings are consistent with three published studies (Benedict & Zivadinov, Reference Benedict and Zivadinov2006; Nauta et al., Reference Nauta, Balk, Sonder, Hulst, Uitdehaag, Fasotti and de Jong2019; and Randolph et al., Reference Randolph, Arnett and Higginson2001), but inconsistent with three other prior studies that reported null results (Benedict et al., Reference Benedict, Munschauer, Linn, Miller, Murphy, Foley and Jacobs2003a; Middleton et al., Reference Middleton, Denney, Lynch and Parmenter2006 and O’Brien et al., Reference O’Brien, Gaudino-Goering, Shawaryn, Komaroff, Moore and DeLuca2007) What might account for these disparate findings? One possible explanation is that the sample for the current study demonstrated higher levels of disability compared to other samples. For example, the mean EDSS for the current sample was 4.32, whereas the mean EDSS for Benedict et al. (Reference Benedict, Munschauer, Linn, Miller, Murphy, Foley and Jacobs2003a) and O’Brien et al. (Reference O’Brien, Gaudino-Goering, Shawaryn, Komaroff, Moore and DeLuca2007) were 3.5 and 3.7, respectively. Higher rates of disability may be reflective of more pronounced difficulties, which may in turn result in higher reports and greater variability of cognitive dysfunction and worse neurocognitive functioning. Future work might benefit from examining EDSS as a potential mediator or moderator between the MSNQ-S and objective neuropsychological performance. This said, these findings suggest that the MSNQ-S may be sensitive to underlying cognitive difficulties – particularly within domains of executive functioning and memory.
Regarding IIV, we did not find that the MSNQ-S significantly predicted either the ISD or MD. In contrast, higher scores on the MSNQ-I (i.e., worse perceived cognitive functioning by informants) significantly predicted greater dispersion of neurocognitive scores as measured by the ISD. These findings suggest that informants may be better able to report on cognitive variability in a way that may not be accessible to those with MS. Still, the maximum discrepancy score was not significantly predicted by either the MSNQ-S or MSNQ-I, and so these measures of IIV may not be particularly helpful in identifying potential cognitive strengths and weakness. Further investigation of the utility of IIV may go beyond evaluating global variability to also include examining variability within specific domains (i.e., Attention IIV, Processing Speed IIV, Executive Function IIV, and Memory IIV). It should be noted that these domain IIV scores were not examined in this study given discrepancies in the number of neuropsychological indices included within each domain, as determined by PCA.
Taken together, these results suggest that both the MSNQ-S and MSNQ-I show similar utility and demonstrate value in predicting objective neuropsychological deficits in several domains, with consistency noted in domains of executive functioning and verbal memory. Thus, these results suggest that verbal memory and executive function deficits may be more salient for individuals, as well as informants. A limitation in making this conclusion stems from the skew in content of items on the MSNQ toward difficulties in executive function and memory, thus leading to a nonunitary evaluation of cognitive functioning compared to what is examined with comprehensive, objective neurocognitive testing. Nevertheless, when considering that the items included within the MSNQ relate to the domains that are typically most impacted by MS, this may not be a true limitation provided that the MSNQ is intended to screen for potential neurocognitive impairment in order to determine referrals for more extensive follow-up neurocognitive testing. Notably, there are other screeners often used to examine cognitive functioning in PwMS, such as the SDMT. While the SDMT has been shown to demonstrate value in detecting cognitive dysfunction in PwMS (Arnett et al., Reference Arnett, Cadden, Roman, Guty, Riegler and Thomas2021; Deloire et al., Reference Deloire, Bonnet, Salort, Arimone, Boudineau, Petry and Brochet2006; Sonder et al., Reference Sonder, Burggraaff, Knol, Polman and Uitdehaag2014), the use of a self-administered screener like the MSNQ may be advantageous as it can be administered by a wider variety of providers rather than only those with specialized training in neuropsychology. Moreover, the SDMT is only one measure of cognitive functioning which, while accurate, is a gross evaluation whereas the MSNQ addresses several domains of cognitive functioning.
Notably, the MSNQ-S and MSNQ-I were moderately correlated in our sample, r(87) = .60, which is important for clinical considerations as not all PwMS may have an informant who can attend appointments with them. Consistent with findings by Benedict and colleagues (2003a, 2006), we found that the MSNQ-S, but not the MSNQ-I, was correlated with depression, with a moderate effect size in our sample, r(87) = .39. Future work should further examine the role of depression, and other mood symptoms seen in MS, on both subjective and objective neurocognitive functioning.
There are several limitations to this study. First, our sample consists of predominantly White, well-educated individuals who are local to central Pennsylvania. Thus, the findings may not replicate in more heterogeneous samples, especially in areas that are less rural. Therefore, future work should include recruitment of diverse samples in order to more comprehensively examine the utility of the MSNQ in predicting objective neurocognitive performance. Relatedly, this study relied upon a community-based sample that is likely to differ from a clinic-based sample. For example, previous work by Benedict et al. (Reference Benedict, Munschauer, Linn, Miller, Murphy, Foley and Jacobs2003a) and O’Brien et al. (Reference O’Brien, Gaudino-Goering, Shawaryn, Komaroff, Moore and DeLuca2007) used clinic-based samples which may help to explain, in part, our disparate findings. This said, clinic-based samples are typically thought to show greater degrees of impairment and disability, while our sample actually demonstrated higher scores of disability on the EDSS compared to the two aforementioned studies. Regardless, more information will be needed regarding domain-specific performance and IIV within clinic-based samples as this may be beneficial in helping to identify those who are at greater risk for cognitive impairment on cognitive screeners and thus warrant further neuropsychological assessment. Lastly, our examination of IIV is limited to only examining variability within performance on a single test battery given the cross-sectional design of this study. While this approach is clinically valuable, as most neuropsychological evaluations are typically completed within one testing session, future work would likely benefit from evaluating variability over time. For example, future studies could implement a longitudinal approach for examining performance variability and subjective assessment of cognitive functioning, particularly as it may relate to mood and/or disease progression. Additionally, given the prevalence of secondary factors (e.g., mood, fatigue, sleep disturbance) that may contribute to cognitive impairment in PwMS (Bamer et al., Reference Bamer, Johnson, Amtmann and Kraft2008; Bruce et al., Reference Bruce, Bruce and Arnett2010; Krupp et al., Reference Krupp, Serafin and Christodoulou2010; Pokryszko-Dragan et al., Reference Pokryszko-Dragan, Zagrajek, Slotwinski, Bilinska, Gruszka and Podemski2016), future work should examine the potential impact of these factors on subjective and objective evaluations of cognitive functioning.
Overall, these findings indicate that the MSNQ-S and MSNQ-I are moderately correlated and demonstrate utility as good neurocognitive screeners, though formal neuropsychological testing, including mood and fatigue screeners, is likely still warranted.
Acknowledgements
The authors thank the participants who were involved in this project as well as the many neurologists in the Pennsylvania region who contributed their time to verifying MS diagnoses and ratings.
Authors contribution
GT conceptualized the study, completed the literature review, wrote sections of the manuscript, helped design and supervise the construction of the database, conceptualized and conducted the statistical analyses, and agrees to be accountable for the content of the work. KR helped design and supervise the construction of the database, and wrote sections of the manuscript, and agrees to be accountable for the content of the work. MB helped design and supervise the construction of the database, and wrote sections of the manuscript, and agrees to be accountable for the content of the work. DO helped to conceptualize the study, wrote sections of the manuscript, helped design the database, supervised the construction of the database, helped to conceptualize the statistical analyses, and agrees to be accountable for the content of the work. PA helped to conceptualize the study, wrote sections of the manuscript, helped design the database, supervised the construction of the database, provided external funding for the study, and agrees to be accountable for the content of the work.
Funding statement
This investigation was supported (in part) by grants to the last author from the National Multiple Sclerosis Society (PP0978 and PP1829).
Conflicts of interest
Peter Arnett, Ph.D. has served on the EMD Serono – Speakers’ Bureau, and served as a consultant for Biogen and Roche Pharmaceuticals.
Open access
