Introduction

The increased distress or anxiety related to repeated or excessive searches online for health-related information is defined as “cyberchondria” (CYB) [1, 2]. CYB is considered an abnormal behaviour pattern, rather than a diagnostic entity or condition [3], and it is widespread among people with high levels of “health anxiety” (HA) [4], which is defined as a state of excessive, unrealistic and persistent worry regarding the suffering of disease [5]. In the absence of significant anxiety, searching for health information online could be a precursor to increased HA [6]; or rather, HA could be a top motivator for searching health-related information online [7]. In 2013, a representative survey in the US reported that 35% of people used the Internet to self-diagnose a medical condition [8]. In 2015, a study reported that 70% of adults used the Internet as their first source for searching health-related information [9].

High levels of HA associated with increased access to health-related information on the Internet can carry considerable repercussions. For example, it can affect work and social development since too much time is spent investigating possible diseases [10]. Furthermore, it predisposes the individual to seek medical attention, undergoes unnecessary exams, wastes medical resources, and creates exposure to possible adverse effects when self-prescription occurs [10–12]. Persons with HA can repeat this behaviour in many opportunities [10, 13] and increase the risk of suicide [14].

The Cyberchondria Severity Scale (CSS) is a self-report instrument designed to facilitate this construct’s multidimensional measure [15]. The CSS consists of 33 items distributed in 5 domains: Excessiveness (escalating/repeated nature of searches), Distress (negative emotional response), Reassurance (searching drives individuals to seek out professional medical advice), Compulsion (web searches interfering with other aspects of on/offline life) and Mistrust (conflict arising when medical professional and online self-diagnosis do not align). Created in English, the CSS has been adapted to different languages such as Turkish [16], German [17] and Portuguese (Brazil) [18]. Their psychometric properties have been reviewed in posterior studies [19–22], arguing differences regarding the more plausible measurement model. In particular, an independent study suggested a Bifactor model for the CSS (General CYB factor, which is orthogonal to its subfactors), concluding that Mistrust should be removed from the poll of domains [20].

Based on this evidence, the original authors developed a short version of the CSS -known as the Cyberchondria Severity Scale– 12 (CSS-12)–to overcome the limitations reported by others [23]. In the CSS-12, the domain Mistrust was eliminated because of its theoretical ambiguity, lack of load in the general CYB factor and low correlation with other domains [19, 20]. Besides, this version only considered three optimal items for each domain due to criticism about the extension of the original version and regarding items that could be omitted because they are not relevant or specific [3, 17]. Thus, the CCS-12 assesses CYB through 4 domains: Excessiveness, Distress, Reassurance and Compulsion, a structure validated through a Bifactor model [23].

Although the Bifactor model has demonstrated adequate psychometric properties, testing a Bifactor-ESEM model is relevant given the intriguing patterns within the cyberchondria [15]. For example, certain Excessiveness items have exhibited a tendency to shift into the Distress and Reassurance dimensions [17]. Items such as Item 5, which did not load within its designated dimension during the original CSS-12 validation [23], raise questions about a potential shared loading with other CSS-12 dimensions.

The CSS-12 is a more robust version to evaluate CYB, but it is not available in Spanish. Indeed, there are no instruments for assessing CYB in Spanish-speakers, partially explaining the absence of research about this critical topic in Latin America. Our objective was to cross-culturally adapt and validate the CSS-12 scale for Peruvian Spanish speakers. We also wanted to determine whether the Bifactor model works as well in our population as in previous studies’ and to explore whether the Bifactor-ESEM is a more suitable model.

Methods

We developed this study in two steps: (a) adaptation to Spanish and (b) psychometric validation. The latter included evaluating the adapted CSS-12 regarding reliability, structure validity and external validity against an independent measure of anxiety.

Results

(b) Psychometric validation

CSS-12 scores.

The CSS-12 total score average was 25.1 (SD 9.1). The average scores for each of the four dimensions were: Excessiveness = 7.8 (S.D. 2.8); Distress = 5.7 (S.D. 2.6); Reassurance = 5.8 (S.D. 2.6); and Compulsion = 5.8 (S.D. 2.5). Item mean ranged from 1.6 (SD 0.9) to 3.0 (SD 1.1), and item-total correlations ranged from 0.58 to 0.76. Item mean, standard deviation, item-total correlations and Cronbach’s alpha if removed are presented in Table 1. Polychoric correlation matrix between items of the CSS-12 is available in S1 Table.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Means (M), standard deviations (SD), item-total correlations, item-rest correlations, and α if removed of CSS-12 items (N = 657).

https://doi.org/10.1371/journal.pone.0292459.t001

Reliability.

The CSS-12 total score demonstrated excellent internal consistency (Cronbach’s alpha = 0.93; McDonald’s omega = 0.93), and internal consistency values were in the adequate–very good range for the subscales with Cronbach’s alpha (Cronbach’s alpha = 0.79–0.83; Table 2) and were very good with McDonald’s omega (Excessiveness: McDonald’s omega = 0.81; Distress: McDonald’s omega = 0.83; Reassurance: McDonald’s omega = 0.81; Compulsion: McDonald’s omega = 0.80).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Reliability and convergent validity of the CSS-12.

https://doi.org/10.1371/journal.pone.0292459.t002

The Pearson’s correlation coefficients calculated from the test-retest for both the items and domains showed very high reliability, with all values being positive and >0.80 (Tables 2 and 3). The global intraclass correlation also revealed very high reliability for the instrument (ICC = 0.93) and domains (ICC = 0.80–0.89; Table 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. CSS-12 items and dimension to which they belong, mean and standard deviation (SD) of each item, and Pearson’s r for test-retest.

https://doi.org/10.1371/journal.pone.0292459.t003

Validity.

Concerning construct validity, fit statistics for one-factor CFA, first-order four-factor CFA, Bifactor, and Bifactor-ESEM models of the CSS-12 are presented in Table 4. AIC and BIC values progressively improved from the most parsimonious one-factor model (Model 1) to the most complex Bifactor-ESEM model (Model 4). CFI, TLI, RMSEA and SRMR also improved their values in a similar way. This empirical evidence supports the most complex Bifactor-ESEM solution. The general CYB factor of the Bifactor-ESEM explains a considerable fraction of the shared variance (ΩH = 0.92; ECV = 0.79), allowing to sum the 12 items for obtaining a total score. Standardized factor loadings for this model are visible in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Goodness of fit of the four CSS-12 measurement models evaluated (N = 657).

https://doi.org/10.1371/journal.pone.0292459.t004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Standardized factor loadings and internal consistency for the Bifactor-ESEM model (N = 657).

https://doi.org/10.1371/journal.pone.0292459.t005

Regarding convergent validity, the correlation between CSS-12 and SHAI was positive and moderate (Pearson’s r = 0.51), showing that some patients with health anxiety are likely to have cyberchondria as well. Weak to moderate correlation was evident when evaluating each subscale with SHAI (Pearson’s r = 0.41–0.51; Table 2).

Discussion

The CSS-12 was successfully adapted to Spanish. It showed excellent reliability for the general CYB scale and good reliability for its subscales. The Bifactor-ESEM model with one general and four specific dimensions showed the best fit and strong one-dimensionality for the general CYB scale. The CSS-12 was positively correlated with the SHAI measure, confirming convergent validity against health anxiety. We are providing a formatted and free-to-use version of the Spanish CSS-12 in the supplemental material.

In general, the Spanish version of the CSS-12 has shown good psychometric properties like the original English version [23]. For example, general reliability is high for both the Peruvian and English populations (all college students), suggesting that CSS-12 internal consistency can be stable across populations. For the CSS-12 dimensions, Cronbach’s alpha remained adequate-to-good in the Spanish compared to the English version [23]. Likewise, the Spanish version is equally reliable than the full version of the CSS [16, 19]. For the Spanish version, we proposed and reported the McDonald’s omega, a reliability measure more accurate and with more feasible assumptions than Cronbach’s alpha [28, 49]. Although the latter is the most reported in previous CSS revisions, our findings showed that the true reliability could have been underestimated in those studies. Evidence from McDonald’s omega estimates and test-retest results of our study support the conclusion that all the CSS-12 dimensions have good reliability (>.80), including the Reassurance dimension (which reported just adequate reliability from Cronbach’s alpha).

A Bifactor solution for the CSS-12 has been largely justified in the literature. Bifactor models can capture common and specific variances separately, helping to understand the construct complexity more clearly [50]. For example, although REAS items are just moderately associated with this dimension, they are a strong representation of the general construct “cyberchondria” (CYB). This distinction cannot be recognised when other less complex models are fitted; for example, a first-order four-dimensional model. This advantage was early identified by the CSS-12 creators [23] and confirmed later by other independent researchers [51] providing support for using the general CYB score [52].

During our study, we explored the applicability of the Bifactor-ESEM model, a measurement model not previously evaluated for the CSS-12. This model represents a step forward towards the improvement of the cyberchondria measurement framework with the CSS. First, some studies conceptualized the full CSS as containing a general CYB factor orthogonal to its subfactors [20, 22]. Then, a more recent study developed the CSS-12, a shorter version of the original CSS, for which this Bifactor model was still the best measurement model for assessing cyberchondria [23]. Nevertheless, at this point, some items that can be substantively connected to more than one specific factor were assumed to be an expression of only one of them. In a Bifactor-ESEM model, there are no zero values pre-set; instead, these parameters are estimated under the assumption that some items load higher on one specific factor and lower on the others (but not necessarily zero). This allows the main estimates to be more accurate [32] and realistic in terms of what is happening with some items that express various facets of the cyberchondria simultaneously.

Prior research has revealed intriguing patterns within the cyberchondria. For example, certain EXC items have been observed to transition into the DIST and REAS dimensions [17]. Considering the conceptual framework of these subscales, the link to DIST can be elucidated by the notion that individuals with erroneous health beliefs may experience heightened distress when interpreting search results. This heightened distress might subsequently drive the inclination for excessive health-related information searches and the quest for reassurance through medical consultations. Similarly, the reclassification of an item from REAS to DIST may be attributed to the relationship between encountering worrisome information online and acting by visiting a healthcare provider [17]. In our study, the CSS-12’s item 2 was initially assigned to the COMP dimension only; however, it is substantively connected with item 1 of the EXC dimension, as our findings have demonstrated. We found that item 2 has a factor loading different from zero for EXC, which may be due to excessive time spent on online searches that interfere with other activities. A similar non-zero loading with more than one specific factor was confirmed for six of the twelve items.

Although the evidence shows the Bifactor model as one of the most promising, this cyberchondria measurement model is not perfect yet. In the Spanish CSS-12, the REAS dimension did not show a clear link with its items, conversely to the strong connection confirmed between the other items and their factors. Nevertheless, this problem is not new since the original English CSS-12 also showed a weak connection between item 5 and the REAS factor [23]. In the future, other cross-cultural revisions of the CSS-12 will enlighten this issue, helping to update the REAS factor measurement (e.g., readjusting one or more items). So far, the practical recommendation made by the CSS-12 authors -use and interpret results from the general CYB scale only–[23] is entirely valid and reliable for the Spanish version as well.

The Spanish and English versions of the CSS-12 have a similar convergent validity with the health anxiety (HA). Convergent validity between the English CSS-12 and the SHAI was previously reported [23], showing a positive correlation with a strength similar to the one we found for the Spanish version. Considering how easy it is to get access to health information online nowadays, it is likely that people with high levels of HA tend to use the Internet to obtain information about it [53] more frequently [54–56] and for longer [57]. This behaviour can trigger an increase in CYB level and sustain or increase the HA level itself [6]. During the creation of the CSS, authors have emphasized that, although CYB and HA are connected, they are not the same construct [7, 22, 23, 58]. For example, those with no prior HA may also experience stress because of searching about a health condition on the Internet [3]. For McElroy et al. [23], a critical factor that differentiates CYB from HA is EXC, which means that people spend an increasing amount of time searching about a health condition on the Internet, which affects their everyday lives [53]. This difference and the focus of the CSS-12 items on the Internet context justify its use as a CYB measure clearly differentiated from HA. Nevertheless, given that CYB is a relatively new construct, more studies are needed to further clarify this association and its directionality.

We have identified some limitations and strengths in our study. Given that the CSS-12 tool was only validated in students from a private Peruvian college, the external validity is limited. Although they are healthy in general, this population is commonly linked to “transient hypochondriasis”, which increases the levels of HA that usually occurs in medical students [59]. This could mark a distinction with the general population, depending on the academic year of study [60]. There were no previously validated tools to quantify CYB in the Spanish language, so this validation is the first to be carried out. This adapted and validated tool is short, practical and novel in the Latino and Spanish context, so its simple application will be valuable to expand knowledge regarding CYB. Although Peruvian Spanish is very similar to other Central and South American Countries’ Spanish (i.e., formal written Spanish, as in this CSS-12 version), some minor language differences could remain. As international guides state [61], it is at the user’s discretion to decide how appropriate this CSS-12 version is for evaluating local Spanish speakers.

In conclusion, the Spanish version of the 12-items Cyberchondria Severity Scale (CSS-12) was successfully adapted for the Peruvian college students’ population. It provides a valid and reliable unidimensional measure of the cyberchondria, which is distinguishable from the more general health anxiety. The Bifactor-ESEM model seems to offer a more accurate and realistic representation of the multifaceted nature of cyberchondria, contributing to a comprehensive understanding of this phenomenon. The CSS-12 can be applied for evaluating cyberchondria in similar populations and future research.

Supporting information

Acknowledgments

We want to thank all the participants of the study as well as their academic institution, Universidad Peruana de Ciencias Aplicadas, for their invaluable cooperation. We also want to thank Prof McElroy and collaborators for permitting us to use the CSS-12, as well as for facilitating unpublished information from their original study that was essential to complete this cross-cultural adaptation. JCB was supported by the National Institute for Health and Care Research ARC North Thames. The views expressed in this publication are those of the authors and not necessarily those of the National Institute for Health and Care Research or the Department of Health and Social Care.