Research paper
Psychometric properties and validation of the Spanish versions of the overall anxiety and depression severity and impairment scales

https://doi.org/10.1016/j.jad.2019.03.063Get rights and content

Highlights

  • This is the first study to explore the psychometric properties of the OASIS and ODSIS, screening tools for anxiety and mood disorders, in a Spanish sample of patients diagnosed with emotional disorders in a specialized mental health setting.

  • The OASIS and ODSIS are easily administered screening tools with an excellent internal consistency, favorable convergent and discriminant validity, and good sensitivity to therapeutic change.

  • A cut-off point of 10 for both instruments results in the best rate of correct clinical/nonclinical classifications.

  • The Spanish adaptations of the OASIS and the ODSIS demonstrated their cross-cultural utility.

  • The OASIS and ODSIS should make screening of anxiety and depressive symptoms in public health settings more feasible and could facilitate adequate referrals and routine evaluation.

Abstract

Background

Anxiety and depressive disorders are the most frequent disorders for which patients seek care in public health settings in Spain. This study aimed at validating the Overall Anxiety Severity and Impairment Scale (OASIS) and the Overall Depression Severity and Impairment Scale (ODSIS), which are brief screening scales for anxiety and depression consisting of only five items each.

Methods

The study was conducted in a Spanish clinical sample receiving outpatient mental health treatment (N = 339). A subsample of participants (n = 219) was assessed before and after receiving a course of cognitive-behavioral treatment.

Results

The results revealed excellent internal consistency estimates (Cronbach's alpha for the OASIS and the ODSIS was 0.87 and 0.94, respectively), along with promising convergent and discriminant validity and test-criterion relationships (i.e., moderate correlation with other measures of depression and anxiety, as well as with neuroticism, quality of life, adjustment, and negative affect). A one-dimensional structure was obtained for the OASIS and the ODSIS. The ROC analyses indicated an area under the curve of 0.83 for the OASIS and the ODSIS when predicting moderate-to-severe anxiety and depression, respectively. Good sensitivity to therapeutic change was also evidence and the analysis of the sensitivity as a function of 1-specificity area suggested a cutoff value of 10 for both scales.

Limitations

Inter-rater reliability of diagnoses with the ADIS-IV interview could not be investigated and the results obtained may not be generalizable to other samples and health settings.

Conclusions

The availability of these two short and psychometrically sound measures should make screening of anxiety and depressive symptoms in routine care more feasible.

Introduction

Anxiety and depressive disorders, also known as emotional disorders (EDs; Barlow, 1991) are common conditions that negatively impact functioning (Campbell-Sills et al., 2009). According to the World Health Organization (WHO), in Spain alone, anxiety and depressive disorders affect nearly 2 million and 2.5 million people, respectively, which amount to up to 5.2% of the population in the country (WHO, 2017). Not surprisingly, EDs are largely prevalent in primary care and specialized mental health settings in Spain (Gutiérrez-Fraile et al., 2011; Muñoz-Navarro et al., 2017, Ruiz-Rodríguez et al., 2017). The present study aims at adapting two short screening tools for anxiety and depressive symptoms to be used in Spanish public care.

Several authors have indicated that diagnosis and monitoring of patients with EDs in Spanish public health settings is problematic. For example, a study by Castro-Rodríguez et al. (2015) suggested that half of the patients with an ED are not correctly diagnosed in primary care. Consequently, a large number of individuals with ED are not properly referred to specialized care and do not receive the most optimal treatment (Martín Jurado et al., 2012). While there might be several factors explaining this, some authors have attributed the previous to healthcare overload, time constraints during consultations, and family physicians’ insufficient familiarity with the assessment and treatment of psychiatric disorders (Muñoz-Navarro et al., 2017). It is important to note that it is at this first level of care where correct identification of those who need specialized interventions is of paramount importance, so quick and straightforward assessment tools for an early detection of EDs are fundamental (Cano Vindel et al., 2012). A similar problem occurs in specialized mental health settings in Spain, where limited resources (e.g., reduced number of professionals, especially psychologists) has shown to negatively impact the quality of services provided (Gabilondo et al., 2011). An example of this is the long waiting lists that currently exist in Spanish public mental health settings, that is, from 45 to up to 75 days between appointments (Chueca et al., 2003, González et al., 2008). A quick and feasible assessment tool for the evaluation of anxiety and depressive symptoms to be used in routine care would benefit psychologists and psychiatrists by reducing the time they require for the assessment of the patients’ progress through treatment.

Many self-report screening measures have been designed to assess specific diagnostic categories (e.g., Panic Disorder Severity Scale [PDSS]; Shear et al., 2001). Although these scales are useful to ascertain whether an individual is likely to meet criteria for a specific disorder, the administration of a large number of screening measures (e.g., one for each ED diagnosis) is not feasible in public health settings. Thus, transdiagnostic measures that capture the spectrum of ED presentations may be particularly useful in such settings (Campbell-Sills et al., 2009). Furthermore, traditional self-report measures typically assume that the frequency of symptoms reflects the severity of the disorder, while ignoring the key role of functional impairment or distress for diagnosis (Ito et al., 2014).

The Overall Anxiety Severity and Impairment Scale (OASIS; Norman et al., 2006) and the Overall Depression Severity and Impairment Scale (ODSIS; Bentley et al., 2014) were developed in response to the limitations of existing screening tools. Both instruments were designed to assess transdiagnostic symptoms and functioning associated with anxiety and depressive disorders, respectively. The OASIS and the ODSIS have five items each and can be administered in only 2–3 min. Both measures have been shown to be useful in the assessment of patients’ progress through treatment (Barlow et al., 2017, Norman et al., 2013, Osma et al., 2015). The OASIS has been validated into English (Norman et al., 2006), Japanese (Ito et al., 2014), and Dutch (Hermans et al., 2014). An online version of this measure has also been recently been validated into Spanish in a sample of individuals with milder depression and anxiety (González-Robles et al., 2018). A single-factor model including all items has been replicated across investigations (Bragdon et al., 2016, Campbell-Sills et al., 2009, González-Robles et al., 2018; Hermans et al., 2014, Ito et al., 2014, Moore et al., 2015, Norman et al., 2006, Norman et al., 2011, Norman et al., 2013) and a good internal consistency (i.e., between 0.80 and 0.94) has been reported (Hermans et al., 2014, Ito et al., 2014, Norman et al., 2006).

The cut-off points for the OASIS have ranged from 5 to 10 in clinical samples (Bragdon et al., 2016, Campbell-Sills et al., 2009, González-Robles et al., 2018, Hermans et al., 2014, Moore et al., 2015, Norman et al., 2013) and from 8 to 9 in studies with clinical and non-clinical populations altogether (Ito et al., 2014, Norman et al., 2011).

Similar to the OASIS, the ODSIS has been validated into English (Bentley et al., 2014) and Japanese (Ito et al., 2015). A single-factor solution has been proposed for the ODSIS and internal consistency findings have been excellent (Bentley et al., 2014, Ito et al., 2015). The cutoff points for the ODSIS have ranged from between 7 and 11 with clinical and non-clinical samples altogether (Bentley et al., 2014, Ito et al., 2015).

The present study goal was to replicate and extend prior research on the psychometric properties of the OASIS and the ODSIS in a Spanish outpatient sample of mental health patients with EDs. Specifically, we aimed at replicating the findings of the validations conducted in the USA, the Netherlands, Spain, and Japan, thus providing further evidence for the transcultural validity of these scales. The study also had a novel goal, that is, to determine the sensitivity of both measures to therapeutic change after a transdiagnostic, cognitive-behavioral treatment for EDs, namely, the Unified Protocol (Barlow et al., 2017; Blind note).

The following hypotheses were proposed: 1) for both scales, a one-dimensional structure will be replicated; 2) an acceptable internal consistency will be obtained; 3) convergent and discriminant validity will be established through significant positive correlations between the OASIS and the ODSIS and measures assessing similar constructs, as well as weaker correlations with measures of different constructs; 4) the ROC curve will reveal an optimal cut-off point of approximately 10 for both measures; and finally, 5) adequate sensitivity to therapeutic change will be indicated by significant and positive correlations between the change detected by the reference measurements (the Beck Anxiety Inventory and Beck Depression Inventory-II) and the change detected by the OASIS and the ODSIS.

Section snippets

Participants

Participants were drawn from a large effectiveness trial of the Unified Protocol (UP) in the Spanish public mental health system (Blind note). The sample for the present study was composed of 339 participants of whom 272 were women (80.2%). Mean age in the sample was 42.63 years (SD = 12.79, range = 18–77). Table 1 shows additional socio-demographic characteristics. The selected participants adults seeking psychological assistance in the national health system. The sample was obtained in 8

Demographic and sex differences in study variables

Analyses were carried out using the statistical package IBM SPSS Statistics version 22.0 for Windows (IBM Corp., 2013) and Mplus version 6 program (Muthen and Muthen, 1998). Firstly, sociodemographic characteristics of the total sample were analyzed (N = 339), as well as the means and standard deviations for all measures. Next, sex differences in the OASIS and the ODSIS were investigated. Because scores in men and women were not normally-distributed (Kolmogorov–Smirnov normality test), a

Results

As observed in Table 1, the most frequent primary diagnoses in the sample (N = 339) were adjustment disorders (n = 77, 22.7%), followed by major depressive disorders (n = 62, 18.3%), and generalized anxiety disorders (n = 43, 12.7%). Ninety nine participants (26.8%) also had a secondary diagnosis, of which major depressive disorder (n = 19, 19.2%) and non-specific anxiety disorder (n = 14, 14.1%) were the most frequent.

Table 2 shows the mean scores and standard deviations for all instruments.

Discussion

This is the first study to explore the psychometric properties of the OASIS and ODSIS in a Spanish sample of patients diagnosed with ED in a specialized mental health setting, and, consequently, the contributions made are innovative and necessary.

The results regarding the internal structure of the OASIS and the ODSIS, as obtained with the CFA, are consistent with other studies and indicate that a one-factor structure for the five items has an optimal fit (Bentley et al., 2014, Bragdon et al.,

Conclusions

This study adds to the growing literature supporting the validity of the OASIS and ODSIS as screening tools for anxiety and mood disorders. It also provides data about their sensitivity to change after a psychological treatment, which makes them useful tools to be included in routine care to rapidly assess the effectiveness of interventions. Additionally, this investigation provides further evidence for the utility of both scales in a new country, namely, Spain.

The validation of the OASIS and

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This research was conducted with the approval of the ethics committee of all collaborating centers and all participants signed an informed consent prior to participation. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed

Funding

Funding for the study was provided by the PI17/00320 project integrated in Plan Estatal de I + D + I 2013–2016 and co-funded by the “ISCIII-Subdirección General de Evaluación y Fomento de la investigación del Fondo Europeo de Desarrollo Regional (FEDER). Otra manera de hacer Europa”. The research team S31_17D has also received funds by the Gobierno de Aragón (Departamento de Innovación, Investigación y Universidad), and FEDER “Construyendo Europa desde Aragón”.

Contributors

All persons who met authorship criteria are listed as authors. All authors certify that they have participated sufficiently in the work to take public responsibility for the content, including participation in the concept, design, analysis, writing, or revision of the manuscript. All authors contributed to and approved the final manuscript.

Acknowledgements

We would like to thank all collaborating centers, all healthcare professionals, and all volunteers and participants that made this study possible.

References (56)

  • Diagnostic and Statistical Manual of Mental Disorders, Author, Washington, DC

    (1994)
  • D.H. Barlow

    Disorders of emotion

    Psychol. Inq.

    (1991)
  • Barlow, D.H., Farchione, T.J., Fairholme, C.P., Ellard, K.K., Boisseau, C.L., Allen, L.B., Ehrenreich-May, J. (2015)....
  • D.H. Barlow et al.

    The unified protocol for transdiagnostic treatment of emotional disorders compared with diagnosis-specific protocols for anxiety disorders: a randomized clinical trial

    JAMA

    (2017)
  • D.H. Barlow et al.

    Unified Protocol for Transdiagnostic Treatment of Emotional Disorders

    (2018)
  • A.T. Beck et al.

    Beck Anxiety Inventory Manual

    (1993)
  • A.T. Beck et al.

    Manual for the Beck Depression Inventory-II

    (1996)
  • K.H. Bentley et al.

    Development and validation of the Overall Depression Severity and Impairment Scale

    Psychol. Assess.

    (2014)
  • D.D. Blake et al.

    The development of a clinician-administered PTSD scale

    J. Trauma Stress.

    (1995)
  • C. Botella et al.

    Trastorno de pánico, evaluación y tratamiento [Panic disorder, evaluation and treatment].Martínez Roca

    Barcelona

    (1997)
  • T.A. Brown et al.

    A proposal for a dimensional classification system based on the shared features of the DSM-IV anxiety and mood disorders: implications for assessment and treatment

    Psychol. Assess.

    (2009)
  • T.A. Brown et al.

    Anxiety Disorder Interview Schedule for DSM-IV (ADIS-IV).Adult and Lifetime version

    (1994)
  • T.A. Brown et al.

    Reliability of DSM-IV anxiety and mood disorders: implications for the classification of emotional disorders

    J. Abnorm. Psychol.

    (2001)
  • J.R. Bullis et al.

    The unified protocol for transdiagnostic treatment of emotional disorders: preliminary exploration of effectiveness for group delivery

    Behav Modif

    (2015)
  • A. Cano-Vindel et al.

    La depresión en atención primaria: prevalencia, diagnóstico y tratamiento [Depression in primary care: prevalence, diagnosis, and treatment]

    Papeles del Psicólogo

    (2012)
  • I.M. Chueca et al.

    ¿Cómo derivamos a salud mental desde atención primaria?

    Atención Primaria

    (2003)
  • Costa, P.T., McCrae, R.R., 1999. Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory...
  • L.R. Derogatis et al.

    The brief symptom inventory: an introductory report

    Psychol. Med.

    (1983)
  • Cited by (37)

    • Multidimensional emotional disorders inventory: Reliability and validity in a Spanish clinical sample

      2023, Journal of Affective Disorders
      Citation Excerpt :

      The Unified Protocol, a cognitive-behavioral intervention, have demonstrated its effectiveness applied in group format in the Spanish National Health System (Osma et al., 2021b). Although some dimensional instruments such as the Overall Anxiety Severity and Impairment Scale (OASIS; Norman et al., 2006) and the Overall Depression Severity and Impairment Scale (ODSIS; Bentley et al., 2014) scales have been adapted for use with clinical Spanish sample (Osma et al., 2019), these tools are used for the evaluation of anxiety and depression symptoms throughout the Unified Protocol intervention, there are no scales that evaluate all the constructs of the EDs in a dimensional way. This fact implies that interventions based on the hybrid approach for EDs are not dimensionally evaluated, because evidence for MEDI has been only provided in community samples so far (Osma et al., 2021a).

    • Psychometric evaluation of a caregiver-report adaptation of the Overall Anxiety Severity and Impairment Scale (OASIS) for use with youth populations

      2022, Journal of Affective Disorders
      Citation Excerpt :

      Items ask, over the past week, how frequently the individual experienced anxiety, how intense/severe anxiety has been, how often the individual engaged in anxiety-related avoidance, how much anxiety interfered with work and home life, and how much anxiety interfered with social relationships. The adult OASIS has exhibited strong psychometric properties (Bragdon et al., 2016; Campbell-Sills et al., 2009; Ito et al., 2015; Moore et al., 2015; Norman et al., 2006, 2011; Osma et al., 2019). To address the need for a similar anxiety measure for use with children and adolescents, the present study adapted the 5-item adult OASIS to yield a brief caregiver-report of past week youth anxiety and interference (i.e., the caregiver-report OASIS-Y).

    • Unified protocol application in fertility problems

      2023, Applications of the Unified Protocol in Health Conditions
    View all citing articles on Scopus
    View full text