Keywords
:COVID-19; Healthcare workers; depression; Anxiety; Stress; reliability; validity
This article is included in the Health Services gateway.
:COVID-19; Healthcare workers; depression; Anxiety; Stress; reliability; validity
COVID-19 is an infectious disease caused by SARS-CoV-2. Since it was identified and isolated in Wuhan, China, COVID-19 has quickly spread to the rest of the world, leading the World Health Organization to declare the situation a pandemic. The disease has affected 500 million people, with less than 1% mortality.
Today, COVID-19 is direly affecting people’s mental and physical health through psychological stress imposed by the disease. Studies have shown an increase in COVID-19-related psychological stress in the general population,1–3 which include depression, anxiety, stress, and post-traumatic stress.4,5 These unprecedented psychological stresses and physical risks have been more noticed among healthcare workers and their families due to COVID-19 infection. In many countries, there are considerably more COVID-19 infections among healthcare workers than among the general population.6 With healthcare workers representing less than 3% of the population, the World Health Organization reports that 14% of COVID-19 cases have been healthcare workers.
Healthcare workers are routinely exposed to stress from their occupation, which is associated with more mental health complaints than others.7,8 Besides experiencing these stresses, healthcare workers are exposed to additional stress from taking care of COVID-19-infected patients, fear of being infected and infecting family members, stigmatization, and isolation. In addition, increased COVID-19 cases and deaths, increased workload, lack of socialization and recreation, and a dearth of personal protective equipment have a tremendous psychological impact on healthcare workers.9
Healthcare workers have reported psychological stress during previous infectious diseases outbreaks such as severe acute respiratory syndrome (SARS), Middle East respiratory syndrome (MERS), H1N1 influenza, and Ebola.10–12 For example, during the 2003 SARS outbreak, studies reported a 5–10% prevalence of moderate-to-severe post-traumatic stress and an increased high risk of depression and anxiety among healthcare workers who were quarantined or frontline.3,13 In the 2015 outbreak of Ebola in Sierra Leone, there was 48.0% anxiety or depression and 76.0% post-traumatic stress among healthcare workers.14
Although the long-term psychological effects of COVID-19 remain unknown, studies on healthcare workers have shown 50.4% depression, 44.6% anxiety, and 71.5% stress in China15; 24.7% depression, 44.6% anxiety and 19.8% stress in Italy16; and 11.4% depression, 23.4% anxiety and 5.7% stress in India.17 A systematic review and meta-analysis showed a high prevalence of moderate-to-severe depression, anxiety and stress among healthcare workers during the COVID-19 pandemic.18 In Ghana, healthcare workers reported an increased prevalence of depression, anxiety, and stress during the early phase of the COVID-19 pandemic.19 Furthermore, studies comparing emergency and non-emergency healthcare workers found that emergency healthcare workers had a high prevalence of depression, anxiety and stress.20,21
With the poor quality of patient care,22 the rising number of medical errors and medical litigation,23 and increased disability and absenteeism due to increased levels of poor mental health among healthcare workers, it has become important to screen for mental health issues among healthcare workers. One of the most commonly used scales for screening mental health is the Depression, Anxiety and Stress Scale (DASS).24 Since it was developed in 1995, the psychometric properties of DASS and its short version, DASS-21 (21 items), have been evaluated in both clinical25 and non-clinical26 samples for validity and reliability. The self-reporting scale has 42 items (DASS-42), which was reduced to 21 to form the short version (DASS-21). Seven items with the highest factor loadings from each subscale of the original DASS were selected to form the DASS-21. DASS-21 has three main subscales: depression, anxiety and stress. The depression scale evaluates hopelessness, self-deprecation, dysphoria, devaluation of life, lack of interest/involvement, anhedonia and inertia. The anxiety scale evaluates autonomic arousal, skeletal muscle effects, situational anxiety, and subjective experience of anxious affect. The stress scale evaluates nervous arousal, difficulty relaxing and being easily upset and impatient.
Many previous studies have reported excellent reliability and internal consistency of the DASS-21 in healthcare workers.4 However, there has been some inconsistency in the factorial structure of DASS-21. This has led to the development of other structural models apart from the original three-factor model. The one-factor model developed by Lovibond and Lovibond24 has shown a good fit in many studies. A two-factor structure of DASS-21 was the best fit in the Brazilian adolescent study.27 However, in the general population, a tripartite structural model consisting of anhedonia, physiology, hyperarousal, and general negativity affect fit better than other proposed DASS-21 structures. The bifactor model (quadripartite) has also been a good fit in several studies.
DASS-21 has been used extensively in various clinical settings and research because of its ability to differentiate between depression distinctly, anxiety and stress clusters of symptoms.24,26,28 DASS-21 is designed to detect the symptoms of depression, anxiety and stress. In the clinical setting, DASS-21 cannot be used to diagnose people with depression or anxiety. DASS-21 alerts the doctor to identify people who are highly susceptible to clinical depression or anxiety, indicating the need for further investigations.24
DASS-21 has been validated for different populations, but, to date, no research paper has previously validated the psychometric properties of DASS-21 in a Ghanaian population. This has led to its limited use as a screening tool for mental health; therefore, it is critical to evaluate the psychometric properties of DASS-21 among healthcare workers. This study aimed to examine the psychometric properties of the DASS-21 scale and evaluate the level of depression, anxiety, and stress among healthcare workers in a tertiary hospital during the SARS-CoV-2 delta variant outbreak in Ghana.
We conducted a cross-sectional study to determine the psychometric property of DASS-21 among healthcare workers in Korle-Bu Teaching Hospital. This study used an online survey method and recruited participants using a convenience sampling technique. Healthcare workers were invited to complete an online survey (https://kbth.gov.gh/kbredcap/) which was distributed through the hospital’s social media platforms. The online survey, created by the research electronic data capture (REDCap) system (RRID:SCR_003445),29,30 was open for two months (June 3–Aug 3 2021). The survey was aimed at all healthcare workers in Korle-Bu Teaching Hospital working in various departments. These included doctors, nurses, administrators, record officers, pharmacists, laboratory scientists, etc.
The inclusion criteria for the study were all healthcare workers who were at the post during the outbreak of the delta variant. Healthcare workers on leave, travelling, ill, and those with known mental health disorders were excluded from the study. A healthcare worker was defined as anyone employed by the Ministry of Health to work at Korle-Bu Teaching Hospital.
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. The Korle-bu Teaching Hospital institutional review boards (IRD: KBTH-ADM/00014/2021) in Ghana approved all study procedures on 2nd February 2021. Electronic informed consent was obtained from all participants.
We estimated the minimum sample size based on one item to 10 participants.31 Therefore, the minimum acceptable sample size based on 21 items of DASS-21 was 210 respondents. However, the total number of healthcare workers participating in the study was 1201.
This study was conducted during the hospital’s SARS-CoV-2 delta variant outbreak. The online survey sent to the participants consists of an introductory statement and an electronic informed consent that briefly explained the study’s aims, purpose, and benefit. It also informed participants that their information would be used for research purposes, kept confidential, and would not be shared with the hospital’s management. After reading the introductory statement, participants who selected “yes” meant that they had consented to participate in the study and were allowed to answer questions in the demographic and eligibility section. Only those who met the inclusion criteria answered the rest of the questionnaire. All answers to the questions were required; thus, there were no missing observations. In total, 1201 healthcare workers were enrolled in the study. All data collected were anonymous and kept confidential.
During the study period, the global incidence of confirmed COVID-19 cases had increased by 25%. In Ghana, the average weekly cases increased from 45 to 2000. Similarly, the total number of confirmed cases in the hospital increased from 5 to 80 per week.
Demographic variables
Demographic variables were obtained from a self-reported online questionnaire that included age, sex duration of work, occupation or profession and working in an emergency unit.
Depression, Anxiety and Stress Scale
DASS-21 assesses the state of depression, anxiety, and stress in the last week. DASS-21 consists of three subscales with seven items, each assessing depression, anxiety, and stress. It uses a 4-point Likert response for each item from 0 to 3 (0 = “did not apply to me at all” to 3 = “applies to me most of the time”). A subscale score is generated by summing relevant responses. The composite score of each subscale was multiplied by two (therefore making DASS-21 (short version) comparable to the DASS-42 scale (original version). A high composite score indicates a higher psychological stress.24,28
All statistical analyses were performed using SPSS (version 26) (RRID:SCR_002865), but the confirmatory factor analysis (CFA) was performed using AMOS (version 26) (RRID:SCR_022686. Descriptive statistics used in this paper included mean and standard deviation, and proportion and percentage.
We first validated the DASS-21 by examining the factorial structure (construct validity), internal consistency, convergent validity, and discriminate validity. Validation assesses the ability of the DASS-21 questionnaire to measure the latent construct (depression, anxiety, and stress) intended to be measured. After validation, we examined measurement invariance across the sex and professional groups. Before validation analysis, Kaiser–Meyer–Olkin (KMO) and Bartlett’s tests were performed to check the suitability of the dataset. The results indicated that our sample was adequate for the CFA (Chi-square χ2 (210) = 13064.5, p < 0.01) and the overall measurement of sampling adequacy was 0.95.
CFA with maximum likelihood estimation was performed to determine the structure and item loading in DASS-21. We tested five models of DASS-21, which included a one-factor model in which all the 21 items load on a general distress factor,24 a two-factor model in which stress and anxiety were combined as one factor and depression as another factor,32 the original three-factor model,24 a bifactor model with general distress (negatively affectivity) as a factor, with depression, anxiety and stress as specific factors26 and a tripartite model in which all items load on a negative affectivity factor, and a specific factor that is made up of anxiety and depression.33
The results from CFA were analyzed based on the following fit indexes and goodness-of-fit criteria: 1) adequate fit: root-mean-square error (RMSE) (<0.08),34,35 comparative fit index (CFI) (>0.9)36; and 2) good fit: RMSE (<0.05)35 and CFI (>0.95).36 The χ2 test result is strongly dependent on the sample size but it is estimated for all models.
The internal consistency of the DASS-21 and its subscales (Depression, Anxiety and Stress) were assessed using Cronbach’s alpha. A cut-off Cronbach alpha value of > 0.7 indicates high internal consistency.37,38 Pearson’s correlation coefficient was used to evaluate the item-total correlation. Items with a very low correlation coefficient of < 0.3 were deleted.37
We assessed both the convergent and discriminant validity of the best-fit model. Convergent validity is the ability of the items to successfully estimate the latent variable(s) of the best-fit model. The average variance extracted (AVE) and composite reliability (CR) were used to assess the convergent validity of the best-fit model. For convergent validity to be significant, a CR value of 7 or greater (CR > 7) and AVE values of 0.5 or greater are required.39
Discriminant validity determines how a model can distinctly identify depression, anxiety and stress from other factors. We assessed discriminant validity by comparing the amount of variance explained by the construct and the share variance with other constructs. Thus, for a model to have discriminate validity, the square root of the AVE of the measured construct should be larger than the correlation coefficients of the other constructs.40
A multi-group CFA with sex and professional group was performed to evaluate measurement invariance. The first was the configural model, in which all parameters were unrestricted across the group, and then the metric model, where there was factor loading to equality between groups. Metric invariance was evaluated as a function of a difference between the unconstrained and constrained models. The lack of invariance was indicated when the χ2 test of goodness of fit was significant (p < 0.05). Since χ2 is sample-size dependent, we also considered a model to be invariant across a group with a slight change (∆) in CFI and root mean square error of approximation (RMSEA)—CFI and RMSEA should not exceed 0.02 and 0.015, respectively.41
A total of 1201 healthcare workers participated in the study. The mean age of the participants was 34 years (SD = 7.1 years), ranging from 20 to 58 years. The participants were predominantly females, nurses, and had worked for less than three years (53.9%). One-fifth of the participants worked in emergency-related units. DASS-21 provided an independent assessment of depression (mean = 13.6, SD = 2.7), anxiety (mean = 15.8, SD = 3.1) and stress score (mean = 12.8, SD = 3.3) with mean scores indicating mild depression, severe anxiety and normal stress levels (Table 1).
The summary of the fit statistics is shown in Table 2, indicating that the one-factor and two-factor models had unacceptable fit values. The fit values for the bifactor and tripartite models provided an improved and acceptable fit. However, the three-factor model (Figure 1) provided the best fit for the data.
The standardized factor loadings indicated higher and excellent factor loadings for depression, anxiety and stress, except item 2 (“Dryness of the month”) with a factor loading of 0.38. Therefore, item 2 was considered for removal. The final three-factor model (Figure 2) was further enhanced by allowing correlation between items from the same subscale using a high modification index of > 10. The modified DASS-21 provided the final best-fit model to data (CFI: 0.961, Tucker-Lewis index (TLI): 0.929, standardized root mean square residual (SRMR): 0.032, RMSEA: 0.071).
From Table 3, the Cronbach alpha for the total DASS-21 scale was adequate. Internal consistency for the three subdomain scales was very good: DASS-21 depression (DASS21-D) was 0.88, DASS-21 anxiety (DASS21-A) was 0.86, and DASS-21 stress (DASS21-S) was 0.81. The item scale and total item scale correlations were satisfactory and ranged from 0.3 to 0.7 and 0.4 to 0.8, respectively.
We estimated the convergent validity based on the value of CR and average variance extracted. From Table 4, all the constructs (depression, anxiety and stress) met the predefined criteria for convergent validity—CR > 0.7, CR > AVE and AVE > 0.5. For discriminant validity, the square of the average variance extracted for depression (0.726), anxiety (0.713) and stress (0.707) constructs were all higher than their correlation with other constructs (off-diagonal) (Table 4). This indicates that depression, anxiety, and stress are distinct elements (discriminant validity), as proposed by Lovibond and Lovibond.25 All constructs passed the convergent and discriminant validity tests.
The multi-group CFA was used to evaluate the degree of measurement invariance of DASS-21 across sex and professional group. Table 5 shows that the configural and metric models for both sex and professional group provided an excellent fit for the data. The invariance test indicates equivalence across males and females. The increase in model constraint reveals no significant difference between the configural and metric (χ2 (17) = 21.1, p > 0.05). For the professional group, configural invariance shows that the data fit adequately to the model without any constraint. The difference between the metric and configural invariance was not statistically significant, which indicates that metric invariance was achieved.
The prevalence of moderate-to- severe symptoms of depression, anxiety and stress among healthcare workers was 14.2% (170), 4.5% (54) and 30.8% (370), respectively (Table 6). Doctors had high psychological stress compared with other healthcare workers. Healthcare workers who worked in the emergency setting or for more than 10 years had a higher prevalence of depression, anxiety, and stress than others. Moderate-to-severe depression, anxiety, and stress symptoms were equivalent across males and females. However, healthcare workers older than 50 years had lower psychological stress incidence (Table 6).
To our knowledge, this study is the first to examine the psychometric properties of DASS-21 and provide evidence for its use as a measure of mental health issues among Ghanaian healthcare workers. Overall, our results support the three-dimension structures of DASS-21 as reported in previous studies.24,28,42 This indicates that DASS-21 can be used as a self-reported tool to assess mental health issues of depression, anxiety, and stress among healthcare workers.
The construct validity of DASS-21 was satisfactory based on the CFA, and all model fit indexes were acceptable. Our CFA showed that the best-fit model for our data was the three-factor oblique model for depression, anxiety, and stress, which has been previously validated in clinical and non-clinical samples.25,26,28,43 The three-factor oblique model was better than the one-factor and two-factor models, indicating that depression, anxiety and stress are distinct constructs, as previously reported.24,44
A modification improved model fitness to items with poor factor loadings (factor loading < 0.4). However, item 2 (“dryness of the mouth”) from the anxiety subscale shows weaker factor loadings than other items in our study sample. Previous studies have questioned the utility of item 2 (“dryness of the mouth”) as a measure of the anxiety construct.25,45–48 On the contrary, some studies have suggested that item 2 (“dryness of the mouth”) performed excellently in the psychological stress construct (one-factor model).45,47 Moreover, Parkitny et al.46 described the “dryness of the mouth” item as a vague response to DASS-21. Besides the initial improvement, the three-factor model was further enhanced using a modification index that allows correlation error between items from the same scale.
Our reliability results for DASS-21 showed an excellent internal consistency for all the subscales, with the Cronbach alpha ranging from 0.81 to 0.93. These results confirmed previous reports about the reliability of DASS-21 in different studies and populations.26,28,49,50 We confirm from previous studies that DASS-21 has a high convergent and discriminant validity of the three constructs.26,42,49,51
In this study, we also examined invariant measurement across sex and professional groups. Our results showed that DASS-21 has a full configural and metric invariance across sex and professional groups. This indicates that the meaning of DASS -21 to measure depression, anxiety, and stress was the same for females and males or doctors, nurses, and non-clinical healthcare workers—there were no differences across sex and professional groups in the DASS-21 structure.
In our study, the prevalence of depression, anxiety, and stress among healthcare workers was 14.4% (DASS-21 depression score > 13), 30.8% (DASS-21 anxiety score > 9) and 4.5% (DASS-21 stress score > 18), respectively. These prevalences of depression, anxiety and stress were lower than those reported in previous studies of Ghanaian healthcare workers during the early phase of the pandemic19 and other studies from China,15 Italy,16 and India.17 The differences in the prevalence of depression, anxiety, and stress may be due to sociocultural differences affecting the individual’s experience and expression of mental health symptoms. This shows that DASS-21 can be used as a screening tool to identify healthcare workers with depression, anxiety and stress. The tool is relevant for the hospital employee assistance program, which aims to identify healthcare workers with mental health issues in order to provide support for them.
The high prevalence of depression and anxiety reported among healthcare workers may be due to the possible impact of the delta variant of SARS-CoV-2 in the hospital. During the COVID-19 outbreak, especially in the vaccine-availability era, when there was widespread disease and an increased number of deaths, it was very common to observe high psychological stress among healthcare workers who may have directly or indirectly managed COVID-19-infected patients. Similar events of the high prevalence of mental health issues among healthcare workers have been observed in the SARS, MERS, H1N1 influenza, and Ebola outbreaks.
With the advent of the COVID-19 vaccine, healthcare workers’ mental status depends on the vaccine’s ability to prevent them from being infected and infecting others. The decrease in vaccine efficacy resulting in increased breakthrough infections severely impacts their mental and physical health. Moreover, the virulent nature of the delta variant, the high influx of infected patients with severe clinical manifestations or hospitalization, high mortality, increased workload, fear of getting infected and the dearth of personal protective equipment all negatively affect the mental health status of healthcare workers.
The strength of this study is that it is the first time a DASS-21 validation has been performed among Ghanaian healthcare workers. In addition, this present study provides novel information about the prevalence of mental health issues among healthcare workers during the delta variant outbreak. One limitation of this study was the self-reported response of DASS-21, which may have led to potential biases or measurement errors. For example, some healthcare workers may have provided a socially acceptable response about their mental health issues because of stigmatization. Additionally, using a convenient sampling method could result in selection bias because only healthcare workers with internet access completed the online questionnaire. Finally, all the respondents were healthcare workers from the same hospital, which may affect the generalization of the findings.
In summary, our results indicate that DASS-21 has good psychometric properties among Ghanaian healthcare workers. We found DASS-21 to be valid and reliable, with the ability to distinctly identify depression, anxiety, and stress from other factors. Our data suggest that this tool is invariant across sex and across doctors, nurses, and non-clinical healthcare workers. Thus, DASS-21 can be used as a self-reporting tool to screen for depression, anxiety, and stress in health workers. This will allow authorities, such as employee assistance programs, to identify healthcare workers at higher risk of developing work-related mental health disorders.
Figshare: Dataset for psychometric evaluation of depression, anxiety, and stress scale among healthcare worker in Ghana during COVID-19 outbreak. Dataset. https://doi.org/10.6084/m9.figshare.21929697. 52
This project contains the following underlying data:
Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).
Views | Downloads | |
---|---|---|
F1000Research | - | - |
PubMed Central
Data from PMC are received and updated monthly.
|
- | - |
Is the work clearly and accurately presented and does it cite the current literature?
No
Is the study design appropriate and is the work technically sound?
Yes
Are sufficient details of methods and analysis provided to allow replication by others?
Yes
If applicable, is the statistical analysis and its interpretation appropriate?
Yes
Are all the source data underlying the results available to ensure full reproducibility?
Yes
Are the conclusions drawn adequately supported by the results?
Yes
References
1. Thiyagarajan A, James TG, Marzo RR: Psychometric properties of the 21-item Depression, Anxiety, and Stress Scale (DASS-21) among Malaysians during COVID-19: a methodological study.Humanit Soc Sci Commun. 2022; 9 (1): 220 PubMed Abstract | Publisher Full TextCompeting Interests: No competing interests were disclosed.
Reviewer Expertise: Epidemiology, Health Promotion, Research Methodology, Communicable and Non-Communicable Diseases, Environmental Health, Global Sustainable Development, Health and Social Services, Nutrition, Ageing and Quality of Life, Social and Behavioural Health, Healthcare Management, Strategic Management, Health Services Management, Health Leadership Management, Quality Management, Medical Education
Alongside their report, reviewers assign a status to the article:
Invited Reviewers | |
---|---|
1 | |
Version 1 01 Mar 23 |
read |
Provide sufficient details of any financial or non-financial competing interests to enable users to assess whether your comments might lead a reasonable person to question your impartiality. Consider the following examples, but note that this is not an exhaustive list:
Sign up for content alerts and receive a weekly or monthly email with all newly published articles
Already registered? Sign in
The email address should be the one you originally registered with F1000.
You registered with F1000 via Google, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Google account password, please click here.
You registered with F1000 via Facebook, so we cannot reset your password.
To sign in, please click here.
If you still need help with your Facebook account password, please click here.
If your email address is registered with us, we will email you instructions to reset your password.
If you think you should have received this email but it has not arrived, please check your spam filters and/or contact for further assistance.
Comments on this article Comments (0)