Utility of a time frame in assessing psychological pain and suicide ideation

Participants

We collected data from general population adults who responded to Facebook advertisements that invited to complete an anonymous online survey. A similar approach was used to collect data on suicidal behavior in an Australian online sample (Batterham, Calear & Van Spijker, 2015). To draw people’s attention, ad texts focused on the link between psychological pain and suicide, the prevalence of suicide and suicide attempts, or that someone close may be experiencing unbearable pain. Another ad indicated that people of any race/ethnicity could participate, regardless of whether they were experiencing psychological pain. In addition, all ads showed a map of the United States (US) and indication that this was a national survey. Inclusion criteria were being at least 18 years old, a resident of the US, and able to read and understand English. Of 708 individuals who visited the survey website, 389 provided consent (55%) to participate. Among those who consented, 133 individuals did not submit their responses (34%). Final analysis was done on 242 respondents with complete psychological pain data: 133 completed the PS and 109 completed the PS-TF.

A comparison of respondents who submitted their data and respondents who dropped out before completing the survey showed no statistical differences in terms of gender, ethnicity, sexual orientation, student status, when they had last experienced psychological pain, when they had experienced their worst psychological pain, whether they wished to be dead during the past week, or whether they had ever attempted suicide. However, respondents who submitted their data were older (32.5 [SD 15.3] vs. 29.7 [SD 14.4], p = .082) and significantly more likely to have used pain medication on the day they completed the survey (χ² = 5.10, p = .024), to have been diagnosed with a mental illness (χ² = 4.51, p = .034), and to have lost someone to suicide (χ² = 7.95, p = .005).

Measures

The survey started with demographic questions and then asked about medication use, psychological pain, tolerance for psychological pain, suicide history, and feelings about death and dying, in order of what was considered increasing sensitivity (i.e., content that participants could perceive as potentially threatening or problematic). Psychological pain tolerance and feelings about death and dying are central to a forthcoming article and will not be discussed here.

Psychache scale

To orient respondents, the PS describes psychological pain as “a hurting feeling inside, often described as pain you feel in your heart or mind. It indicates how much you hurt emotionally or mentally” (Holden et al., 2001). The PS instructions then continued with “The following statements refer to your psychological pain, NOT your physical pain. Please indicate how frequently each of the following occurs”. The PS has 13 items with higher scores reflecting greater psychological pain. Nine items are scored on a 5-point frequency scale ranging from never to always, corresponding to a value of 1–5. Four more items, reflecting pain intensity, are scored on a 5-point symmetrical scale ranging from strongly disagree to strongly agree, also corresponding to a value of 1–5. The total score is obtained by summing the item scores, resulting in a total score between 13 and 65. The original paper-version of the PS is shown in the ‘Appendix’.

The PS is well validated in diverse populations, including university students, homeless men, outpatients with depression, and male prison inmates (Mills, Green & Reddon, 2005; Patterson & Holden, 2012; Troister & Holden, 2012; Li et al., 2014; Xie et al., 2014). We found excellent internal consistency reliability for our sample with Cronbach’s α = .96 and no inter-item correlations lower than 0.43.

Psychache scale with time frame

The PS-TF uses the same items as the original PS; however, to have respondents focus on current psychological pain, we modified the last sentence of the instructions at the top of the scale to read “Please indicate how frequently each of the following has been occurring during the past week, including today”. This time frame was chosen because psychological pain is a lasting state that takes time for resolution (Meerwijk & Weiss, 2011) and we assumed psychological pain to linger with varying intensity for days at least. Moreover, the widely used Beck Scale for Suicide ideation (Beck, Brown & Steer, 1997) uses the same time frame to assess thoughts of suicide and suicidal behavior. We found excellent internal consistency reliability for the PS-TF with Cronbach’s α = .94 and no inter-item correlation lower than 0.37.

Suicidal ideation and behavior

We used respondents’ wish to be dead during the past week as a dichotomous measure of suicide ideation (yes/no). The ‘wish to be dead’ criterion often features among initial questions to assess suicide risk; for example, in the Columbia Suicide Severity Rating scale (Posner et al., 2011) and in the MINI Neuropsychiatric interview (Sheehan et al., 1998). In addition, we asked whether they had attempted suicide and if so, how often and how long ago for the most recent attempt (past week, month, year, or longer ago).

Procedure

Data were collected from September 2015–June 2016. We used Research Electronic Data Capture (REDCap™) to develop our online survey and collect the data. REDCap™ is a secure and HIPAA-compliant electronic data capture tool (Harris et al., 2009). Visitors to the survey web site first arrived on the informed consent page, describing the nature of the survey and resources available to them should the survey make them feel uncomfortable or upset (e.g., the National Suicide Prevention Lifeline phone number). By submitting their age, city, and ZIP code on the consent page, respondents self-identified as being eligible and initiated the actual survey. Based on the time of written consent in seconds (odd/even), participants were assigned to a group that completed the original PS or a group that completed the PS-TF.

Upon completion of the survey, respondents were offered a choice to participate in a drawing to win a $50 debit card. As this was an anonymous survey and respondents could submit their data more than once, even though they were instructed not to, we developed an algorithm to search the data for potential duplicate entries. The algorithm compared age, gender, city, and ZIP code. We then manually compared these entries and retained the first entry without missing data. Of potential duplicates identified by the algorithm, five entries were considered actual duplicates and discarded. The Institutional Review Board at the University of California, San Francisco, approved the study (IRB# 16-18686).

Data analysis

We calculated means (SD) and frequencies of sociodemographic variables for participants who completed the PS or PS-TF and tested for differences between the two groups using Student’s t and Pearson’s χ². Missing data were omitted pairwise. Depending on when psychological pain was last experienced, pain scores were right skewed, violating assumptions for a parametric model. Transforming the data did not satisfactorily normalize their distribution. Therefore, we used Wilcoxon rank sum tests (also known as Mann–Whitney) to test our first hypothesis regarding differences in PS and PS-TF scores when participants reported pain experienced currently versus during more distant time frames. Scores were examined for psychological pain experienced “today”, “during the past week”, and “more than a week ago”. We applied a Bonferroni correction to reduce the risk of type-I error related to multiple testing.

Hypothesis two was assessed by comparing the scales’ efficiency in distinguishing respondents with and without suicide ideation. We determined the PS and PS-TF’s sensitivity, specificity, and positive and negative predictive value (PPV, NPV) at multiple cutoff values. Sensitivity was calculated as True Positives (TP)/(TP + False Negatives (FN)) and specificity was calculated as True Negatives (TN)/(TN + False Positives (FP)). Positive predictive value was calculated as TP/(TP + FP) and negative predictive value was calculated as TN/(TN + FN). The 95% confidence intervals (CI) for the difference score between the two scales were determined with a bootstrap approach for each of these metrics (Altman, 2000). We report results obtained with 10,000 steps and the basic bootstrap method to calculate CIs, but similar values were obtained with other methods. Screening efficiency is visualized in a receiver-operating curve (ROC) and optimum cutoff values were determined by minimizing the Euclidian distance between the top left of the ROC plot and the respective curves. R version 3.2.4 was used for all analyses, and statistical significance was assumed at p < .05.

Sample description

Table 1 shows sociodemographic characteristics of respondents. No statistically significant differences were observed between the PS group and the PS-TF group, although the racial background of respondents randomized to the PS group was more often white (84 vs. 58, χ² = 2.736, p = .098). Overall, respondents were more often female (59%) than male (39%). We asked respondents questions to determine whether they belonged to specific “vulnerable populations”. Among them, few respondents indicated being pregnant (n = 4), being imprisoned (n = 1), or being hospitalized (n = 2) at the time they completed the survey. With respect to sexual orientation, 47 respondents (20%) described themselves as gay/lesbian, bisexual, asexual, or unsure about their sexual orientation. With respect to mental illness, 50 respondents (21%) indicated having been diagnosed with a mental illness. Sixty-one respondents (25%) indicated having attempted suicide at least once, seventeen of which (7%) made an attempt during the past year. Forty respondents (17%) indicated having lost someone close to suicide. With respect to pain medication, 62 respondents (26%) indicated being on pain medication on the day they completed the survey.

Compared to the 2010 US census records (US Census Bureau, 2016), our sample was younger and more often female. Regarding race, our sample was less often of white or Asian descent and more often of Black or African American, American Indian/Alaska Native, Native Hawaiian and other Pacific Islander, or of mixed descent. Among US adults, 0.6% reported having attempted suicide during the past year and 18.5% reported having any mental illness (Substance Abuse and Mental Health Services Administration, 2014), against 7% and 21% of respondents in our survey, respectively.

Hypothesis testing for original PS versus PS-TF

The majority of participants reported psychological pain during the past week, including today (see Table 1). The overall mean PS and PS-TF scores were 30.2 (SD 13.4) and 31.6 (SD 12.7), respectively. Hypothesis 1 was not supported. A Wilcoxon rank-sum test among participants who experienced psychological pain longer than a week ago, showed no statistically significant difference in psychological pain (W = 739, p = .150) between the PS group (Mdn = 16, n = 44) and the PS-TF group (Mdn = 19, n = 41). Cliff’s delta, an ordinal measure of effect size (Cliff, 1993), suggested a small effect (delta = 0.181). As expected among participants who experienced psychological pain within the past week, we found no statistically significant differences in psychological pain between the PS group and the PS-TF group (see Fig. 1).

Overall, 56 respondents (23%) indicated a wish to be dead during the past week. Table 2 shows their psychological pain scores. The scales’ screening efficiency in terms of identifying respondents with and without a wish to be dead is visualized in Fig. 2. Screening accuracy, as indicated by the area under the ROC (AUC), was similar for both scales (.785 and .784 for the PS and PS-TF, respectively). Table 3 shows screening metrics for both scales against multiple cutoff values. The optimum cut-off value for a wish to be dead was 36 for the PS and 38 for the PS-TF. As can be seen from Table 3, specificity and PPV at the optimum cutoff value were higher for the PS-TF (.81 and .58) than for the PS (.75 and .45), albeit at a 6% reduction in sensitivity and a 4% reduction in negative predictive value (.93 vs. .89). 95% CIs for difference scores did not indicate statistically significant differences (sensitivity: −.19–.29, specificity: −.18–.07, positive predictive value: −.35–.09, negative predictive value: −.05–.14).