Googling for Suicide–Content and Quality Analysis of Suicide-Related Websites: Thematic Analysis

Background Suicide represents a public health concern, imposing a dramatic burden. Prosuicide websites are “virtual pathways” facilitating a rise in suicidal behaviors, especially among socially isolated, susceptible individuals. Objective The aim of this study is to characterize suicide-related webpages in the Italian language. Methods The first 5 most commonly used search engines in Italy (ie, Bing, Virgilio, Yahoo, Google, and Libero) were mined using the term “suicidio” (Italian for suicide). For each search, the first 100 webpages were considered. Websites resulting from each search were collected and duplicates deleted so that unique webpages could be analyzed and rated with the HONcode instrument Results A total of 65 webpages were included: 12.5% (8/64) were antisuicide and 6.3% (4/64) explicitly prosuicide. The majority of the included websites had a mixed or neutral attitude toward suicide (52/64, 81.2%) and had informative content and purpose (39/64, 60.9%). Most webpages targeted adolescents as an age group (38/64, 59.4%), contained a reference to other psychiatric disorders or comorbidities (42/64, 65.6%), included medical/professional supervision or guidance (45/64, 70.3%), lacked figures or pictures related to suicide (41/64, 64.1%), and did not contain any access restraint (62/64, 96.9%). The major shortcoming to this study is the small sample size of webpages analyzed and the search limited to the keyword “suicide.” Conclusions Specialized mental health professionals should try to improve their presence online by providing high-quality material.


Introduction
Suicide represents a public health concern, imposing a dramatic burden both in epidemiological and societal terms [1]. According to the latest available 2021 report of the World Health Organization (WHO) in 2019, Italy had a crude rate of 6.7 (95% CI 6.0-8.5) suicides per 100,000 people, with a male:female ratio of 2.9 [2].
Within the current eHealth era, in which the new information and communication technologies are being increasingly exploited to obtain health-related information [3], the internet can be characterized as having a Yin-Yang, paradoxical nature [4]. On the one hand, there exists a statistically significant correlation between a pathological use of the web and suicidality (either in terms of suicidal ideation or nonsuicidal self-injury). Prosuicide websites and online suicide pacts are "virtual pathways" that facilitate the emergence of suicidal behaviors, especially among socially isolated and susceptible individuals. On the other hand, the internet can serve as an effective tool for suicide monitoring and tracking [3,5] and can play a major role in suicide prevention [6,7], particularly for those vulnerable individuals who would be otherwise difficult to reach with conventional approaches [8,9].
Understanding the determinants of suicide-related web searches is, therefore, of crucial importance [10]. According to Wong and collaborators [11], the categories of the most visited suicide-related webpages can be classified into entertainment (30.13%), scientific information (18.31%), and community resources (14.53%). Among the 1314 accessed webpages in their study, prosuicide websites represented only a small fraction (0.15%). The most used search terms were "committing suicide with a gas oven," "hairless goat," "pictures of murder by strangulation," and "photo of a severe burn." Suicide-related digital behavior has been assessed in different countries and different languages [12][13][14][15]. However, to the best of our knowledge, there is a dearth of information concerning suicide-related webpages in the Italian language. As such, the aim of this study is to address this gap in knowledge.

Suicide-Related Website Selection and Inclusion
The first 5 most commonly used search engines in Italy (namely, Bing, Virgilio, Yahoo, Google, and Libero) were mined with the search term "suicidio" (Italian for suicide). For each search, the first 100 webpages were considered. Websites resulting from each search were collected and duplicates deleted so that webpages could be analyzed and rated by 2 authors independently (MO and DP).
Quality assessment of the included webpages was performed with the HONcode instrument, which comprises the following sections: authoritative (if the qualifications of the authors are mentioned), complementarity (if the site aims at supporting but not replacing or substituting the physician-patient relationship), privacy (if visitors' data are protected and respected), attribution (if sources of published information in the medical and health-related pages are explicitly mentioned and referred to), justifiability (if claims related to potential benefits or performance are evidence based), transparency (if the presentation of the online material is accessible, with accurate email contacts), financial disclosure (if eventual funding sources are identified and mentioned), and advertising policy (if advertising and editorial materials are separate and identifiable) [16].
The agreement between the 2 authors was assessed with the Cohen κ statistic.
Furthermore, the 2 authors filled in an ad hoc predesigned Excel spreadsheet (Microsoft Corp) containing questions concerning the attitude of the webpage toward suicide (prosuicide, antisuicide, mixed or neutral), content and purpose of the webpage (supporting or emotional, informative, preventive), supervision or guidance (if the webpage was user-generated or handled by a professional figure, such as a psychiatrist or a psychologist), reference to age groups (adolescents or older people) targeted or referred to by the webpage (this simple dichotomic stratification was applied based on the availability of information provided and considering that the majority of internet users consist of adolescents), reference to other psychiatric disorders or comorbidities, the presence of figures or pictures related to suicide, and the presence of access restraint.
Moreover, for each included webpage, the number of comments' likes, and shares, if any, was recorded.

Statistical Analysis
Before commencing any statistical handling and processing of the data, figures were visually inspected for potential outliers. The normality of data distribution was checked using the D'Agostino-Pearson omnibus test. Univariate analysis (chi-squared test and the chi-squared test for trend, Fisher's exact test, t test, and analysis of variance or their nonparametric versions in case of violation of normal distribution) was performed for the variables under study. Multivariate linear regression analysis was performed to shed light on the determinants of the HONcode score.
All statistical analyses were performed with the commercial SPSS for Windows, version 24.0 (IBM Corp). Figures with P values less than .05 were considered statistically significant.

Results
Sixty-four unique websites were retrieved and analyzed. Concerning the attitude toward suicide, 4 webpages were in favor of suicide (6.3%), whereas 8 sites were against suicide (12.5%). The remaining webpages (52/64, 81.2%) were neutral toward suicide. The overrepresentation of neutral pages in tone was statistically significant (χ 2 Table 2. At the univariate analysis (Table 3), the HONcode score was higher in mixed or neutral webpages (P<.001), in websites with a preventive purpose (P=.02), with professional or medical supervision or guidance (P<.001), with references to other psychiatric disorders or comorbidities (P=.003), and without access restraint (P=.03). No statistically significant differences could be detected in terms of references to age groups (adolescents versus older people, P=0.81) or the presence of figures or pictures (P=.41).
Furthermore, webpages with medical supervision or guidance had more an informative and preventive content and purpose than did supporting or emotional ones (χ 2 2 =9.19, P=.01; χ 2 1 for trend=3.19, P=.07). Those webpages with professional supervision or guidance also had a more positive or mixed/neutral attitude toward suicide, and this was statistically significant (P=.002). In contrast, no differences were found in terms of references to other psychiatric disorders or comorbidities, references to age groups, access restraint, and the presence of figures or pictures related to suicide.
In the multivariate linear regression analysis (Table 4), the predictors of the HONcode score were professional/medical or supervision/guidance of the webpages (regression coefficient=3.94; P<.001) and references to psychiatric disorders as comorbidities (regression coefficient=1.46; P=.003).
Meanwhile, no differences were found in terms of likes, comments, or shares either in the univariate or the multivariate analyses.

Main Findings and Comparison With the Literature
To the best of our knowledge, this is the first study to assess the quality and content of suicide-related webpages in the Italian language.
We found that the percentage of explicitly prosuicide websites was greater than that reported by Wong et al [11] but less than that reported by Chen et al (16.3%), who analyzed 375 linked webpages in Chinese [12]. According to the authors, 41.3% of the included webpages were antisuicide, and the majority of the prosuicide sites were user-generated (96.7%). Interestingly, searches using the search term "ways to kill yourself" (31.7%) and "painless suicide" (28.3%) generated much larger numbers of potentially harmful webpages than did the keyword "suicide" (4.3%).
A much larger portion of prosuicide websites (42%) was found by Sakarya et al [17], who analyzed 100 webpages. Thirteen percent of the studied sample was found to have content or material that may be considered protective against suicidal behavior, a result in line with our findings. However the authors found that "protective" websites did not have any kind of medical supervision or guidance from specialized mental health professionals. In contrast to this, we managed to find a statistically significant association between these 2 variables.
Til and Niederkrotenthaler [18] performed a cross-cultural study by comparing the suicide-related web searches in Austria and the USA. They found that in both countries, protective outweighed harmful website characteristics by approximately 2:1, a finding in line with our result. Authors were able to confirm the observation of Wong et al: websites retrieved with method-related search terms (eg, how to hang yourself) contained more harmful information (USA: P<.001; Austria: P<.05) and fewer protective characteristics (USA: P<.001; Austria: P<.001) compared to the term "suicide" [11]. Conversely, help-related search terms (eg, suicidal thoughts) yielded more webpages with positive and protective characteristics (USA: P=.07; Austria: P<.01). Interestingly, authors found that websites retrieved with the US search engines generally had more protective characteristics (P<.001) than did those retrieved with Austrian search engines. Moreover, resources with dangerous or harmful characteristics were better ranked than those with positive or protective characteristics (USA: P<.01; Austria: P<.05).
In another study, Thornton and colleagues [13] found that among Google searches retrieved related to suicide, a high portion of them were irrelevant webpages (n=136, approximately 26% of the entire sample). Of the 329 relevant websites, the majority were suicide preventive (about 68%); however, a considerable proportion of sites expressed mixed/ambiguous (22%) or neutral (8%) attitudes toward suicide, and 1% were explicitly prosuicide, figures which were slightly different from ours.
These inconsistent literature findings could be explained by the cultural differences across countries in which suicide-related web searches are performed, as shown by the research of Till and Niederkrotenthaler [15], as well as policies concerning the web. Overall, the combination of societal data and online behavior monitoring provide the best indication of risks [19].
The presence of antisuicide online material is of crucial importance for 2 major reasons: (1) according to a recent review of the literature [6], individuals at risk for suicide are probably more prone to look for suicide-related information online, and searching online information related to suicide is a proxy of suicidal ideation; and (2) according to a randomized controlled trial, education professional suicide prevention websites appear to increase and improve suicide prevention-related knowledge, especially among vulnerable and socially isolated individuals [18]. Therefore, exposure to high-quality curated websites may be associated with a reduction in suicidal thoughts and actions. Recent research has shown that the proactive suicide prevention online model is robust for identifying people that are at risk of suicide [20].

Strengths and Limitations of the Study
Despite some strengths, including being the first study to systematically assess the quality of suicide-related webpages in the Italian language, our investigation suffers from some limitations that should be properly acknowledged. The major shortcoming is the small sample size of webpages analyzed and the search limited to the keyword "suicide." Future research should also explore other words indirectly related to suicide as well as other terms related to the ways suicide can be committed (ie, method-related search terms) and explore whether the semantic differential has an impact on suicide-related webpages in terms of content and quality. Moreover, caution should be used when generalizing the present findings to other countries, as our results could be affected by country-specific cultural issues and may not reflect the nature of other settings. Further, although we assessed the reference to other psychiatric disorders or comorbidities in the webpages included in our study, we were not able to establish a link between mental illness and suicide in terms of browsing websites. This warrants dedicated, ad hoc studies specifically exploring this topic. Another aspect that should be investigated concerns the size or burden of the psychological effect imposed by browsing suicide-related websites.
Specialized mental health professionals should take into account these findings, make an effort to improve their presence online, and provide high-quality material. However, given the aforementioned limitations, further research in the field is warranted.