Valid measurement of DSM-5 persistent complex bereavement disorder and DSM-5-TR and ICD-11 prolonged grief disorder: The Traumatic Grief Inventory-Self Report Plus (TGI-SR + )

Introduction: When grief reactions after bereavement are so intense that they impair daily functioning, a diagnosis of disturbed grief may apply. Slightly differing criteria-sets for disturbed grief are included in the ICD-11, the DSM-5


Introduction
The death of a loved one is a potentially stressful life-event that may result in grief reactions, such as yearning for the deceased, difficulty moving on with life, and intense sadness.The majority of bereaved people is able to adjust to life without the deceased and does not need professional support during this adaptation process [1][2][3].However, an estimated one out of ten people bereaved by a natural cause and one out of two people bereaved by an unnatural cause report severe and disabling grief reactions that merit clinical attention [4,5].
Intense and prolonged grief reactions that severely impair daily functioning are labelled Persistent Complex Bereaved Disorder (PCBD) in the DSM-5 [6].PCBD is included in section III as condition for further study.PCBD encompasses 16 symptoms, including four symptoms representing separation distress (e.g., "Preoccupation with the circumstances of the death") and 12 symptoms representing reactive distress and social or identity disruption (e.g., "Difficulty with positive reminiscing about the deceased").In the forthcoming text revision of the DSM-5 (the DSM-5-TR), PCBD will be replaced by Prolonged Grief Disorder (PGD) and moved to section II [7,8].DSM-5-TR PGD includes two separation distress criteria ("Intense yearning/longing for the deceased person", "Preoccupation with thoughts or memories of the deceased person") and eight accompanying symptoms (e.g., "Intense loneliness") [7].PGD is also included in the ICD-11 [9].While they carry the same name, the time criterion (12 vs. 6 months post-loss, respectively), number of symptoms (10 vs. 12, respectively), and content of the symptoms of PGD in DSM-5-TR and ICD-11 differ.ICD-11 PGD includes two separation distress items (e.g., "Persistent preoccupation with the deceased") and ten accompanying symptoms (e.g., "Inability to experience positive mood").
Apart from PCBD as per DSM-5, PGD as per ICD-11, and PGD as per DSM-5-TR, several other criteria sets for disturbed grief have been proposed in recent years.These include criteria for PGD proposed in 2009 by Prigerson et al. [10] (PGD-2009), a beta-draft version of the ICD-11 PGD criteria (PGD-BD) proposed by Maercker et al. [11], and criteria for complicated grief (CG) put forth by Shear et al. [12].Recent studies comparing prevalence rates and indices of validity of two or more of these sets have shown that there are significant differences between the sets (for an overview see [13]).For instance, compared to PGD ICD-11 criteria, PCBD criteria have been found to yield considerably lower prevalence rates [14][15][16] and to perform relatively poorly at identifying clinical cases of disturbed grief [17].
Importantly, prior studies examining the performance of different criteria sets have been limited by the fact that items assessing symptoms of disturbed grief were taken from different instruments; for instance, in comparing the aforementioned six sets of criteria, items were combined from outdated grief measures and measures of depression as indicators of these criteria [16].There is an urgent need for one single measurement instrument, tapping putative markers of disturbed grief according to different criteria sets.There is a specifically strong need for an instrument tapping the most recent sets, including PCBD as per DSM-5, PGD as per ICD-11, and PGD as per DSM-5-TR as these strongly influence contemporary and future research efforts and clinical decision making.Very recently, instruments have been developed to assess ICD-11 PGD [18] or DSM-5-TR PGD criteria [19].However, having one single valid diagnostic instrument to assess all relevant criteria sets for disturbed grief is important because it enables direct comparisons of diagnostic performance of these criteria sets.
To assess multiple criteria sets for disturbed grief, Boelen and Smid [20] previously developed the 18-item Traumatic Grief Inventory-Self Report.The TGI-SR was developed to measure both PCBD per DSM-5 [6] and PGD as proposed by Prigerson et al. [10].Two preliminary studies supported the reliability and convergent and criterion validity of the TGI-SR [20,21]; for instance, its items formed a unitary dimension, were distinct from symptoms of depression, and differentiated well between relevant subgroups of bereaved people.However, with the establishment of PGD in ICD-11 and PGD in DSM-5-TR, criteria from these new diagnoses will likely become central to future bereavement research and care.It is thus imperative to have an instrument assessing these PGD symptoms.Therefore, we constructed four additional items, tapping criteria from PGD in ICD-11 and DSM-5-TR that were not previously captured by the TGI-SR and named this expanded measure "TGI-SR+".By performing the research presented in this paper we aimed to examine the psychometric properties of the TGI-SR+, using newly gathered data from two bereaved samples.
Sample 1 (N = 278) was a community sample of Dutch adults bereaved by various causes.Sample 2 (N = 270) consisted of Dutch adults who lost loved ones in a traffic accident.Psychometric properties of the TGI-SR+ were evaluated for each sample separately by examining the: i) factor structure, ii) internal consistency, iii) temporal stability (only in community bereaved sample), iv) convergent validity, v) known-groups validity, vi) number of people meeting criteria for probable caseness of DSM-5 PCBD, DSM-TR PGD, and ICD-11 PGD, and vii) optimal clinical cut-off score for combinations of items representing criteria for (a) DSM-5 PCBD, (b) DSM-5-TR PGD, and (c) ICD-11 PGD.
Regarding the factor structure, we expected an acceptable fit for the unidimensional model for DSM-5 PCBD, but a better fit for the 3-factor PCBD model and an acceptable fit for the 1-factor ICD-11 PGD model [14][15][16].Based on recent research [16,19], we also expected an acceptable fit for a unidimensional model for DSM-5-TR PGD.We expected strong inter-item correlation for each criteria set representing good internal consistency (McDonald's omega (ω) > 0.70; [22].Strong correlations (r > 0.50) were also expected between the symptom levels for each criteria set assessed at the first and second time point, reflecting temporal stability.To test convergent validity, we examined to what extent DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD levels were related to neighboring syndromes.Based on the findings of systematic reviews [23,24], we expected that summed item scores for each criteria set would be strongly positively associated with posttraumatic stress disorder (PTSD) and depression severity (r > 0.50).Furthermore, we expected that these summed item scores would be higher for women (vs.men), relatively lower (vs.higher) educated people, more recently (vs.less recently) bereaved people, people who lost a more closely (vs.more remotely) related loved one, and people confronted with unnatural (vs.natural) deaths [4,16,23,25].Probable caseness (i.e., the percentage of people meeting diagnostic criteria) for each criteria set was calculated using respective diagnostic scoring rules for each set.Lastly, we determined the optimal cut-off scores to distinguish between people meeting vs. not meeting criteria for probable cases of DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD.

Participants Sample 1: community bereaved sample
Sample 1 was a community sample of Dutch adults bereaved of a family member or other close person due to various causes.Data were collected as part of an ongoing longitudinal survey study on characteristics and correlates of post-loss mental health, started in May 2019.Participants were recruited via advertisements on the content network of Google AdWords and via a website of a national organization of psychologists.Advertisements linked directly to an information page about the goals and procedure of the study.Data of 278 adults, bereaved at least one year earlier were used in the current study, of which 87 had already provided data at a six month follow-up.Participants signed informed consent forms and this study was approved by the ethics committee of the University of Groningen.

Participants Sample 2: people who lost loved ones in a traffic accident (TrafVic)
Sample 2 consisted of people who lost loved ones in a traffic accident.Data were collected in the on-going "TrafVic project" examining the consequences of, and care after, the death of a loved one due to a traffic accident [26,27].Recruitment for an online survey study took place between December 2018 and April 2020.Dutch adults who lost a spouse, family member, or friend due to a traffic accident at least 12 months earlier were recruited for this study.Two-hundred-seventy-three people completed the survey.Data from three participants were excluded because their loss occurred less than 12 months earlier yielding a sample size of 270.Recruitment of participants took place using various strategies.The majority (n = 220; 82%) were recruited via Victim Support (a Dutch organization offering practical, judicial, and emotional support to trauma victims); 21 (8%) were recruited via social media (e.g., Facebook) and another 21 (8%) were recruited via a family member or friend who pointed them to our study.The remaining eight (3%) participants were recruited via other pathways.This study was approved by the ethics committee of the University of Groningen and participants signed an informed consent form.

TGI-SR+
The TGI-SR is an 18-item self-report questionnaire to assess (partially) overlapping symptoms of PCBD as per DSM-5 and earlier proposals of PGD by Prigerson et al. [20].The 18 items of the TGI-SR are presented in Supplemental Materials Table 1 (item 1 through item 18).The TGI-SR was developed before the release of the ICD-11 [9] and included some, but not all current ICD-11 PGD criteria (for an overview see [13]).After inspection of the TGI-SR items, we concluded that eight of twelve ICD-11 PGD criteria map onto TGI-SR items.In order to assess all 12 ICD-11 PGD criteria, LL, GS, and PB wrote four additional items after discussion about the item content (see Supplemental Materials Table 1, item 19 through item 22).This expanded measure was called the TGI-SR+.In the current study, we used the Dutch TGI-SR+.The TGI-SR+ can be downloaded for free in different languages, for instance Dutch, English, French, German, Greek, Norwegian, Swedish, and Turkish, from (https://osf.io/rqn5k/) In 2020, the DSM steering committee proposed to change the 16 DSM-5 PCBD criteria from section III (i.e., "Conditions for further study") to 10 PGD criteria for inclusion in section II of the DSM-5-TR [7].After inspection of these criteria, we found that all DSM-5-TR PGD criteria are covered by the TGI-SR+ items (see Supplemental Materials, Table 1).
Participants from Sample 1 and Sample 2 rated to what extent they experienced each symptom during the past month on 5-point Likert scales with 1 = never, 2 = rarely, 3 = sometimes, 4 = frequently, and 5 = always.Following prior research [20], a symptom was considered endorsed when rated "frequently" or "always".
To meet DSM-5-TR PGD criteria [7], at least one of the two Criterion B (i.e., separation distress) items (item 1 and 3), at least three of the eight Criterion C (cognitive, emotional, and behavioral) symptoms, and the Criterion D (i.e., functional impairment) item (item 13) should be endorsed.All Criterion C symptoms are tapped by one of the TGI-SR+ items (items 6, 9, 10, 11, 18, 19, and 21), except for one symptom (C4 criterion: "Intense emotional pain (e.g., anger, bitterness, sorrow) related to the death"), which is captured by two TGI-SR+ items (items 2 and 8).The highest score on one of these two items is therefore used to represent the C4 criterion.
Probable cases for ICD-11 PGD are identified by the endorsement of at least one Criterion B (i.e., separation distress) symptom (items 1 and 3), at least one Criterion C (cognitive, emotional, and behavioral) symptom (items 2, 5, 8, 9, 10, 16, 19, 20, 21, and 22; [9,28]), and the Criterion D (i.e., functional impairment) item (item 13).Prior research has suggested that this diagnostic scoring rule is too liberal because it results in relatively high prevalence rates and low diagnostic agreement with DSM-5 PCBD criteria.A more conservative scoring rule of at least five additional symptoms resulted in the highest agreement rates with DSM-5 PCBD criteria [15,16].We therefore applied both the liberal and conservative scoring rules to identify probable ICD-11 PGD cases.

Depression
In Sample 1, symptoms of depression were assessed with the Quick Inventory of Depressive Symptomatology-Self Report (QIDS-SR) [29,30].Its 16 items represent the nine symptom domains of a major depressive disorder as per DSM-5 (APA, 2013).Participants rated each item (e.g., "Energy level") on 4-point scales with anchors 0 (e.g., "There is no change in my usual level of energy") through 3 (e.g., "I really cannot carry out most of my usual daily activities because I just don't have the energy") that best described how they felt during the past seven days.Psychometric properties of the QIDS-SR are adequate [30].
McDonald's omega in the current study was 0.84.
In Sample 2, the depression subscale of the Hospital Anxiety and Depression Scale (HADS-D) was used [31,32].It consists of seven items (e.g., "I feel cheerful") rated on 4-point scales ranging from 0 (e.g., "Hardly at all") through 3 (e.g., "Definitely as much") during the past week.The HADS-D has been shown to be a reliable and valid measure to assess severity and caseness of depression in a variety of samples [33].In Sample 2, the McDonald's omega was 0.92.

Posttraumatic stress
Only in Sample 2, PTSD levels were assessed, using the PTSD Checklist for DSM-5 (PCL-5) [34,35].Participants reported how often they were bothered by each of the 20 symptoms (e.g., "Being 'superalert' or watchful or on guard?") on 5-point Likert scales with anchors 0 = not at all and 4 = extremely.References to the "stressful event" in the instruction and items were replaced by "the death of your loved one(s) due to a traffic accident".The PCL-5 has shown to be a reliable and valid measure [34]; in our Sample 2, ω = 0.93.

Statistical analyses 2.4.1. Factor structure
The factor structure of items representing symptoms of DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD, respectively, were examined using confirmatory factor analyses (CFAs) for Sample 1 and Sample 2 separately.We used confirmatory, instead of exploratory, factor analysis, because we had a priori hypotheses about the factor structure based on prior research [14][15][16]19].Supplemental Materials Table 1 shows how TGI-SR+ items map onto symptoms of each criterion set.The analyses were performed in MPlus version 8.4 [36].The following fit statistics were evaluated [37]: Comparative Fit Index (CFI) and Tucker Lewis Index (TLI), with values >0.90 representing acceptable fit (and values >0.95 representing excellent fit), and root-mean-square error of approximation (RMSEA) and standardized root mean square residual (SRMR), with values <0.10 representing acceptable fit (and values <0.05 excellent fit).The statistical fit of nested models was compared using chi-square difference tests.Information criteria were compared between nested models, whereby lower Akaike and Bayesian information criteria (AIC and BIC) reflected better fit.Lastly, parsimony was taken into account for model selection, whereby less complex models with less parameters were preferred over more complex models [37].No items were missing for the TGI-SR+ in Sample 1 and Sample 2. Based on absolute kurtosis (〈< 10) and skewness (< 3) values, univariate normal distribution of the data was assumed.Therefore, the maximum likelihood estimator was used in the CFAs.
For DSM-5 PCBD, we compared the fit of a unidimensional model, a 2-factor model (symptom clusters B and C), and a 3-factor model (with symptoms C1 through C6 representing a second factor and symptoms C7 through C12 loading on a third factor).For DSM-5-TR PGD, a unidimensional model and a 2-factor model (symptom clusters B and C as separate factors) were tested.For ICD-11 PGD, we compared the fit of a unidimensional model with a 2-factor model (symptom clusters B and C as separate factors).See Table 1 Supplemental Materials for an overview.

Temporal stability
The temporal stability of summed scores of the DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD items were examined using data from 87 participants from Sample 1 who completed the TGI-SR+ twice, with a six-month interval.Test-retest analyses were conducted using Pearson correlation analyses.

Convergent validity Sample 1 and Sample 2
Pearson's correlations were calculated to examine the associations of the summed scores of the items representing DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD symptoms with depression levels (Sample 1 and 2) and posttraumatic stress levels (Sample 2).

Known-groups validity
t-tests and correlation analyses were conducted to examine differences in severity levels of DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD in relation to background variables (gender and educational level) and loss-related variables (time since loss, relationship to the deceased (Sample 1 and 2), and cause of loss (Sample 1 only)).

Rates of cases vs. non-cases using diagnostic scoring rules
Percentages of people meeting criteria for probable caseness of DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD were calculated using diagnostic scoring rules as explained in Section 2.3.1.above.

Determining possible clinical cut-off
Receiver Operating Characteristic (ROC) analyses were performed to determine the optimal cut-off score on the summed score of all 22 TGI-SR+ items for the identification of probable cases of DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD, respectively.Optimal cut-off scores were also calculated for the summed scores of the items included in each criteria sets.In a ROC curve, the true-positive rate (i.e., sensitivity) is plotted against the false-positive rate (i.e., 1-specifity) for each possible cut-off score.A Youden's Index (i.e., sensitivity rates -(1-specificity rates)) below 0.70 represents a poor accuracy of the respective score for distinguishing between (probable) caseness and non-caseness.Values between 0.70 and 0.80 are considered fair, between 0.80 and 0.90 are considered good, and between 0.90 and 1 are excellent [38].In all ROC analyses, we combined the data from Sample 1 and Sample 2 to increase the sample size (n = 548).

Sample characteristics
Table 1 shows characteristics of the two samples.In Sample 1, nine out of ten participants were female.The sample was on average middleaged and about half of the sample had a university degree.Three out of four people lost a loved one due to a natural cause.In about half of the cases the deceased loved one was a partner.Mean time since loss was 2.7 years.In Sample 2, three out of four participants were female.They were middle-aged on average and four out of ten completed university.In 10% of the cases, people lost multiple loved ones due to a traffic accident.The most common type of loss was the loss of a child.On average, the loss took place five years earlier.

Factor structure
Table 2 shows the fit indices for all factor models for Sample 1 and Sample 2. Standardized factor loadings for the best fitting models are reported in Supplemental Materials Table 2.In Sample 1, the 3-factor PCBD model yielded the best fit (2-factor model over 1-factor model: Δ χ 2 (Δ DF) = 128.58(1),p < .001;3-factor model over 2-factor model: 31(2), p < .001).However, almost all fit indices of the 3-factor PCBD model were not acceptable.Modification indices indicated a strong association between the error-terms of items B3 ("intrusive thoughts or images related to the person") and B4 ("intrusive thoughts and images related to circumstances of death").We presumed that these correlations reflected non-random measurement error stemming from content overlap.Accordingly, we compared a final model in which these error-terms were allowed to correlate with a model without these correlated error-terms.The model with correlated error terms yielded an acceptable fit and a significantly better fit than the model without the correlated errors (Δ χ 2 (Δ DF) = 83.54(1), p < .001).The three DSM-5 PCBD factors represent separation distress (factor 1), reactive distress (factor 2), and social/identity disruption (factor 3).For DSM-5-TR PGD, the 1-factor and 2-factor model had similar fit indices.The 2-factor DSM-5-TR model did not show a better fit than the 1-factor model (Δ χ 2 (Δ DF) = 1.53 (1), p > .05).We therefore selected the most parsimonious 1-factor model as optimal solution.Considering the ICD-11 PGD model, the 2-factor ICD-11 PGD model had a better fit than  the 1-factor model (Δ χ 2 (Δ DF) = 4.26 (1), p < .05).However the CFI, TLI, and RMSEA values were the same for both models and were all below the threshold of acceptable fit.Also considering the high association between the two factors (r = 0.93, p < .001), the 1-factor model was retained.Modification indices indicated that strong correlations existed between the error-terms of some of the ICD-11 PGD items. 1 We compared the 1-factor model in which these error-terms were allowed to correlate with the 1-factor model without these correlated error-terms.The model with correlated error terms yielded acceptable fit estimates and fit significantly better than the model without the correlated errors (Δ χ 2 (Δ DF) = 151.69(6), p < .001).
In Sample 2, the 3-factor DSM-5 PCBD model had the best fit as evidenced by acceptable CFI, TLI, and RMSEA values, an excellent SRMR value, the lowest AIC and BIC values, and a significant Chi-square dif-

Internal consistency
For Sample 1, McDonald's omega values were 0.93, 0.92, and 0.92 for TGI-SR+ items measuring DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD symptoms, respectively.In Sample 2, these values were 0.94, 0.92, and 0.93, respectively.Omega values for the three subscales corresponding with the three factors of PCBD were 0.87, 0.83, and 0.89 for Sample 1 and 0.86, 0.81, and 0.90 for Sample 2. These findings support the internal consistency of the TGI-SR+ items.

Table 3
Pearson correlations between symptom levels of disturbed grief and PTSD and depression.

Temporal stability
In a subsample of Sample 1 (n = 87) strong associations were found between the scores at two time points six months apart for items assessing DSM-5 PCBD (r = 0.77), DSM-5-TR PGD (r = 0.78), and ICD-11 PGD (r = 0.77, all ps < 0.001).The findings demonstrate the temporal stability of the TGI-SR+.

Convergent validity
Table 3 shows the associations of the summed scores of the 16 items representing DSM-5 PCBD symptoms, 10 items representing DSM-5-TR PGD symptoms, and 12 items representing ICD-11 PGD symptoms with symptom levels of depression (Sample 1 and 2) and PTSD (Sample 2 only).All associations were positive, strong, and significant.

Known-groups validity
For Sample 1, people who were relatively lower educated, more recently bereaved, lost a spouse or child (vs.other relative or close person), due to suicide, accidents, or homicide (vs.natural causes) reported significantly higher summed scores on items measuring DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD.Women reported higher DSM-5-TR PGD scores than men (see Table 4).
For Sample 2, we found that lower (vs.higher) educated people, more recently bereaved people, and people who lost a spouse/child (vs.other relative) reported significantly higher summed scores on items assessing DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD.These scores did not differ by gender (see Table 4).

Determining cut-off scores for probable caseness
We combined the data from Sample 1 and Sample 2 (N = 548) for determining the optimal cut-off scores.See Supplemental Table 3-6 for a summary of outcomes of the ROC analyses.The optimal cut-off when using the TGI-SR+ total score (using all 22 items) was ≥75 for DSM-5 PCBD (AUC = 0.952 (95% CI: 0.934-0.970)).With this score, 94% of the PCBD cases were correctly identified and 15% incorrectly identified as a PCBD case; Youden's index was good J = 0.82.For DSM-5-TR PGD the optimal cut-off score was ≥71 when using TGI-SR+ total score (AUC = 0.925 (95% CI: 0.903-0.946)).Using this score results in 89% correctly identified DSM-5-TR PGD cases and 17% incorrectly identified cases (J = 0.71; this is fair).A total score on the TGI-SR+ of ≥75 was the optimal score to identify ICD-11 PGD cases, when applying the conservative diagnostic scoring rule of at least five additional symptoms (AUC = 0.956 (95% CI: 0.941-0.972)).With this score, 92% of the ICD-11 PGD cases were correctly identified and 10% incorrectly identified as a ICD-11 PGD case.The Youden's index was good J = 0.82.When using  the liberal diagnostic scoring rule of at least one addition symptom for ICD-11 PGD, the optimal cut-off score was ≥71 (AUC = 0.899 (95% CI: 0.873-0.925)).With this score, 82% of the ICD-11 PGD cases were correctly identified and 18% incorrectly identified as a ICD-11 PGD case.The Youden's index was poor J = 0.65.We also calculated optimal cut-off scores for the identification of people meeting criteria for the three grief disorders, when only summing up items representing symptoms DSM-5 PCBD (scores ranging from 16 through 80), DSM-5-TR PGD (scores ranging from 10 through 50), or ICD-11 PGD (scores ranging from 12 through 60); this excluded the functional impairment item (item 13).A score of ≥53 yielded the optimal cut-off for distinguishing probable PCBD cases from non-cases, ≥33 for DSM-5-TR PGD, and ≥ 41 for ICD-11 PGD (when applying the conservative diagnostic scoring rule of at least 5 additional symptoms; and ≥ 40 when applying the liberal scoring rule).See Supplemental Table 7-10 for details.In Supplemental Table 11 probable caseness based on cut-off scores are displayed.

Discussion
The aim of the present research was to evaluate psychometric properties of the TGI-SR+, a new self-report measure to assess symptoms of DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD.Data were derived from a community sample of people who lost loved ones to various causes and a sample of people who lost loved ones in traffic accidents.We started with examining the factor structure of items representing criteria for DSM-5 PCBD, DSM-5-TR PGD, and ICD-11.As expected, in both samples, the 3-factor DSM-5 PCBD model, 1-factor DSM-5-TR PGD, and 1-factor ICD-11 PGD model yielded adequate fit.These findings accord with prior factor analytic research [14][15][16]18,19].In both samples, the items representing DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD demonstrated good internal consistency (ω > 0.90).Furthermore, in Sample 1, evidence was found that the TGI-SR+ has sound test-retest reliability.
Strong associations were found between TGI-SR+ scores and depression and PTSD symptoms.These findings attest to the convergent validity of the TGI-SR+ and are largely consistent with prior research [23,24] including factor analytic and latent class analytic studies demonstrating that disturbed grief overlaps with, yet is distinguishable from these neighboring syndromes [39][40][41][42][43][44].
Known-group validity was also demonstrated as the three criteriasets differentiated well between relevant subgroups of bereaved people as determined by reviews of risk-factors of disturbed grief [4,23,25].As expected, we found that having a lower education level, being more recently bereaved and the loss of child/spouse (vs.other relationship) were related to higher levels of disturbed grief in both samples.For Sample 1, we also found that an unnatural loss (vs.natural loss) was related to higher disturbed grief levels.We could not confirm that scores on the criteria sets consistently differed between male and female participants.Only in the community sample, but not in the traffic bereaved sample, did females show higher PGD DSM-5-TR symptom levels than males.It is likely that the relatively small size of these effects, together with the low number of males in both samples led to power problems in detecting gender differences consistently.
We continued with calculating probable caseness of DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD.In Sample 1 and Sample 2 the lowest prevalence rates were found for DSM-5 PCBD (25% and 15%, respectively), followed by DSM-5-TR PGD (32% vs. 30%, respectively).For ICD-11 PGD two scoring rules were used.Not unexpectedly, the more liberal scoring rule (requiring ≥1 additional symptom) resulted in the highest prevalence rates (34% and 33%, in Samples 1 and 2, respectively).The more conservative rule (≥ 5 additional symptoms), resulted in prevalence rates of ICD-11 PGD of 27% and 24%.A recent review of studies comparing prevalence rates across criteria sets of disturbed grief also concluded that PCBD is a relatively conservative algorithm, which only corresponds with PGD prevalence rates when a conservative scoring rule is used [45].Whereas some authors found that five additional ICD-11 PGD criteria yielded most correspondence between both symptom sets [16], others found that six additional criteria yielded most correspondence [46].In this study, it also appears that an (even) more conservative scoring rule would have yielded the best agreement in what might classify as disturbed grief.This is one of the first studies to examine the prevalence of the newly revised PGD DSM-5-TR criteria set.It appears to be a relatively lenient criteria set, corresponding more strongly with the prevalence rates of the lenient version PGD ICD-11 rather than the conservative version of PGD ICD-11 and PCBD.
These observations are also reflected in our analyses of the determination of the best possible cut-off score to define probable caseness of DSM-5 PCBD, DSM-5-TR PGD, and ICD-11 PGD, using the combined data from Sample 1 and 2 (N = 548).For determination of caseness of DSM-5 PCBD and ICD-11 PGD (with ≥5 additional symptoms), the optimal cut-off score when summing up all 22 TGI-SR+ items was ≥75; for DSM-5-TR PGD it was somewhat lower, namely ≥71.In addition, findings indicated that a cut-off of ≥71 was also the optimal score using the liberal diagnostic scoring rule for ICD-11 PGD (with ≥1 additional symptom), however this resulted in poor specificity and sensitivity rates.It should be noted that using these cut-off scores resulted in relatively higher rates of probable cases compared with rates when using the diagnostic scoring rules.Pending replication of our findings, we advise to use the diagnostic scoring rules to determine probable caseness, instead of the cut-off scores.When cut-off scores are used, we advise to use a cut-off score of ≥71 when determining probable caseness of DSM-5TR PGD and ICD-11 PGD and ≥ 75 for DSM-5 PCBD.

Strengths and limitations
Strengths of this study include the use of two large bereaved samples with different compositions to conduct a comprehensive assessment of reliability and validity of the TGI-SR+.However, there were also some limitations to consider.First, both samples were voluntary response samples resulting in an overrepresentation of middle-aged females that typically participate in grief research [47].This has likely influenced some of the results in this study.For example, the low numbers of men may have affected our power to detect gender differences in disturbed grief symptoms.A second limitation is that we only validated the TGI-SR+ in the Dutch language.To ensure wide usage and uptake of this new instrument, we have included translations of this scale in multiple languages (see https://osf.io/rqn5k/).Prior to the use of these instruments, we recommend conducting validation studies of the TGI-SR+ in these and other languages.Over time, this may enable cross-cultural examinations of different criteria sets of disturbed grief using a single validated measure.Third, some aspects of the validity of the TGI-SR+ were not assessed in our study should be examined further.For instance, a further examination of its predictive validity using large-scale longitudinal designs and relevant outcome measures (e.g., quality of life, functional impairment) appears warranted.

Implications
The validation of the TGI-SR+ forms a critical step to enable systematic research into the characteristics of contemporary criteria sets for disturbed grief.It offers a time-efficient way of assessing all relevant disturbed grief symptoms derived from current and forthcoming versions of the ICD and the DSM.Clinicians may find it helpful to get a first indication of the levels of disturbed grief experienced by their patients, aligning with the diagnostic system that is used in their country.Researchers are likely to benefit from an instrument that allows for comparisons of different disturbed grief symptom sets across studies.Moreover, the free availability of the TGI-SR+ in multiple languages will enable validation studies within and across different languages, which in turn will facilitate investigations on the characteristics of grief disorders across cultures [48].

Conclusions
In summary, the present study demonstrated that the new TGI-SR+ is a reliable and valid self-report instrument to comprehensively assess the DSM-5 PCBD, ICD-11 PGD and DSM-5-TR PGD criteria sets.Specifically, we demonstrated that the factor structures of these criteria sets align with prior findings, demonstrating construct validity of the TGI-SR+.Moreover, our results support the TGI-SR+'s internal consistency and temporal stability, as well as convergent validity and known-groups validity.These promising results suggest that the TGI-SR+ can be applied by both clinicians and researchers to assess disturbed grief across different contexts.In the long run, we hope that this study will provide an important first step towards systematic, comparative research on the antecedents, characteristics, and consequences of disturbed grief across languages and cultures.

Role of the funding source
The work on people who lost loved ones in a traffic accident (Sample 2) was funded by Fonds Slachtofferhulp.Maarten C. Eisma was supported by a Netherlands Organization for Scientific Research (NWO) Veni grant [Grant ID: 016.veni195.113].The funders did not play a role in the study design, collection, analysis or interpretation of the data, in the writing of the report or in the decision to submit the article for publication.

Note. DSM- 5 =
5th edition of the Diagnostic and Statistical Manual of Mental Disorders; DSM-5-TR = 5th edition of the Diagnostic and Statistical Manual of Mental Disorders Text Revision; ICD-11 = 11th edition of the International Classification of Diseases; N.a.= not applicable; PCBD = persistent complex bereavement disorder; PGD = prolonged grief disorder.

Note. DSM- 5 =
5th edition of the Diagnostic and Statistical Manual of Mental Disorders; DSM-5-TR = 5th edition of the Diagnostic and Statistical Manual of Mental Disorders Text Revision; ICD-11 = 11th edition of the International Classification of Diseases; PCBD = Persistent complex bereavement disorder; PGD = Prolonged grief disorder; PTSD = Posttraumatic Stress Disorder.a Five people were excluded because they did not complete the depression measure.* p < .001.

)
Note.DSM-5 = 5th edition of the Diagnostic and Statistical Manual of Mental Disorders; DSM-5-TR = 5th edition of the Diagnostic and Statistical Manual of Mental Disorders Text Revision; ICD-11 = 11th edition of the International Classification of Diseases; PCBD = Persistent Complex Bereavement Disorder; PGD = Prolonged Grief Disorder; S1 = Community bereaved sample; S2 = Sample of people who lost loved ones in a traffic accident.*** p < .001;** p < .01;* p < .05.