Psychometric analysis of the Psychiatric Diagnostic Screening Questionnaire (PDSQ) and determinants of psychopathology in two outpatient clinics in Navarre (Spain)

ABSTRACT Background: The self-report Psychiatric Diagnostic Screening Questionnaire PDSQ is designed to screen Axis I psychiatric disorders. We aim to determine its psychometric properties in Spanish outpatients and assess its relationship with two interviews (for psychopathology and for personality disorders) and clinical/demographic variables. Methodology: We administered the study questionnaire, the Mini International Neuropsychiatric Interview Plus (MINI-Plus), the Standardised Assessment of Personality Abbreviated Scale (SAPAS), and the List of Threatening Experiences Questionnaire (LTE-Q) to 375 patients at two public outpatient centres. Reliability of the study questionnaire was evaluated (Cronbach’s alpha, α) and known-group validity measured by comparing groups based on demographic and clinical variables (binary logistic regression analysis) and MINI-Plus diagnoses (Mann-Whitney U). The diagnostic accuracy of the study questionnaire score was analysed taking the MINI-Plus diagnoses as the gold standard (ROC analysis). Results: Internal consistency was adequate across all PDSQ scales (α >0.7; mean α=0.85). Known-group comparisons were satisfactory. Female and male patients showed higher prevalence of internalizing and externalizing diagnoses, respectively. Younger age, more life events and limitations, higher SAPAS scores, and lower economic levels were linked to a greater number of PDSQ diagnoses. Inter-group differences were found for all PDSQ scales based on the corresponding MINI-Plus diagnoses. Mean values of sensitivity, AUC, and negative predictive value were 88.7, 0.82, and 96.7, respectively. Conclusions: When applied to a sample of Spanish outpatients, the PDSQ exhibits satisfactory psychometric properties and adequate relationships with the psychopathology and personality interviews, and clinical and demographic variables. The study questionnaire is suitable for assessing comorbidity and psychopathology dimensions.

Psychometric analysis of the Psychiatric Diagnostic Screening Questionnaire (PDSQ) and determinants of psychopathology in two outpatient clinics in Navarre (Spain) Estudio psicométrico del Psychiatric Diagnostic Screening Questionnaire (PDSQ) y determinantes de psicopatología en dos centros ambulatorios de Navarra (España INTRODUCTION A high percentage of patients attending Mental Health Services have more than one psychiatric diagnosis 1 .Reliable psychiatric diagnosis of comorbidity is key to provide adequate treatment 2 .
Clinicians tend to under-recognize comorbidity in mental health routine settings when unstructured clinical interviews are used in comparison to structured research evaluations 2,3 .
Semi-structured interviews, such as the Mini-International Neuropsychiatric Interview (MINI) 4 or the Structured Clinical Interview for DSM (SCID) 5 , are considered gold standards for assessing comorbidity, but they may take more time than available in daily clinical work 6 .Screening scales, like the DSM-5 Cross-Cutting Symptom Measure 7 , may offer better comorbidity assessment than unstructured clinical interviews 3,8 .
The Psychiatric Diagnostic Screening Questionnaire (PDSQ) is a self-report instrument designed to screen for the most common Axis I psychiatric disorders (as outlined in the DSM IV) in outpatient mental health settings.It is based on the most-often recorded and reported diagnoses in community surveys and clinical samples.The developers of the PDSQ aimed to address the under-recognition of comorbidity in daily clinical practice associated with unstructured interviews [8][9][10][11] .It is understood that PDSQ cannot offer the same degrees of reliability and validity as structured interviews when making diagnoses.However, it offers a reasonable estimate of the overall prevalence of commonly encountered conditions 3 .
The PDSQ is intended to be administered and scored before the patient's initial visit with the clinician in order to improve the efficiency of the diagnostic evaluation by guiding clinicians towards areas that require more assessment 8,11,12 .
The PDSQ is considered a good screening and diagnostic aid due to its capacity to identify comorbidities.It shows good psychometric properties, described by the authors of the scale 2,8,10,11,13,14 and in different countries 15,16 .The PDSQ has been translated into Spanish using a forward-backward translation process and validated for its use in Spanish alcohol-dependent patients 6 .
The PDSQ shows a positive relationship with other diagnostic instruments such as the SCID 8,16 and the Personality Assessment Inventory (PAI) 9 , and has been employed to study the validity of other instruments assessing depression and anxiety 17 .
The PDSQ has been administered in: 1) comorbidity studies of patients reporting substance use 15 , heavy youth drinking 18 , or borderline personality disorder 19 ; 2) studies of pregnant and postpartum women 20 and patients suffering from pain 21 ; and 3) studies of patients with specific mental health disorders such as panic disorder 22 , psychotic symptoms 23 , subclinical generalized anxiety disorder 24 , post-traumatic stress 25 or suicidal ideation 26 .It has also been used to create a network structure of self-reported psychopathological dimensions in common mental disorders 27 .
Although empirical support for the PDSQ has grown, a broader empirical grounding is needed to comprehensively establish its validity 9 .The authors of the PDSQ 8 , for example, recommend replicating the psychometric study in public community health centres because it was originally developed in a private adult practice, where there may have been a lower prevalence of other diagnoses such as chronic and persistent mental illness or patients with lower economic status.Perkey et al. 9 suggest that further research should focus on convergent and divergent validity with other established psychopathology instruments introducing criteria such as life space data (e.g., education) and personality measures.
In this study, we aim to determine the psychometric properties of the PDSQ when applied to a sample of Spanish outpatients from two public centres.We also explored the relationship between the PDSQ and two established interviews for psychopathology (MINI-Plus) and for personality disorders (SAPAS), as well as the relationship between the PDSQ and clinical and demographic variables METHODOLOGY Participants Participants were a consecutive sample of patients who attended one of two outpatient centres of the Mental Health Network of Navarre between June 2012 and March 2016.The inclusion criteria were: patients older than 16 years, referred from the General Practitioner consultation, and undertaking their first consultation at either of the two participating mental health centres.Patients with organic mental disorders or whose cognitive levels prevented them from completing the questionnaires were excluded.

Data-collection procedures and instruments
All patients who met the inclusion criteria were interviewed, given oral and written information about the study, and invited to participate.Patients who agreed to participate and signed the informed consent were asked to complete three self-reported scales (without any medical professional being present) before their initial diagnostic evaluation consultation at the outpatient centre.
1) The Psychiatric Diagnostic Screening Questionnaire (PDSQ) 8 analyses current and recent symptoms (previous two weeks and six months before evaluation), as well as whether the patient has ever experienced or witnessed a traumatic event.
The PDSQ is a 125-item instrument consisting of 13 scales that allows obtaining information from six areas (Appendix I).The resulting global score (from 0 to 100) provides information on the severity of the overall psychopathology.Participants responded to each item using a yes/no format (yes=1 / no=0).Each scale is scored as a dichotomous variable indicating the presence or absence of the diagnosis based on the cut-off points established by the authors or as a continuous variable (that can be converted to a 0-100 scale), where higher scores indicate a greater number of symptoms on that scale 8 ; thus, scales may be considered psychopathology dimensions.
2) The Standardised Assessment of Personality, Abbreviated Scale (SAPAS) 28 is is an eight-item interview used to screen the presence of personality disorders, rather than the specific type of these.Each question is to be answered with a yes or no (yes =1 / no =0).The resulting score ranges from 0 to 8; a higher score indicates a greater likelihood of having a personality disorder.A score ≥ 3 indicates the presence of a personality disorder according to the SCID 30 .
3) The List of Threatening Experiences Questionnaire (LTE-Q) 29 , or Brugha's scale, assesses the presence of major stressful life events with considerable longterm contextual threat in the preceding six months.This instrument has been validated in Spain 31 .It comprises 12 items with yes/no answers (yes =1 / no =0).Total scores range from 0 to 12 (higher scores indicate a greater number of life events).The LTE-Q also assesses the degree of limitation or negative consequences (none =0, low =1, middle =2, and high =3) caused by each event, providing a global value (0 to 36); higher scores indicate a greater number of events and degree of limitations.
During this first consultation, the treating psychiatrists (who have previously been trained in administering the PDSQ) also administered the Mini-International Neuropsychiatric Interview-Plus (MINI-Plus 5.0.0) 4 , a structured and standardized diagnostic interview used to determine the presence of the 23 most common Axis I psychiatric disorders (DSM-IV-TR and ICD-10).The MINI-Plus is the MINI version created for research.Once the interview concluded, the interviewers indicated the presence or absence of any disorder.For this study, we administered the Spanish version of the questionnaire 32 .
The PDSQ has been previously assessed in a sample of Spanish alcoholic patients.However, we wanted to know whether the wording of the items was adequate for other diagnostic groups.Thus, the first 40 patients were invited to complete a short debriefing questionnaire on the time they had taken to complete the PDSQ instrument, and asked if they had found any of the items confusing or upsetting.
The following socio-demographic data were obtained from study participants: -gender: female, male; -age (years); -household composition: living alone, living with family, shared home, residence; -educational attainment: less than compulsory, compulsory, post-compulsory below university, university; -place of birth: Spain, other; -economic status: low, medium-low, medium, high.The study was approved by the Drug Research Ethics Committee of Navarre and conducted in accordance with the ethical principles outlined in the Declaration of Helsinki.

Statistical analysis
Clinical and demographic characteristics and questionnaire scores were presented as frequency and percentage or mean and standard deviation (SD).
The internal consistency reliability of the scales was measured by Cronbach's alpha (a); the internal consistency was considered as adequate if a >0.70 33 .
Assessment of questionnaire validity was performed by known-group comparison to discriminate between subgroups of patients.First, we used binary logistic regression analysis using PDSQ di-chotomous scores (which indicate the presence/ absence of a diagnosis) as dependent variables.
Explanatory (independent) variables were age, gender, economic level (two groups: 1= medium-low, medium and high; 2= low), stressful life events and limitations caused by these events (LTE-Q scores), and personality (SAPAS scores).
A higher age was expected to be related to less anxiety and drug abuse/dependence, and more major depressive disorders 1,34,35,36 .A higher number of internalizing diagnoses (major depressive disorder, anxiety, bulimia, hypochondriasis, and somatoform disorders) were expected in female patients and a higher number of externalizing diagnoses (alcohol and drug abuse/dependence) in male patients 34,35,37 .A higher rate in a PDSQ diagnosis was expected to associate to a lower economical level 38 , a greater number of life events and limitations caused by these events 39,40 , and a greater likelihood of a diagnosis of personality disorder in general 41,42 .
Known-group comparison was also performed by comparing the PDSQ scores in each scale of the questionnaire (considered continuous variables ranging from 0 to 100) for groups that presented the corresponding diagnoses in the MINI-Plus interview.The Mann-Whitney U test was performed because PDSQ scores did not follow a normal distribution (Kolmogorov-Smirnov, p <0.001 for all PDSQ areas); effect size was calculated based on Cohen's D.
PDSQ diagnoses were compared to those obtained by the MINI-Plus interview (gold standard).Diagnostic accuracy of the PDSQ was analysed by means of sensitivity, specificity, positive and negative predictive values (PPV, NPV), and area under the curve (AUC) from ROC analysis.The authors of the PDSQ suggest a sensitivity ≥ 90% for using the scale in clinical practice 8 .

RESULTS
Of the 394 patients invited to participate in the study, 375 (95.2%) completed the PDSQ.The reasons for not completing the questionnaire were patient refusal (n=10), not having glasses (n=4), difficulty in understanding Spanish (n=2), lack of time (n=2), and intellectual disability (n=1).All patients who completed the PDSQ, answered over 95% of the items.When an answer was missing, we selected the most frequent answer (yes/no) in their corresponding scale of the PDSQ.Overall, it took patients under 20 minutes to complete the PDSQ and no item was considered upsetting or confusing.Three patients who had a diagnosis different from psychosis were surprised to be asked about psychotic symptoms because they considered those specific questions had nothing to do with their situation.However, this was not sufficient reason to change the wording of those items, since the patients did not report being upset by them.
Mean age of study patients was 40.2 years (range: 16-86 years), 80.3% of them lived with their family, and 41.1% reported a low economic level.Mean number of stressful life events was 1.3, with a mean SAPAS score of 3.9.Results of the MINI-Plus interview revealed various diagnoses, the most common being major depressive and anxiety disorders (Table 1).

Descriptive statistics of the PDSQ questionnaire
Median number of comorbidities (PDSQ diagnoses per patient) was 4.0: major depressive disorder, the various anxiety disorders (posttraumatic stress, panic, and generalized anxiety), obsessive-compulsive disorder, and somatization disorder (in decreasing frequency).A PDSQ diagnosis of psychosis was found in 28.5 % of the study sample (Table 2).
Total mean score for the PDSQ was 29.9.The highest mean scores (corrected to a scale of 0 to 100) were for major depressive disorder, panic disorder, and generalized anxiety disorder (Table 2).
The percentage of respondents at floor (patients with the lowest scores) was low for major depressive and generalized anxiety disorders, and for total PDSQ score, and high for psychosis and alcohol and drug abuse/dependence.Percentage of responders at ceiling (patients with the highest scores) was low for most scales.The whole range of scores was presented for all individual scales.Total PDSQ score ranged between 0 and 80.8.
Internal consistency reliability was rather good (a > 0.85) for most PDSQ scales except psychosis, somatization, and obsessive compulsive disorder (a > 0.7); mean a= 0.85 (Table 2).Known-group comparison: significant relationships were found among the variables.A higher number of generalized anxiety disorder, social phobia, and drug abuse/dependence diagnosis was seen in younger patients.Gender differences were found for eight scales; higher prevalence of major depressive, anxiety disorders, bulimia, and somatization disorders was observed in female patients, while male patients showed higher prevalence of alcohol and drug abuse/dependence.Lower economic status was associated to higher number of diagnoses in obsessive compulsive disorder and drug abuse or dependence.Higher number of life events and limitations was linked to higher number of diagnoses in nine PDSQ scales.Higher SA-PAS score was related to higher number of diagnoses in nine scales (Table 3).
For patients diagnosed with a disorder on the MINI-Plus interview, the values on the corresponding PDSQ scales were significantly higher (with effect sizes ranging from 0.59 to 2.18) (Table 4).Mean sensitivity of the thirteen PDSQ scales was 88.7%, six of which had a sensitivity >0.90.All scales had a negative predictive value ≥0.90.The av-erage specificity score was 69, and mean AUC 0.82 (Table 5).

DISCUSSION
Here, we present the results of a psychometric study of the PDSQ applied to a sample of Spanish outpatients from two public centres.We have also analysed the relationship between the PDSQ and the MINI-Plus and SAPAS interviews, and clinical and demographic variables.
The high level of completion rate, the low number of missing items, the lack of items considered upsetting or confusing, and the heterogeneity in education levels of the study sample indicates that the questionnaire was well accepted and item presentation adequate.Our results confirm that the PDSQ can be incorporated in outpatient daily clinical practice without causing disruption 9 .The percentage of unanswered items was similar to that reported elsewhere 20 .The time required to complete the questionnaire supports its developers' aim to help clinicians attain accurate comorbidity diagnoses in a time-efficient manner 9 .
The high percentage of patients living with their families may be an issue specific to our country 43,44 .
The high frequency of comorbidities observed in our study is in line with those recorded in prevalence studies for our country 45,46 and supports the PDSQ as a suitable screening tool 9 .These comorbidity levels are probably favoured by the current classification systems (DSM5 and ICD), where symptoms can be common to several disorders.The PDSQ is considered a suitable tool for studying the relationships between diagnoses and the dimensions that may underlie psychopathology 9 , such as those proposed by Watson et al. for emotional disorders 47 .
In this study, the variable comorbidity is used merely to describe the study sample (a population with a high degree of comorbidity).Our sample therefore represents people with this feature and no modelling or testing has been affected by its use.
The highest prevalence and mean scores for mayor depression and anxiety disorders found with the PDSQ are in line with studies on the prevalence of these disorders among the Spanish general population 34,35 .A rather high prevalence of somatization disorder and lower prevalence of psychosis (with a low mean score) were expected, since study patients were attending outpatient centres 48 .The differences in some PDSQ scales (such as obsessive compulsive disorder) in relation to the frequency of cases and mean scores may be related to the low cut-off criterion for these scales.
The higher prevalence of possible diagnoses with the PDSQ compared to the MINI-Plus interview may be explained by the role of the PDSQ as a screening tool for guiding clinicians towards areas that need to be evaluated in detail during consultation 2,8,10 .
The differences in frequency of respondents at floor for the PDSQ scales -low for major depressive and generalized anxiety disorders and high for psychosis, alcohol and drug abuse or dependence -may be due to the fact that the symptoms of the first group of scales can be present in several diagnoses, whereas the second group of symptoms are more diagnosis-specific.These floor levels, combined with the wide range of scores in the PDSQ scales and total score, as well as the low ceiling effects, indicate that the questionnaire may be a good assessment tool when the scales are considered as psychopathology dimensions with different severity levels (from normality to sub-threshold and full diagnosis) 8 .
Reliability analysis results were satisfactory and in line with those found by the developers of the PDSQ and in psychometric studies conducted in Spain and Romania.Although the alpha coefficients of somatization, psychosis, and obsessive compulsive disorder scales were low in some studies, in ours these three scales reached the 0.7 cutoff value for alpha.
Known-group validity analyses are supported by the results from the comparisons of patient subgroups.Studies of the Spanish general population 34,35 and other contexts 49 have reported that prevalence of social phobia decreases as patients grow older.Prevalence of generalized anxiety disorder also decreases with age 37 .
Unlike the results of the present study, a relationship between age and depression has been observed in two studies performed with the Spanish general population , 34,35 .The relationship between increasing age and lower prevalence of drug consumption found in our study is in line with a study conducted by the Spanish Ministry of Heath 36 .No relationship between age and alcohol abuse/dependence was found among the Spanish general population 1 .
The observed gender differences are in line with those found among the Spanish general population, where female patients have a higher risk of internalizing diagnoses (anxiety and mood disorders) and male patients a higher risk of externalizing diagnoses (alcohol abuse/dependence) 1,34,35 .The relationship between economic level and frequency of diagnosis is in line with the results of the Spanish National Health Survey 50 -which reported a higher frequency of diagnosis among lower social classes -and with a study of the Spanish general population, which communicated earning gaps between individuals with and without mental disorders 38 .
In our study, there is a clear relationship between life events and the limitations caused by these events and the presence of a diagnosis.A recent meta-analysis indicates correlations of 0.33 and 0.35 between stressful life events and internalizing and externalizing psychopathology, respectively 39 .Moreover, negative life events have been associated to agoraphobia, panic, generalized anxiety, major depressive and generalized anxiety disorders, as well as alcohol and drug abuse/dependence 40.The relationship between a higher SAPAS score and a higher frequency of diagnosis is in line with some studies; Huang et al. 41,42 found that 51.2% of the patients with a personality disorder were diagnosed with at least one other mental problem.
As in the Romanian validation study 16 , knowngroups validity is also supported by the differences in all inter-group PDSQ scales based on the corresponding MINI-Plus diagnosis.
The sensitivity, specificity, negative predictive values, AUC coefficients, and mean values in our study are satisfactory and in line with those found by the authors of the questionnaire, though in our case the bulimia and psychosis scales have higher sensitivity values (difference ≥ 0.13) and the panic scale lower sensitivity values (difference ≥ 0.12) 8 .High sensitivity and moderate specificity mean values were also reported in the Spanish study with alcoholic patients 6 and the validation study for Romania 16 .
Most PDSQ scales meet the 0.90 sensitivity coefficient recommended by the developers of the questionnaire or were close to it.This threshold, as well as a high negative predictive value and a rather high specificity value, is recommended so that all cases are detected (to avoid false negatives).Although these criteria may favour the presence of false positives, these are considered less of a problem for a screening questionnaire since the main problem is the time clinicians need to deter-mine a lack of a diagnosis 2 .Moreover, the authors of the PDSQ suggest that the presence of false positives may partly be related to the presence of subthreshold forms of the disorders 8 .
There are several strengths to this study: its sample size, the fact that the patients were from public outpatient mental health centres, the analysis with established psychopathology (MINI-Plus) and personality disorder (SAPAS) interviews, and clinical and demographic data.Contrarily, this study could have benefited from a longitudinal design that measured psychopathology (frequencies and dimensions) before and after an intervention, so that responsiveness to changes could be analysed.Further studies should assess psychometric and comorbidity in patients with psychosis to determine the utility of the PDSQ in this population.
In conclusion, the PDSQ has satisfactory psychometric properties when applied to a sample of Spanish outpatients from two public centres in Navarre.It also shows satisfactory relationships with established psychopathology and personality interviews, and clinical and demographic variables.The PDSQ is a suitable tool for assessing a diagnosis and determining the dimension of the psychopathology.

Table 1 .
Sociodemographic and clinical characteristics of the patients of the study *: mean and standard deviation; 1: diagnoses based on MINI-Plus interview, patients may have more than one diagnosis; 2: DSM IV diagnoses (based on MINI Plus interview) not included in the PDSQ scale; the most frequent diagnoses in this group are presented.

Table 2 .
Frequencies in the Psychiatric Diagnostic Screening Questionnaire (PDSQ) scales, reliability and comorbidity Comorbidities: number of PDSQ diagnoses per patient; mean: mean score on the scale (0-100); SD: standard deviation; floor %: percentage of patients who reported the lowest score; ceiling %: percentage of patients who reported the highest score; CI: confidence interval.

Table 3 .
Association between independent factors and presence of each Psychiatric Diagnostic Screening Questionnaire (PDSQ) scale by logistic regression analysis CI: confidence interval; values in bold: a significant relationship was found between an independent factor and a PDSQ area.TLE-Q: List of Threatening Events Questionnaire; SAPAS: Standardised Assessment of Personality, Abbreviated Scale; *: economic level was grouped on two levels: medium-low, medium and high levels (n= 221) and low level (n= 154).
OR: odds ratio;

Table 4 .
Inter-group differences in Psychiatric Diagnostic Screening Questionnaire (PDSQ) scores based on the presence of a diagnosis using the MINI-Plus PDSQ: Psychiatric Diagnostic Screening Questionnaire; MINI-Plus: Mini-International Neuropsychiatric Interview-Plus; N: frequency of patients diagnosed with the MINI-Plus interview in the same category as the PDSQ scale; Yes/No: PDSQ scores of patients with (or without) the corresponding disorder when using the MINI-Plus; SD: standard deviation; *: Cohen's D effect size; all p-values (Mann-Whitney U tests) were < 0.001.

Table 5 .
Diagnostic accuracy of the Psychiatric Diagnostic Screening Questionnaire (PDSQ) and its scales based on the diagnoses using the MINI-Plus as gold standard