Validity of symptom assessment in psychotic disorders: information variance across different sources of history

Abstract

Background: Factors such as poor insight, amotivation, suspiciousness, disorganized speech and attentional problems may interfere with the ability of acutely ill individuals with psychotic disorders to provide a valid account of their symptoms. This study was designed to determine the degree to which history provided by such subjects is consistent with that obtained from other sources. Method: Fifty-five subjects presenting with psychotic disorders were multiply evaluated with a semi structured interview, the Comprehensive Assessment of Symptoms and History (CASH), which includes the Scales for the Assessment of Negative and Positive symptoms (SANS and SAPS). One interviewer assessed the severity of the patient's symptoms in the previous month based solely on information provided by the patient. A second rater made symptom ratings based on information obtained from a “best informant”. Following this, a consensus rating was established based on an extensive evaluation and review of all sources of information. An item-by-item comparison of the three sets of symptom ratings was then quantified by paired t-tests, simple and multiple correlations. Results: Positive and negative symptoms ratings based on the subjects' report were significantly lower than the corresponding consensus ratings. However, those based on the informants' report did not differ from consensus ratings for negative symptoms. Information obtained from subjects and from informants together accounted for a majority of the variance of the consensus ratings. Conclusions: These findings demonstrate that assessment limited to patients' own reports are likely to underestimate psychopathology in acutely ill patients with psychotic disorders. Obtaining corroborative history from a family member may substantially improve the validity of the assessment of negative symptoms.

Introduction

Several factors may interfere with the ability of individuals with schizophrenia and other psychotic disorders to precisely describe their symptoms and provide a valid psychiatric history. Perhaps the most common of these factors is lack of insight, i.e. the subject does not experience the phenomenon in question as a symptom or problem Amador et al., 1993, Amador et al., 1994, David, 1990. Other factors include disorganized speech, suspiciousness, attentional problems, and amotivation, all of which are common in schizophrenia.

Despite an intuitive recognition of this problem, mental health professionals continue to rely heavily on patients' subjective reporting in their diagnostic and phenomenological assessment. This is understandable in light of several factors. First, many aspects of abnormal mental phenomena, such as hallucinations, appear to be inherently subjective and there are simply no better means available to assess these symptoms than to rely on the patient's own report. Second, issues of confidentiality and the need for the subject's consent to release or obtain information from other sources provide a disincentive to include others in the assessment. Perhaps the most important factor stems from administrative and economic pressures to evaluate and manage patients as quickly as possible—it is simply more practical and less time consuming to rely on the patient as the major source of information—they are always there.

Over the past several decades, there has been a growing recognition for the value of quantitative assessment of clinical phenomena Andreasen, 1982, Andreasen et al., 1988, Spitzer et al., 1978. Rating scales and structured interviews have become an expected component of clinical research studies, and with the rise of managed care, they are making their way into clinical practice as well. A substantial amount of attention has been directed at the reliability of these instruments, and investigators are typically expected to demonstrate adequate reliability in the use of these instruments when data generated from them are presented or published. Yet to date, surprisingly little attention has been paid to the validity of these assessments or to the fact that much of the information reflected in these data come directly from patient's own report. Especially problematic in this regard are fully structured instruments in which raters are essentially bound to the subjects' responses. For example, the most widely cited prevalence estimates for psychiatric disorders come from the Epidemiological Catchment Area study, in which such an instrument (the Diagnostic Interview Schedule; Robins et al., 1981) was used. If indeed patients with psychotic disorders tend to under-report the presence and/or severity of characteristic symptoms of these disorders, the population prevalence of these disorders may be substantially underestimated (Pulver and Carpenter, 1983).

Thus while diagnostic instruments and rating scales help to structure and quantify psychiatric signs and symptoms, the information can ultimately only be as good as its source. Empirical studies are needed to quantify the effects of sources of information on the validity of psychiatric assessment, and to determine which types of information sources may be most useful, efficient and appropriate for specific types of signs and symptoms. The need for studies of this kind was cited as a “research priority” by an NIMH appointed panel of consultants in formulating a national plan to improve clinical services and services research for the chronically mentally ill (Attkisson et al., 1992).

Patients with psychiatric disorders, particularly those with cognitive impairment, psychotic disorders, substance misuse, or personality disorders, may be unable or unwilling to provide accurate historical information about their symptoms and treatment, thereby requiring collateral sources of information to assist the clinician in completing the psychiatric evaluation. Information from relatives and friends have been reported to contribute valuably in the assessment of psychiatric patients in general Brewin et al., 1990, Small et al., 1965, and in patients with dementia (Carr et al., 2000) or those with personality disorders Modestin and Puhan, 2000, Zimmerman et al., 1986. Informant reports about illicit drug use have generally been found to be less helpful O'Donnell et al., 1998, Weiss et al., 2000. The value of collateral sources of information in the evaluation of patients with psychotic disorders has not been systematically studied.

This study was designed to address aspects of this problem, with the following specific goals: (1) To compare SANS and SAPS ratings based strictly on patients' subjective report in single interviews with those based on an exhaustive evaluation using all sources of information and multiple assessments; and (2) to compare SANS and SAPS ratings based on a single interview with a “best informant” (e.g. patient's mother) to these “consensus” ratings. 3) To determine the amount of variance in consensus ratings which can be accounted for by single interviews with each of these sources, i.e. patient and informant, both uniquely and together. Simply stated, we wanted to see how well acutely ill patients with psychotic disorders could describe their symptoms.