Abstract
Multiple informants - compared to single informants - better inform the clinical assessment and the diagnosis of psychopathology. The Operations Triad Model (OTM; De Los Reyes et al. 2013a) provides researchers with a conceptual framework for integrating information from multiple informants into research settings. We simplified this model by: 1) identifying context and insight as the critical factors necessary for determining if multiple informants improve diagnostic accuracy and 2) providing decision-making heuristics for determining when and how to use multiple informants in clinical research and practice. We focused on how symptoms can vary across situations (i.e., context) and how individuals can lack the awareness to accurately report symptoms (i.e., insight) to improve interpretations of informant discrepancies.
Similar content being viewed by others
References
Achenbach, T. M. (2006). As others see us: clinical and research implications of cross-informant correlations for psychopathology. Current Directions in Psychological Science, 15(2), 94–98. doi:10.1111/j.0963-7214.2006.00414.x.
Achenbach, T. M. (2011). Commentary: Definitely more than measurement error: but how should we understand and deal with informant discrepancies? Journal of Clinical Child & Adolescent Psychology, 40(1), 80–86. doi:10.1080/15374416.2011.533416.
Achenbach, T. M., Krukowski, R. A., Dumenci, L. I., & Masha, Y. (2005). Assessment of adult psychopathology: meta-analyses and implications of cross-informant correlations. Psychological Bulletin, 131(3), 361–382. doi:10.1037/0033-2909.131.3.361.
Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101(2), 213-232. doi:10.1037/0033-2909.101.2.213.
Addis, M. E. (2008). Gender and depression in men. Clinical Psychology: Science and Practice, 15(3), 153–168. doi:10.1111/j.1468-2850.2008.00125.x.
Addis, M. E., & Cohane, G. H. (2005). Social scientific paradigms of masculinity and their implications for research and practice in men's mental health. Journal of Clinical Psychology, 61(6), 633–647. doi:10.1002/jclp.20099.
Alonso, P., Menchón, J. M., Segalàs, C., Jaurrieta, N., Jiménez-Murcia, S., Cardoner, N., et al. (2008). Clinical implications of insight assessment in obsessive-compulsive disorder. Comprehensive Psychiatry, 49(3), 305–312. doi:10.1016/j.comppsych.2007.09.005.
Amador, X. F., Flaum, M., Andreasen, N. C., Strauss, D. H., Yale, S. A., Clark, S. C., & Gorman, J. M. (1994). Awareness of illness in schizophrenia and schizoaffective and mood disorders. Archives of General Psychiatry, 51(10), 826–836. doi:10.1001/archpsyc.1994.03950100074007.
American Psychiatric Association [APA]. (2013). The diagnostic and statistical Manual of mental disorders (5th ed.). Washington, DC: Author.
Barlow, D. H. (2008). Clinical handbook of psychological disorders: A step-by-step treatment Manual (4th ed.). New York: Guilford.
Beck, A. T., Baruch, E., Balter, J. M., Steer, R. A., & Warman, D. M. (2004). A new instrument for measuring insight: the Beck cognitive insight scale. Schizophrenia Research, 68(2), 319–329. doi:10.1016/S0920-9964(03)00189-0.
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory-II [BDI-II]. San Antonio: Psychological Corporation.
Belendiuk, K. A., Clarke, T. L., Chronis, A. M., & Raggi, V. L. (2007). Assessing the concordance of measures used to diagnose adult ADHD. Journal of Attention Disorders, 10(3), 276–287. doi:10.1177/1087054706289941.
Bellack, A. S., Green, M. F., Cook, J. A., Fenton, W., Harvey, P. D., Heaton, R. K., et al. (2007). Assessment of community functioning in people with schizophrenia and other severe mental illnesses: a white paper based on an NIMH-sponsored workshop. Schizophrenia Bulletin, 33(3), 805–822. doi:10.1093/schbul/sbl035.
Bloch, M. H., Angeli Landeros-Weisenberger, M. D., Rosario, M. C., Pittenger, C., & Leckman, J. F. (2008). Meta-analysis of the symptom structure of obsessive-compulsive disorder. The American Journal of Psychiatry, 165(12), 1532–1542. doi:10.1176/appi.ajp.2008.08020320.
Carlson, E. N., Vazire, S., & Oltmanns, T. F. (2011). You probably think this paper's about you: narcissists' perceptions of their personality and reputation. Journal of Personality and Social Psychology, 101(1), 185. doi:10.1037/a0023781.
Carlson, E. N., Vazire, S., & Oltmanns, T. F. (2013). Self-other knowledge asymmetries in personality pathology. Journal of Personality, 81(2), 155–170. doi:10.1111/j.1467-6494.2012.00794.x.
Carpenter, K. M., & Addis, M. E. (2000). Alexithymia, gender, and responses to depressive symptoms. Sex Roles, 43(9), 629–644. doi:10.1023/A:1007100523844.
Chambless, D. L., & Ollendick, T. H. (2001). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52(1), 685–716. doi:10.1146/annurev.psych.52.1.685.
Clark, L. A., & Watson, D. (1988). Mood and the mundane: Relations between daily life events and self-reported mood. Journal of Personality and Social Psychology, 54(2), 296. doi:10.1037/0022-3514.54.2.296.
Clark, L. A., & Watson, D. (1991). Tripartite model of anxiety and depression: psychometric evidence and taxonomic implications. Journal of Abnormal Psychology, 100(3), 316–336. doi:10.1037/0021-843X.100.3.316.
Coles, M. E., Frost, R. O., Heimberg, R. G., & Rhéaume, J. (2003). “not just right experiences”: perfectionism, obsessive–compulsive features and general psychopathology. Behaviour Research and Therapy, 41(6), 681–700. doi:10.1016/S0005-7967(02)00044-X.
Coles, M. E., Heimberg, R. G., Frost, R. O., & Steketee, G. (2005). Not just right experiences and obsessive–compulsive features: experimental and self-monitoring perspectives. Behaviour Research and Therapy, 43(2), 153–167. doi:10.1016/j.brat.2004.01.002.
Connelly, B. S., & Ones, D. S. (2010). An other perspective on personality: meta-analytic integration of observers' accuracy and predictive validity. Psychological Bulletin, 136(6), 1092. doi:10.1037/a0021212.
Cooper, L. D., Balsis, S., & Oltmanns, T. F. (2012). Self-and informant-reported perspectives on symptoms of narcissistic personality disorder. Personality Disorders: Theory, Research, and Treatment, 3(2), 140. doi:10.1037/a0026576.
Couture, S. M., Penn, D. L., & Roberts, D. L. (2006). The functional significance of social cognition in schizophrenia: a review. Schizophrenia Bulletin, 32, S44–S63. doi:10.1093/schbul/sbl029.
Cox, B. J., Pagura, J., Stein, M. B., & Sareen, J. (2009). The relationship between generalized social phobia and avoidant personality disorder in a national mental health survey. Depression and Anxiety, 26(4), 354–362.
Coyne, J. C. (1976). Depression and the response of others. Journal of Abnormal Psychology, 85(2), 186–193. doi:10.1037/0021-843X.85.2.186.
Cuijpers, P., van Straten, A., Andersson, G., & van Oppen, P. (2008). Psychotherapy for depression in adults: a meta-analysis of comparative outcomes studies. Journal of Consulting and Clinical Psychology, 76, 909–922. doi:10.1037/a0013075.
David, A. S., Bedford, N., Wiffen, B., & Gilleen, J. (2012). Failures of metacognition and lack of insight in neuropsychiatric disorders. Philosophical Transactions of the Royal Society B: Biological Sciences, 367(1594), 1379–1390. doi:10.1098/rstb.2012.0002.
De Los Reyes, A., Bunnell, B. E., & Beidel, D. C. (2013b). Informant discrepancies in adult social anxiety disorder assessments: links with contextual variations in observed behavior. Journal of Abnormal Psychology, 122(2), 376–386. doi:10.1037/a0031150.
De Los Reyes, A., & Kazdin, A. E. (2004). Measuring informant discrepancies in clinical child research. Psychological Assessment, 16(3), 330–334. doi:10.1037/1040-3590.16.3.330.
De Los Reyes, A., Goodman, K. L., Kliewer, W., & Reid-Quinones, K. (2010). The longitudinal consistency of mother–child reporting discrepancies of parental monitoring and their ability to predict child delinquent behaviors two years later. Journal of Youth and Adolescence, 39(12), 1417–1430.
De Los Reyes, A., Thomas, S. A., Goodman, K. L., & Kundey, S. M. (2013a). Principles underlying the use of multiple informants' reports. Annual Review of Clinical Psychology, 9, 123–149. doi:10.1146/2Fannurev-clinpsy-050212-185617.
Eikenaes, I., Hummelen, B., Abrahamsen, G., Andrea, H., & Wilberg, T. (2013). Personality functioning in patients with avoidant personality disorder and social phobia. Journal of Personality Disorders, 27(6), 746–763.
Finlay, W. M., & Lyons, E. (2001). Methodological issues in interviewing and using self-report questionnaires with people with mental retardation. Psychological Assessment, 13(3), 319. doi:10.1037/1040-3590.13.3.319.
Fioravanti, M., Carlone, O., Vitale, B., Cinti, M. E., & Clare, L. (2005). A meta-analysis of cognitive deficits in adults with a diagnosis of schizophrenia. Neuropsychology Review, 15(2), 73–95. doi:10.1007/s11065-005-6254-9.
Fleeson, W. (2001). Toward a structure-and process-integrated view of personality: traits as density distributions of states. Journal of Personality and Social Psychology, 80(6), 1011. doi:10.1037//0022-3514.80.6.1011.
Foa, E. B., Franklin, M. E., Perry, K. J., & Herbert, J. D. (1996). Cognitive biases in generalized social phobia. Journal of Abnormal Psychology, 105(3), 433.
Galione, J. N., & Oltmanns, T. F. (2013). Identifying personality pathology associated with major depressive episodes: incremental validity of informant-reports. Journal of Personality Assessment, 95(6), 625–632. doi:10.1080/00223891.2013.825624.
Giesler, R. B., Josephs, R. A., & Swann Jr., W. B. (1996). Self-verification in clinical depression: the desire for negative evaluation. Journal of Abnormal Psychology, 105(3), 358. doi:10.1037/0021-843X.105.3.358.
Gilovich, T., & Savitsky, K. (1999). The spotlight effect and the illusion of transparency: egocentric assessments of how we are seen by others. Current Directions in Psychological Science, 8(6), 165–168.
Grucza, R. A., & Goldberg, L. R. (2007). The comparative validity of 11 modern personality inventories: predictions of behavioral acts, informant-reports, and clinical indicators. Journal of Personality Assessment, 89, 167–187. doi:10.2307/3149759.
Hunsley, J., & Mash, E. J. (2005). Introduction to the special section on developing guidelines for the evidence-based assessment (EBA) of adult disorders. Psychological Assessment, 17(3), 251–255. doi:10.1037/1040-3590.17.3.251.
Hutchinson, A. D., & Mathias, J. L. (2007). Neuropsychological deficits in frontotemporal dementia and Alzheimer’s disease: a meta-analytic review. Journal of Neurology, Neurosurgery & Psychiatry, 78(9), 917–928. doi:10.1136/jnnp.2006.100669.
Hyler, S. E., Rieder, R. O., Williams, J. B., Spitzer, R. L., Lyons, M., & Hendler, J. (1989). A comparison of clinical and self-report diagnoses of DSM-III personality disorders in 552 patients. Comprehensive Psychiatry, 30, 170–178. doi:10.1016/0010-440X(89)90070-9.
Jones, S., & Miller, J. D. (2012). Psychopathic traits and externalizing behaviors: a comparison of self-and informant-reports in the statistical prediction of externalizing behaviors. Psychological Assessment, 24(1), 255. doi:10.1037/a0025264.
Jorm, A. F. (1996). Assessment of cognitive impairments and dementia using informant-reports. Clinical Psychology Review, 16(1), 51–73. doi:10.1016/0272-7358(95)00056-9.
Klonsky, D. E., Oltmanns, T. F., & Turkheimer, E. (2002). Informant-reports of personality disorder: relation to self-reports and future directions. Clinical Psychology: Science and Practice, 9(3), 300–311. doi:10.1093/clipsy.9.3.300.
Kraemer, H. C., Measelle, J. R., Ablow, J. C., Essex, M. J., Boyce, W. T., & Kupfer, D. J. (2003). A new approach to integrating data from multiple informants in psychiatric assessment and research: mixing and matching contexts and perspectives. American Journal of Psychiatry, 160(9), 1566–1577. doi:10.1176/appi.ajp.160.9.1566.
Kwang, T., & Swann, W. B. (2010). Do people embrace praise even when they feel unworthy? A review of critical tests of self-enhancement versus self-verification. Personality and Social Psychology Review, 14(3), 263–280. doi:10.1177/1088868310365876.
Leckman, J. F., Grice, D. E., Boardman, J., Zhang, H., Vitale, A., Bondi, C., et al. (1997). Symptoms of obsessive-compulsive disorder. American Journal of Psychiatry, 154(7), 911–917.
Lieberman, L., Liu, H., Huggins, A. A., Katz, A. C., Zvolensky, M. J., & Shankman, S. A. (2016). Comparing the validity of informant and self-reports of personality using laboratory indices of emotional responding as criterion variables. Psychophysiology. doi:10.1111/psyp.12680.
Lukowitsky, M. R., & Pincus, A. L. (2013). Interpersonal perception of pathological narcissism: a social relations analysis. Journal of Personality Assessment, 95(3), 261–273. doi:10.1080/00223891.2013.765881.
Mangone, C. A., Hier, D. B., Gorelick, P. B., Ganellen, R. J., Langenberg, P., Boarman, R., & Dollear, W. C. (1991). Impaired insight in Alzheimer's disease. Journal of Geriatric Psychiatry and Neurology, 4(4), 189–193. doi:10.1177/089198879100400402.
McKnight, P. E., & Kashdan, T. B. (2009). The importance of functional impairment to mental health outcomes: a case for reassessing our goals in depression treatment research. Clinical Psychology Review, 29, 243–259. doi:10.1016/j.cpr.2009.01.005.
McKnight, P. E., Monfort, S. S., Kashdan, T. B., Blalock, D. V., & Calton, J. M. (2015). Anxiety symptoms and functional impairment: a systematic review of the correlation between the two measures. Clinical Psychology Review, 45, 115–130. doi:10.1016/j.cpr.2015.10.005.
Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Minneapolis: University of Minnesota Press.
Meehl, P. E. (1956). Wanted—A good cook-book. American Psychologist, 11(6), 263–272. doi:10.1037/h0044164.
Metzler-Baddeley, C. (2007). A review of cognitive impairments in dementia with Lewy bodies relative to Alzheimer's disease and Parkinson's disease with dementia. Cortex, 43(5), 583–600. doi:10.1016/S0010-9452(08)70489-1.
Mischel, W. (1968). Personality and assessment. New York: Wiley.
Mischel, W., & Shoda, Y. (1998). Reconciling processing dynamics and personality dispositions. Annual Review of Psychology, 49, 229–258. doi:10.1146/annurev.psych.49.1.229.
Mischel, W., Mendoza-Denton, R., & Shoda, Y. (2002). Situation-behavior contingencies as a locus of consistency in personality. Current Directions in Psychological Science, 11(2), 50–54. doi:10.1111/1467-8721.00166.
Mullen, R., Howard, R., David, A., & Levy, R. (1996). Insight in Alzheimer's disease. International Journal of Geriatric Psychiatry, 11(7), 645–651. doi:10.1002/(SICI)1099-1166(199607)11:7<645::AID-GPS366>3.0.CO;2-P.
Plassman, B. L., Langa, K. M., Fisher, G. G., Heeringa, S. G., Weir, D. R., Ofstedal, M. B., et al. (2007). Prevalence of dementia in the United States: the aging, demographics, and memory study. Neuroepidemiology, 29(1–2), 125–132. doi:10.1159/000109998.
Poulsen, S., Lunn, S., Daniel, S. I. F., Folke, S., Mathieson, B. B., Katznelson, H., & Fairburn, C. G. (2014). A randomized controlled trial of psychoanalytic psychotherapy or cognitive-behavioral therapy for bulimia nervosa. American Journal of Psychiatry, 171, 109–116. doi:10.1176/appi.ajp.2013.12121511.
Quee, P. J., van der Meer, L., Bruggeman, R., de Haan, L., Krabbendam, L., Cahn, W., et al. (2010). Insight in psychosis: relationship with neurocognition, social cognition and clinical symptoms depends on phase of illness. Schizophrenia Bulletin, 133, 1–9. doi:10.1093/schbul/sbq133.
Ray, J. V., Hall, J., Rivera-Hudson, N., Poythress, N. G., Lilienfeld, S. O., & Morano, M. (2013). The relation between self-reported psychopathic traits and distorted response styles: a meta-analytic review. Personality Disorders: Theory, Research, and Treatment, 4(1), 1. doi:10.1037/a0026482.
Rottenberg, J., Gross, J. J., & Gotlib, I. H. (2005). Emotion context insensitivity in major depressive disorder. Journal of Abnormal Psychology, 114(4), 627. doi:10.1037/0021-843X.114.4.627.
Rottenberg, J. E., & Johnson, S. L. (2007). Emotion and psychopathology: Bridging affective and clinical science. Washington, DC: American Psychological Association.
Samuel, D. B. (2015). A review of the agreement between clinicians’ personality disorder diagnoses and those from other methods and sources. Clinical Psychology: Science and Practice, 22(1), 1–19. doi:10.1111/cpsp.12088.
Samuel, D. B., Sanislow, C. A., Hopwood, C. J., Shea, M. T., Skodol, A. E., Morey, L. C., et al. (2013). Convergent and incremental predictive validity of clinician, self-report, and structured interview diagnoses for personality disorders over 5 years. Journal of Consulting and Clinical Psychology, 81(4), 650–659. doi:10.1037/a0032813.
Shoda, Y., Mischel, W., & Wright, J. C. (1994). Intraindividual stability in the organization and patterning of behavior: incorporating psychological situations into the idiographic analysis of personality. Journal of Personality and Social Psychology, 67(4), 674. doi:10.1037/0022-3514.67.4.674.
Shoda, Y., Wilson, N. L., Chen, J., Gilmore, A. K., & Smith, R. E. (2013). Cognitive-affective processing system analysis of intra-individual dynamics in collaborative therapeutic assessment: translating basic theory and research into clinical applications. Journal of Personality, 81(6), 554–568. doi:10.1111/jopy.12015.
Smith, S. R. (2007). Making sense of multiple informants in child and adolescent psychopathology: a guide for clinicians. Journal of Psychoeducational Assessment, 25, 139–149. doi:10.1177/0734282906296233.
Swann, W. B., Wenzlaff, R. M., & Tafarodi, R. W. (1992). Depression and the search for negative evaluations: more evidence of the role of self-verification strivings. Journal of Abnormal Psychology, 101(2), 314–317. doi:10.1037/0021-843X.101.2.314.
Steketee, G. S. (1993). Treatment of obsessive-compulsive disorder. New York: Guilford.
Uher, R., Perlis, R. H., Placentino, A., Dernovšek, M. Z., Henigsberg, N., Mors, O., et al. (2012). Self-report and clinician-rated measures of depression severity: can one replace the other? Depression and Anxiety, 29(12), 1043–1049. doi:10.1002/da.21993.
Vazire, S., & Carlson, E. N. (2011). Others sometimes know us better than we know ourselves. Current Directions in Psychological Science, 20(2), 104–108. doi:10.1177/0963721411402478.
Watts, A. L., Lilienfeld, S. O., Edens, J. F., Douglas, K. S., Skeem, J. L., Verschuere, B., & LoPilato, A. C. (2016). Does response distortion statistically affect the relations between self-report psychopathy measures and external criteria? Psychological Assessment, 28(3), 294. doi:10.1037/pas0000168.supp.
Zametkin, A. J., & Ernst, M. (1999). Problems in the management of attention-deficit–hyperactivity disorder. New England Journal of Medicine, 340(1), 40–46. doi:10.1056/NEJM199901073400107.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
Lisa A. Alexander, Patrick E. McKnight, David J. Disabato, & Todd B. Kashdan declare they have no conflicts of interest.
Experiment Participants
We did not collect data from participants in this study.
Funding
No extramural or intramural funding support the authors during the project.
Informed Consent
We did not collect data from participants for the current manuscript. Thus, it was not necessary for us to obtain informed consent.
Ethical Approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Rights and permissions
About this article
Cite this article
Alexander, L.A., McKnight, P.E., Disabato, D.J. et al. When and How to use Multiple Informants to Improve Clinical Assessments. J Psychopathol Behav Assess 39, 669–679 (2017). https://doi.org/10.1007/s10862-017-9607-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10862-017-9607-9