Abstract
The need to train accurate, not necessarily agreeing, observers is discussed. Intraobserver consistency as an intermediate criterion in such training is proposed and contrasted with the more familiar criterion of interobserver agreement. Videotaped observations of social interactions between handicapped and nonhandicapped preschoolers provided the medium for examining the criterion agreement of four observers trained against each type of standard. Observers generally failed to show high levels of criterion agreement whether trained to a within- or to a between-observer agreement standard. The results varied somewhat with the frequency of behaviors, however. Correlations between interobserver agreement and intraobserver consistency were variable but somewhat higher when interobserver agreement was the training criterion than when intraobserver consistency was the criterion. Correlations between interobserver agreement and criterion agreement ranged from — .16 to .89 during interobserver agreement training. Correlations between intraobserver consistency and criterion agreement ranged from — .23 to .99 during intraobserver consistency training.
Similar content being viewed by others
References
Allport, G. W. (1937).Personality: A psychological interpretation. New York: Holt.
Baum, W. M. (1975). Time allocation in human vigilance.Journal of the Experimental Analysis of Behavior, 23(1), 45–54.
Bijou, S. W., Peterson, R. F., & Ault, M. H. (1968). A method to integrate descriptive and experimental field studies at the level of data and empirical concepts.Journal of Applied Behavior Analysis, 1, 175–191.
Boice, R. (1983). Observational skills.Psychological Bulletin, 93, 3–29.
Cone, J. D. (1981). Psychometric considerations. In M. Hersen & A. S. Bellack (Eds.),Behavioral assessment: A practical handbook (2nd ed., pp. 38–68). New York: Pergamon.
Cone, J. D., & Foster, S. L. (1982). Direct observation in clinical psychology. In P. C. Kendall & J. N. Butcher (Eds.),Handbook of research in clinical psychology. New York: Wiley.
Gewirtz, H. B., & Gewirtz, J. L. (1969). Caretaking settings, background events and childrearing environments: Some preliminary trends. In B. M. Foss (Ed.),Determinants of infant behaviour, Vol. IV. London: Methuen.
Hartmann, D. P., & Wood, D. D. (1982). Observational methods. In A. S. Bellack, M. Hersen, & A. E. Kazdin (Eds.),International handbook of behavior modification and therapy. New York: Plenum.
Johnson, S. M., & Bolstad, O. D. (1973). Methodological issues in naturalistic observation: Some problems and solutions for field research. In L. A. Hamerlynck, L. C. Handy, & E. J. Mash (Eds.),Behavior change: Methodology, concepts, and practice. Champaign, IL: Research Press.
Johnston, J. M., & Pennypacker, H. S. (1980).Strategies and tactics of human behavior research. Hillsdale, NJ: Lawrence Erlbaum Associates.
Kapust, J. A., & Nelson, R. O. (1984). Effects of the rate and spatial separation of target behaviors on observer accuracy and interobserver agreement.Behavioral Assessment, 6, 253–262.
Kazdin, A. E. (1975).Behavior modification in applied settings. Homewood, IL: Dorsey Press.
Kazdin, A. E. (1977). Artifact, bias, and complexity of assessment: The ABCs of reliability.Journal of Applied Behavior Analysis, 10, 141–150.
Reid, J. B. (1982). Observer training in naturalistic research. In D. P. Hartmann (Ed.),Using observers to study behavior (pp. 37–50). San Francisco: Jossey-Bass.
Weinrott, M. R., & Jones, R. R. (1984). Overt versus covert assessment of observer reliability.Child Development, 55, 1125–1137.
Wildman, B. G., Erickson, M. T., & Kent, R. N. (1975). The effect of two training procedures in observer agreement and variability of behavior ratings.Child Development, 46, 520–524.
Yarrow, M. R., & Waxler, C. Z. (1979). Observing interaction: A confrontation with methodology. In R. B. Cairns (Ed.),The analysis of social interactions: Methods, issues, and illustrations (pp. 37–65). Hillsdale, NJ: Lawrence Erlbaum & Associates.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wolfe, V.V., Cone, J.D. & Wolfe, D.A. Social and solipsistic observer training: Effects on agreement with a criterion. J Psychopathol Behav Assess 8, 211–226 (1986). https://doi.org/10.1007/BF00959833
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00959833