Abstract
The extent to which empirical results can lead to methodological conclusions is investigated. No specific limitations are found to be involved in drawing conclusions on the plausibility of an artifact (Campbell).
Two other types of ‘meta-research’ do appear to be problematic: (a) empirically based inferences on the kind of roles adopted by subjects with respect to the experiment (Weber and Cook) are tenuous, since the role-playing may well enter into the meta-research itself; (b) a similar intricacy arises with research on experimenter expectancy effects (Rosenthal). Possible ways of correcting for artifacts in meta-research, and the potential threat to scientific discourse that is associated with these corrections, are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barber, T. X. (1969), ‘Invalid arguments, postmortem analyses, and the experimenter bias effect’, Journal of Consulting and Clinical Psychology 33, 11–14.
Barber, T. X., Calverley, D. S., Forgione, A., McPeake, J. D., Chaves, J. F. and Bowen, B. (1969), ‘Five attempts to replicate the experimenter bias effect’, Journal of Consulting and Clinical Psychology 33, 1–6.
Barber, T. X. and Silver, M. J. (1969), ‘Fact, fiction, and the experimenter bias effect’, Psychological Bulletin Monograph Supplement 70, No. 6, Pt. 2.
Barber, T. X. and Silver, M. J. (1968), ‘Pitfalls in data analysis and interpretation: A reply to Rosenthal’ Psychological Bulletin Monograph Supplement 70, No. 6, Pt. 2.
Campbell, D. T. (1969), ‘Prospective: Artifact and control’, in R. Rosenthal and R. L. Rosnow (eds.), Artifact in Behavioral Research (New York: Academic Press).
Campbell, D. T. and Stanley, J. C. (1966), Experimental and Quasi-Experimental Designs for Research (Chicago: Rand McNally).
Friedrichs, R. W. (1970), A Sociology of Sociology (New York: Free Press).
Holzkamp, K. (1972), Kritische Psychologie (Frankfurt, Germany: Fischer).
Kuhn, T. S. (1970), The Structure of Scientific Revolutions (Chicago: University of Chicago Press).
Lana, R. E. (1969), ‘Pretest sensitization’, in R. Rosenthal and R. L. Rosnow (eds.), Artifact in Behavioral Research (New York: Academic Press).
Levy, L. H. (1967), ‘Awareness, learning, and the beneficient subject as expert witness’, Journal of Personality and Social Psychology 6, 365–370.
Lyons, J. ‘On the psychology of the psychological experiment’, in C. Scheerer (ed.), Cognition: Theory, Research, Promise (New York: Harper, 1964).
Polanyi, M. (1968), ‘Logic and psychology’, American Psychologist 23, 27–43.
Resnick, J. H. and Schwartz, T. (1973), ‘Ethical standards as an independent variable in psychological research’, American Psychologist 28, 134–139.
Rosenberg, M. J. (1969), ‘The conditions and consequences of evaluation apprehension’, in R. Rosenthal and R. L. Rosnow (eds.), Artifact in Behavioral Research (New York: Academic Press).
Rosenthal, R. (1966), Experimenter Effects in Behavioral Research (New York: Appleton-Century-Crofts).
Rosenthal, R. (1968), ‘Experimenter effects and the reassuring nature of the null hypothesis procedure’, Psychological Bulletin Monograph Supplement 70, No. 6.
Rosenthal, R. (1969a), ‘Interpersonal expectations: Effects of the experimenter's hypothesis’, in R. Rosenthal and R. L. Rosnow (eds.), Artifact in Behavioral Research (New York: Academic Press).
Rosenthal, R. (1969b), ‘On not so replicated experiments and not so null results’, Journal of Consulting and Clinical Psychology 33, 7–10.
Rosenthal, R. and Fode, K. L. (1963), ‘Three experiments in experimenter bias’, Psychological Reports 12, 491 -511.
Rosenthal, R. and Rosnow, R. L. (eds.) (1969), Artifact in Behavioral Research (New York: Academic Press).
Rosnow, R. L. and Seels, J. M. (1970), ‘Reactive effects of pretesting in attitude research’, Journal of Personality and Social Psychology 15, 338–343.
Scriven, M. (1964), ‘Views of human nature’, in T. W. Wann, Behaviorism and Phenomenology (Chicago: University of Chicago Press).
Sherwood, J. J. and Nataupsky, M. (1969), ‘Predicting the conclusions of negro-white intelligence research from biographical characteristics of the investigator’, Journal of Personality and Social Psychology 8, 53–58.
Sigall, H., Aronson, E. and Van Hoose, T. (1970), ‘The cooperative subject: Myth or reality?’, Journal of Experimental Social Psychology 6, 1–10.
Watzlawick, P., Beavin, J. H. and Jackson, D. D. (1967), Pragmatics of Human Communication (New York: Norton).
Weber, S. J. and Cook, T. D (1972), ‘Subject effects in laboratory research: An examination of subject role, demand characteristics, and valid inference’, Psychological Bulletin 77, 273–295.
Wiggins, J. S. (1973), Personality and Prediction: Principles of Personality Assessment (Reading, Mass.: Addison).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hofstee, W.K.B. Limitations of methodological experiments. Theor Decis 9, 77–92 (1978). https://doi.org/10.1007/BF00138120
Issue Date:
DOI: https://doi.org/10.1007/BF00138120