Abstract
Interpretations of data in qualitative research may be biased for many reasons. This paper explores three commonly overlooked problems from a rather positivist point of view and deals with them mainly through the lens of cognitive psychology and survey methodology. The first problem is that researchers and readers of the research tend to trust retrospective data too much even though it is known that our memory is highly reconstructive. The second problem is that we often create interpretations too quickly and do not ground them in data well. The third problem is inappropriately generalising our findings because we underrate the variability of the phenomena studied. The aim is not to employ quantitative criteria in qualitative research but to show that especially in cases where we seek more objectivity (e.g., factual information about events) and less about the subjective phenomenal world (e.g., how people perceive these events from today’s perspective), cognitive psychology or survey methodology can offer valuable insights. Recommendations about what researchers should be careful of and how to increase the objectivity of the interpretations are offered.
[1] American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author. Search in Google Scholar
[2] Anginer, D., Fisher, K. L., Statman, M. (2007). Stocks of admired companies and despised ones. Available at SSRN: http://ssrn.com/abstract=962168 or http://dx.doi.org/10.2139/ssrn.962168 (February). 10.2139/ssrn.962168Search in Google Scholar
[3] Asch S. E. (1946). Forming impressions of personality. Journal of Abnormal and Social Psychology, 41, 258–290. http://dx.doi.org/10.1037/h005575610.1037/h0055756Search in Google Scholar
[4] Bahrick, H. P. (1983). The cognitive map of a city: Fifty years of learning and memory. In G. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (pp. 125–163). New York: Academic Press. Search in Google Scholar
[5] Bahrick, H. P., Bahrick, P. O., & Wittlinger, R. P. (1975). Fifty years of memory for names and faces: A cross-sectional approach. Journal of Experimental Psychology: General, 104(1), 54–75. http://dx.doi.org/10.1037/0096-3445.104.1.5410.1037/0096-3445.104.1.54Search in Google Scholar
[6] Baron, J. (2008). Thinking and deciding (4th ed.). Cambridge: Cambridge University Press. Search in Google Scholar
[7] Bradburn, N., Sudman, S., & Wansink, B. (2004). Asking questions. Rev. ed. San Francisco: Jossey-Bass. Search in Google Scholar
[8] Brenner, L. A., Koehler, D. J., & Tversky, A. (1996). On the evaluation of one-sided evidence. Journal of Behavioral Decision Making, 9(1), 59–70. http://dx.doi.org/10.1002/(SICI)1099-0771(199603)9:1<59::AID-BDM216>3.0.CO;2-V10.1002/(SICI)1099-0771(199603)9:1<59::AID-BDM216>3.0.CO;2-VSearch in Google Scholar
[9] Cassel, E., Bernstein, D. A. (2007). Criminal behavior (2nd ed.). Mahwah: Lawrence Erlbaum Associates. 10.4324/9780203936535Search in Google Scholar
[10] Carey, B. (2011, November 3). Fraud case seen as a red flag for psychology research. New York Times, p. 3. Search in Google Scholar
[11] Cialdiny, R. B. (2009). Influence: science and practice (5th ed.) Boston: Pearson Education, Inc. Search in Google Scholar
[12] Conway, M. A., Meares, K., & Standart, S. (2004). Images and goals. Memory, 12(4), 525–531. http://dx.doi.org/10.1080/0965821044400015110.1080/09658210444000151Search in Google Scholar
[13] Creswell, J. W. (2013). Qualitative inquiry & research design (3rd ed). Thousand Oaks: Sage. Search in Google Scholar
[14] Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and metaanalysis. Hove: Routledge. Search in Google Scholar
[15] Davies, M. F. (2003). Confirmatory bias in the evaluation of personality descriptions: Positive test strategies and output interference. Journal of Personality and Social Psychology, 85(4), 736–744. http://dx.doi.org/10.1037/0022-3514.85.4.73610.1037/0022-3514.85.4.736Search in Google Scholar
[16] Ericsson, K. A. (Ed.). (2009). Development of professional expertise: Toward measurement of expert performance and design of optimal learning environments. Cambridge: Cambridge University Press. 10.1017/CBO9780511609817Search in Google Scholar
[17] Fowler, F. J. (2014). Survey research methods (5th ed.). Thousand Oaks: Sage. Search in Google Scholar
[18] Friese, S. (2014). Qualitative data analysis with ATLAS.ti. (2nd ed.). Thousand Oaks: Sage. Search in Google Scholar
[19] Guillebeau, C. (2012). The $100 startup: Reinvent the way you make a living, do what you love, and create a new future. New York: Crown Business. Search in Google Scholar
[20] Heath, C. & Heath, D. (2007). Made to stick. Why some ideas survive and others die. New York: Random House. Search in Google Scholar
[21] Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus and Giroux. Search in Google Scholar
[22] Kline, R. B. (2013). Beyond significance testing: Statistics reform in the behavioral sciences (2nd ed). Washington, DC: APA. http://dx.doi.org/10.1037/14136-00010.1037/14136-000Search in Google Scholar
[23] Klayman, J., & Ha, Y.-W. (1987). Confirmation, disconfirmation, and information in hypothesis testing. Psychological Review, 94, 211–228. http://dx.doi.org/10.1037/0033-295X.94.2.21110.1037/0033-295X.94.2.211Search in Google Scholar
[24] Loftus, E. F., & Palmer, J. C. (1974). Reconstruction of auto-mobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behaviour, 13, 585–589. http://dx.doi.org/10.1016/S0022-5371(74)80011-310.1016/S0022-5371(74)80011-3Search in Google Scholar
[25] Law, J. (2004). After method: Mess in social science research. Oxon: Routledge. Search in Google Scholar
[26] Loftus, E. F. (1996). Eyewitness testimony. Cambridge: Harvard University Press. Search in Google Scholar
[27] Mason, J. (2002). Qualitative researching (2nd ed.). London: Sage Publications. Search in Google Scholar
[28] Mason, M. (2010). Sample size and saturation in PhD studies using qualitative interviews. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 11(3). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/1428 Search in Google Scholar
[29] Miles, M. B., Huberman, A.M., & Saldaña, J. (2014). Qualitative data analysis: A methods sourcebook. Thousand Oaks: Sage. Search in Google Scholar
[30] Morewedge, C. K., & Kahneman, D. (2010). Associative processes in intuitive judgment. Trends in Cognitive Sciences, 14(10), 435–440. http://dx.doi.org/10.1016/j.tics.2010.07.00410.1016/j.tics.2010.07.004Search in Google Scholar
[31] Neusar, A. (2009). Jaké zdroje informací používáme při usuzování o příčinách vlastního chování [Sources of information we use when reasoning about the causes of our behaviour]. E-psychologie, 3(2). Available at: http://e-psycholog.eu/pdf/neusar.pdf Search in Google Scholar
[32] Neusar, A. (2011). Kdy se to jenom stalo? (Ne)dokonalost paměti na osobní a veřejné události [When did it happen? (Im)perfection of memory for personal and public events]. Brno: MSD. Or dissertation in English available at: http://is.muni.cz/th/195480/fss_d Search in Google Scholar
[33] Neusar, A., Hoferková, J., & Ježek, S. (2011). Přesnost datace mediálně známých veřejných událostí [Dating accuracy of well-known public events]. Mediální studia, 02, 120–151. Search in Google Scholar
[34] Peräkyl, A. (2004). Reliability and validity in research based upon transcripts. In D. Silverman (Ed.), Qualitative research: Theory, method and practice (2nd ed., pp. 282–303). Search in Google Scholar
[35] Ptčáková, K. (2012) Professional curiosity engaged in policy sociology. Human Affairs, 22(4), 475–491. http://dx.doi.org/10.2478/s13374-012-0039-710.2478/s13374-012-0039-7Search in Google Scholar
[36] Schacter, D. L. (1999). The seven sins of memory: Insights from psychology and cognitive neuroscience. American Psychologist, 54(3), 182–203. http://dx.doi.org/10.1037/0003-066X.54.3.18210.1037/0003-066X.54.3.182Search in Google Scholar
[37] Schwarz, N. (2007). Retrospective and concurrent self-reports: The rationale for real-time data capture. In A. A. Stone, S. S. Shiffman, A. Atienza, & L. Nebeling (Eds.), The science of real-time data capture: Self-reports in health research (pp. 11–26). New York: Oxford University Press. Search in Google Scholar
[38] Schwarz, N., Strack, F., & Mai, H. (1991). Assimilation and contrast effects in part-whole question sequences: A conversational logic analysis. Public Opinion Quarterly, 55(1), 3–23. http://dx.doi.org/10.1086/26923910.1086/269239Search in Google Scholar
[39] Sedlmeier, P., & Betsch, T. (2002) (Eds.). Etc. Frequency processing and cognition. Oxford: OUP. 10.1093/acprof:oso/9780198508632.001.0001Search in Google Scholar
[40] Skowronski, J. J., Betz, A. L., Thompson, C. P., Walker, R. W., & Shannon, L. (1994). The impact of differing memory domains on event-dating processes in self and proxy reports. In N. Schwarz & S. Sudman (Eds.), Autobiographical memory and the validity of retrospective reports (pp. 217–231). New York: Springer-Verlag. http://dx.doi.org/10.1007/978-1-4612-2624-6_1510.1007/978-1-4612-2624-6_15Search in Google Scholar
[41] Spry, Y. (2011). Performative autoethnography: critical embodiments and possibilities. In N. K. Denzin, Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (4th ed., pp. 497–512). Thousand Oaks: Sage Publications. Search in Google Scholar
[42] Stanovich, K. E., & West, R. F. (2002). Individual differences in reasoning: Implications for the rationality debate? In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases: The psychology of intuitive judgment (pp. 421–440). New York, NY US: Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511808098.02610.1017/CBO9780511808098.026Search in Google Scholar
[43] Sternberg, R. J., & Sternberg, K. (2010). The psychologist’s companion: A guide to writing scientific papers for students and researchers (5th ed.). Cambridge: Cambridge University Press. http://dx.doi.org/10.1017/CBO978051176202410.1017/CBO9780511762024Search in Google Scholar
[44] Stoklasa, J., Talášek, T., & Musilová, J. (2014). Fuzzy approach — a new chapter in the methodology of psychology? Human affairs, 24(2), 189–203. 10.2478/s13374-014-0219-8Search in Google Scholar
[45] Rosenzweig, P. (2007). The halo effect: … and the eight other business delusions that deceive managers. New York: Simon & Schuster. Search in Google Scholar
[46] Taleb, N. N. (2010). The black swan: The impact of the highly improbable (2nd ed). New York: Random House. Search in Google Scholar
[47] Tourangeau, R., Rips, L. J., & Rasinski, K. (2000). The psychology of survey response. Oxford: OUP. http://dx.doi.org/10.1017/CBO978051181932210.1017/CBO9780511819322Search in Google Scholar
[48] Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin, 76(2), 105–110. http://dx.doi.org/10.1037/h003132210.1037/h0031322Search in Google Scholar
[49] Vranka, M. (2013). Bariéry kritického myšlení v psychotherapy. In A. Neusar, M. Charvát, M. Dolejš, D. Janečková, R. Procházka (Eds.), PhD Existence III. Olomouc: Palacký University. Available at: http://www.ff.upol.cz/fileadmin/user_upload/FFkatedry/psychologie/Konference/PhD_existence_III/sbornik_odbornych_prispevku_tisk.pdf Search in Google Scholar
[50] Whitten, W. B., & Leonard, J. M. (1981). Directed search through autobiographical memory. Memory & Cognition, 9(6), 566–579. http://dx.doi.org/10.3758/BF0320235110.3758/BF03202351Search in Google Scholar
© 2014 Institute for Research in Social Communication, Slovak Academy of Sciences
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.
