Skip to main content
SpringerLink
Log in

Reliability and Validity of Prisoner Self-Reports Gathered Using the Life Event Calendar Method

  • Original Paper
  • Published:
Journal of Quantitative Criminology Aims and scope Submit manuscript

An Erratum to this article was published on 17 February 2011

Abstract

Data collection using the life event calendar method is growing, but reliability is not well established. We examine test–retest reliability of monthly self-reports of criminal behavior collected using a life event calendar from a random sample of minimum and medium security prisoners. Tabular analysis indicates substantial agreement between self-reports of drug dealing, property, and violent crime during a baseline interview (test) and a follow-up (retest) approximately 3 weeks later. Hierarchical analysis reveals that criminal activity reported during the initial test is strongly associated with responses given in the retest, and that the relationship varies only by the lag in days between the initial interview and the retest. Analysis of validity reveals that self-reported incarceration history is strongly predictive of official incarceration history although we were unable to address whether subjects could correctly identify the months they were incarcerated. African Americans and older subjects provide more valid responses but in practical terms the differences in validity are not large.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. The life event calendar (LEC) method is alternatively referred to as the event-history calendar method, the icon calendar method, the timeline method, the timeline-follow back method, and the life-history calendar method.

  2. The focus in this paper is monthly self-reports, but the LEC method is not defined by a particular calendar length. Rather, the calendar is a function of research questions and prior theory. Thus, calendars may appropriately be constructed to document changes occurring hourly, daily, weekly, monthly, yearly, or over multi-year periods across the life course.

  3. Our protocol initially specified interviews with female prisoners. This part of our request was denied by the ODRC institutional review board (IRB) because of the large number of ongoing research projects being conducted in the Ohio prisons that house females.

  4. ODRC’s decision about which institutions we could visit was based largely on how many research projects had been recently approved, and which institutions those projects were conducted in. The institutions we were granted access to had not fielded as many recent studies.

  5. Confidentiality is protected by a Certificate of Confidentiality from the National Institutes of Health.

  6. The retest was much shorter because it contained less than half the questions included in the test, with a focus on criminal behavior and other topics collected with the calendar.

  7. We paired male and female interviewers together in the majority of interviews. Two out of eight primary interviewers are African American, and half are female. The remaining interviewers are White males. We also attempted to balance age when assigning interviewers.

  8. Each subject’s responses were recorded simultaneously on a paper calendar that was kept in front of the subject for reference and on the electronic version of the calendar maintained by the second interviewer (the laptop was positioned so the subject could see the calendar screen).

  9. We focus on whether subjects correctly specified the months in which they committed crimes. Out of necessity to reduce the length of the interview, we did not collect frequency data for each month crimes were reported. Rather, if subjects reported that they committed crime in any month a follow-up question asked subjects to report the frequency of offending in the typical month.

  10. There is a substantial body of literature positing a relationship between the salience of a criminal incident and accuracy of recall. We attempted to model the three components of the criminal behavior scale independently as binary outcomes, but the HLM program was unable to compute robust standard errors for those models. Tabular results for each component are presented in “Appendix”, indicating minimal difference in reliability among the component items of the criminal behavior scale.

  11. At level-1 we model: ηij = log(λ ij ), where λ ij is the event rate reflecting the number of self-reported crimes during the retest and ηijk is the log of the event rate. Note that while λ ij is constrained to be non-negative, log(λ ij ) can take on any value. The predicted log event rate can be converted to an event rate by generating λ ij  = exponential{ηij}.

  12. Subjects were not asked to provide the dates of incarceration in state prisons, thus precluding examination of the timing issue.

  13. Arrest data maintained by ODRC (our data source) is less consistently recorded as a result of missing pre-sentence investigation (PSI) paperwork from the criminal history records of a significant proportion of cases. Collection and analysis of arrest data also requires significant resources due to the amount of time required to hand code official data from ODRC data bases into a file matching the 18 month calendar. We hope to have resources to collect and compare monthly self-reports of arrests with monthly official arrests in the future.

References

  • Aiken LS, West SG (1991) Multiple regression: testing and interpreting interactions. Sage Publications, Newbury Park, CA

    Google Scholar 

  • Axinn WG, Pearce LD, Ghimire D (1999) Innovations in life history calendar applications. Soc Sci Res 28:243–264

    Article  Google Scholar 

  • Babbie E (1995) The practice of social research, 7th edn. Wadsworth, Belmont, CA

    Google Scholar 

  • Bachman G (1970) The impact of the family background and intelligence on tenth grade boys. Youth in transition, vol. II. University of Michigan, Institute for Social Research, Ann Arbor

  • Belli RF (1998) The structure of autobiographical memory and the event history calendar: potential improvements in the quality of retrospective reports in surveys. Memory 6(4):383–406

    Article  Google Scholar 

  • Belli RF, Shay WL, Stafford FP (2001) Event history calendars and question list surveys: a direct comparison of interviewing methods. Public Opin Q 65:45–74

    Article  Google Scholar 

  • Blumstein A, Cohen J, Roth JA, Visher C (eds) (1986) Criminal careers and “career criminals”. National Academy Press, Washington, DC

    Google Scholar 

  • Bradburn NM, Rips LJ, Shevell SK (1987) Answering autobiographical questions: the impact of memory and inference on surveys. Science 236:157–161

    Article  Google Scholar 

  • Cannell CF, Kahn RL (1968) Interviewing. In: Gardner L, Aronson E (eds) The handbook of social psychology, 2nd edn. Addison-Wesley Publishing Company, Reading, MA, pp 526–595

    Google Scholar 

  • Carmines EG, Zeller RA (1979) Reliability and validity assessment. Sage University Paper Series on Quantitative Applications in the Social Sciences, 07-017. Sage, Newbury Park, CA

  • Caspi A et al (1996) The life history calendar: a research and clinical assessment method for collecting retrospective event-history data. Int J Methods Psychiatr Res 6:101–114

    Article  Google Scholar 

  • Chaiken JM, Chaiken MR (1982) Varieties of criminal behavior. Rand Corporation, Santa Monica, CA

    Google Scholar 

  • Clemmer D (1940) The prison community. Christopher Publishing House, Boston, MA

    Google Scholar 

  • Day C et al (2004) Reliability of heroin users’ reports of drug use behavior using a 24 month timeline follow-back technique to assess the impact of the Australian heroin shortage. Addict Res Theory 12(5):433–443

    Article  Google Scholar 

  • Engel LS, Keifer MC, Zahm SH (2001) Comparison of a traditional questionnaire with an icon/calendar-based questionnaire to assess occupational history. Am J Ind Med 40:502–511

    Article  Google Scholar 

  • Farrington DP (1973) Self-reports of deviant behavior: predictive and stable? J Crim Law Criminol 64:99–110

    Article  Google Scholar 

  • Farrington DP, Loeber R, Stouthamer-Loeber M, Van WB, Kamman LS (1996) Self-reported delinquency and a combined delinquency seriousness scale based on boys, mothers, and teachers: concurrent and predictive validity for African-Americans and Caucasions. Criminology 34:493–517

    Article  Google Scholar 

  • Fendrich M, Vaughn CM (1994) Diminished lifetime substance use over time: an inquiry into differential underreporting. Public Opin Q 58:96–123

    Article  Google Scholar 

  • Fontana A, Frey JH (2003) The interview: from structured questions to negotiated text. In: Denzin NK, Lincoln YS (eds) Collecting and interpreting qualitative materials. Sage, Thousand Oaks, CA, pp 645–672

    Google Scholar 

  • Freedman D et al (1988) The life history calendar: a technique for collecting retrospective data. Sociol Methodol 18:37–68

    Article  Google Scholar 

  • Golub A, Johnson BD, Taylor A, Liberty HJ (2002) The validity of arrestees’ self-reports: variations across questions and persons. Justice Quart 19(3):477–502

    Article  Google Scholar 

  • Hagan J, McCarthy B (1998) Mean streets: youth crime and homelessness. Cambridge University Press, New York, NY

    Google Scholar 

  • Hindelang MJ, Hirschi T, Weis JG (1981) Measuring delinquency. Sage, Beverly Hills, CA

    Google Scholar 

  • Horney J, Wayne Osgood D, Marshall IH (1995) Criminal careers in the short-term: intra-individual variability in crime and its relation to local life circumstances. Am Sociol Rev 60:655–673

    Article  Google Scholar 

  • Hughes EC (1945) Dilemmas and contradictions of status. Am J Sociol 50(5):353–359

    Article  Google Scholar 

  • Huizinga D, Elliot DS (1986) Reassessing the reliability and validity of self-report delinquency measures. J Quant Criminol 2(4):293–327

    Google Scholar 

  • Jolliffe D et al (2003) Predictive, concurrent, prospective and retrospective validity of self-reported delinquency. Crim Behav Mental Health 13:179–197

    Article  Google Scholar 

  • Junger-Tas J, Marshall IH (1999) The self-report methodology in crime research. Crime Justice 25:291–367

    Article  Google Scholar 

  • Kinnear PR, Gray CD (2006) SPSS 14 made simple. Psychology Press, New York, NY

    Google Scholar 

  • Kruttschnitt C, Carbone-Lopez K (2006) Moving beyond the stereotypes: women’s subjective accounts of their violent crime. Criminology 44(2):321–352

    Article  Google Scholar 

  • Laub JH, Sampson RJ (2003) Shared beginnings, divergent lives: delinquent boys to age 70. Harvard University Press, Cambridge, MA

    Google Scholar 

  • Lewis D, Mhlanga B (2001) A life of crime: the hidden truth about criminal activity. Int J Market Res 43(2):217–240

    Google Scholar 

  • Lin N, Ensel WM, Lai WG (1997) Construction and use of the life history calendar: reliability and validity of recall data. In: Gotlib IH, Wheaton B (eds) Stress and adversity over the life course. Cambridge University Press, New York, NY, pp 249–272

    Chapter  Google Scholar 

  • Litwin MS (1995) How to measure survey reliability and validity. Sage, Thousand Oaks, CA

    Google Scholar 

  • Luke DA (2004) Multilevel modeling. Sage, Thousand Oaks, CA

    Google Scholar 

  • MacKenzie DL, de Li S (2002) The impact of formal and informal social controls on the criminal activities of probationers. J Res Crime Delinq 39(3):243–276

    Article  Google Scholar 

  • Mensch BS, Kandel DB (1988) Underreporting of substances use in a national longitudinal youth cohort: individual and interviewer effects. Public Opin Quart 52:100–124

    Article  Google Scholar 

  • Morris NA, Slocum LA (2010) The validity of self-reported prevalence, frequency, and timing of arrest: an evaluation of data collected using a life event calendar. J Res Crime Delinq 47(2):210–240

    Article  Google Scholar 

  • Northrup DA (1997) The problem of the self-report in survey research. Institute for Social Research, North York, Ontario, Canada

    Google Scholar 

  • Raudenbush SW, Bryk AS (2002) Hierarchical linear models. Sage, Thousand Oaks, CA

    Google Scholar 

  • Roberts J, Mulvey EP, Horney J, Lewis J, Arteret ML (2005) A test of two methods of recall for violent events. J Quant Criminol 21(2):175–193

    Article  Google Scholar 

  • Singleton R, Straits BC (1999) Approaches to social research, 3rd edn. Oxford University Press, New York, NY

    Google Scholar 

  • Sobell LC et al (1988) Reliability of a timeline method: accessing normal drinkers’ reports of recent drinking and a comparative evaluation across several populations. Br J Addict 83(4):393–402

    Article  Google Scholar 

  • Sudman S, Bradburn NM (1974) Response effects in surveys. Aldine Publishing Company, Chicago, IL

    Google Scholar 

  • Sudman S, Bradburn NM, Schwarz N (1996) Thinking about answers: the application of cognitive processes to survey methodology. Jossey-Bass Publishers, San Francisco, CA

    Google Scholar 

  • Thornberry T (1989) Panel effects and the use of self-reported measures of delinquency in longitudinal studies. In: Klein M (ed) Cross-national research in self-reported crime, delinquency. Kluwer Academic Publishers, Los Angeles

    Google Scholar 

  • Thornberry TP, Krohn M (2000) The self-report method for measuring delinquency and crime. Measurement and analysis of crime and justice, vol 4. National Institute of Justice, Washington, DC

  • Viera AJ, Garrett JM (2005) Understanding interobserver agreement: the kappa statistic. Fam Med 37:360–363

    Google Scholar 

  • Weis JG (1986) Issues in the measurement of criminal careers. In: Blumstein A et al (eds) Criminal careers and “career criminals”, vol II. National Academy Press, Washington, DC, pp 1–51

    Google Scholar 

  • Wheaton B, Gotlib IH (1997) Trajectories and turning points over the life course: concepts and themes. In: Gotlib IH, Wheaton B (eds) Stress and adversity over the life course: trajectories and turning points. Cambridge University Press, New York, NY, pp 1–25

    Chapter  Google Scholar 

  • Whitbeck LB, Hoyt DR, Yoder KA (1999) A risk amplification model of victimization and depressive symptoms among runaway and homeless adolescents. Am J Commun Psychol 27(2):273–296

    Article  Google Scholar 

  • Wittebrood K, Nieuwbeerta P (2000) Criminal victimization during one’s life course: the effects of previous victimization and patterns of routine activities. J Res Crime Delinq 37(1):91–122

    Article  Google Scholar 

  • Yacoubian GS (2003) Assessing the efficacy of the calendar method with Oklahoma city arrestees. J Crime Justice 26(1):117–131

    Google Scholar 

  • Yoshihama M, Clum K, Crampton A, Gillespie B (2002) Measuring the lifetime experience of domestic violence: application of the life history calendar method. Violence Vict 17(3):297–317

    Article  Google Scholar 

  • Yoshihama M, Gillespie B, Hammock AC, Belli RF, Tolman RM (2005) Does the life history calendar method facilitate the recall of intimate partner violence? Comparison of two methods of data collection. Social Work Res 29(3):151–163

    Google Scholar 

Download references

Acknowledgments

We gratefully acknowledge research support from the Department of Sociology and especially thank Bob Kaufman, then Chair, for listening to our pleas for help, the Criminal Justice Research Center (CJRC), the Initiative in Population Research (IPR), and the Center for Urban and Regional Analysis (CURA) at The Ohio State University. We are thankful for important contributions from several graduate students including: Rachael Gossett, James Hein, Brianne Hillmer, Ross Kaufman, Anita Parker, Grace Sherman, Matthew Valasik, and Shawn Vest. We thank Julie Horney for graciously providing us with a computer assisted version of a previously used event calendar instrument which provided the starting point for this project. We thank the anonymous reviewers and the Editors for their comments which we think helped improve the paper. We also thank the Ohio Department of Rehabilitation and Correction, especially Gayle Bickle and the staff at the Madison, London, Southeastern, and Richland correctional institutions, for facilitating this research. Finally, data collection would not have been possible without the good will and professionalism shown by the prisoners who agreed to participate without compensation.

Author information

Authors and Affiliations

  1. California State University at Chico, Chico, CA, USA

    James E. Sutton

  2. The Ohio State University, Columbus, OH, USA

    Paul E. Bellair

  3. Ohio Department of Rehabilitation and Correction, Columbus, OH, USA

    Brian R. Kowalski

  4. University of Oregon, Eugene, OR, USA

    Ryan Light

  5. Ohio University, Lancaster, OH, USA

    Donald T. Hutcherson

Authors
  1. James E. Sutton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul E. Bellair
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Brian R. Kowalski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ryan Light
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Donald T. Hutcherson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul E. Bellair.

Additional information

Paul E. Bellair is principal investigator.

An erratum to this article can be found at http://dx.doi.org/10.1007/s10940-011-9132-z

Appendix

Appendix

See Table 7.

Table 7 Cross-tabulation of components of the self-reported crime measure (drug, property, and violent crime) during test and re-test
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sutton, J.E., Bellair, P.E., Kowalski, B.R. et al. Reliability and Validity of Prisoner Self-Reports Gathered Using the Life Event Calendar Method. J Quant Criminol 27, 151–171 (2011). https://doi.org/10.1007/s10940-010-9101-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10940-010-9101-y

Keywords

Search

Navigation