Skip to main content
SpringerLink
Log in

Missing Data in Sociological Research: An Overview of Recent Trends and an Illustration for Controversial Questions, Active Nonrespondents and Targeted Samples

  • Published:
The American Sociologist Aims and scope Submit manuscript

Abstract

In an age of telemarketers, spam emails, and pop-up advertisements, sociologists are finding it increasingly difficult to achieve high response rates for their surveys. Compounding these issues, the current political and social climate has decreased many survey respondents’ likelihood of responding to controversial questions, which are often at the heart of much research in the discipline. Here we discuss such implications for survey research in sociology using: a content analysis of the prevalence of missing data and survey research methods in the most cited articles in top sociology journals, a case study highlighting the extraction of meaningful information through an example of potential mechanisms driving the non-random missing data patterns in the Religion Among Academic Scientists dataset, and qualitative responses from non-responders in this same case. Implications are likely to increase in importance given the ubiquitous nature of survey research, missing data, and privacy concerns in sociological research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. Harzig, A.W. 2011. Publish or Perish, version #3, available at www.harzing.com/pop.htm

  2. In order to ensure an equal coverage across the three journals, the top 35 articles since 2000 were identified and downloaded for further examination of data collection techniques and handling of any noted missing data. We further standardized the selection process by total cites per year so not to give more weight to articles published earlier in the decade based on total cites. We do understand that there is still a lag in regards to publication and citation timing, but believe that we have collected a representative sample of the most visible sociology publications over the past decade.

  3. After the RAAS study began, the “Top American Research Universities” project moved to Arizona State University. See http://mup.asu.edu/, accessed April 17, 2009.

  4. Psyc 17, conducted January 3, 2006.

  5. This individual did not participate in the survey.

  6. See Ladd and Lipset, “The Politics of Academic Natural Scientists and Engineers.”

  7. The 1998 GSS had 2,832 respondents, although only half of the sample was asked the expanded set of religion and spirituality questions. The 2004 GSS had 2,812 respondents. Where possible, we used data from the GSS 2006 for the comparisons of scientists with the general population. See Davis et al. (2007).

References

  • Abraham, K., Maitland, A., & Bianchi, S. (2006). Nonresponse in the American time use survey: who is missing from the data and how much does it matter? Public Opinion Quarterly, 70(5), 676–703.

    Article  Google Scholar 

  • Allison, P. (2002). Missing data (Sage quantitative applications in the social sciences series). Sage Publishing.

  • Alosh, M. (2009). The impact of missing data in a generalized integer-valued autoregression model for count data. Journal of Biopharmaceutical Statistics, 19, 1039–1054.

    Article  Google Scholar 

  • Babbie, E. (2007). Practice of social research (11th ed.). Thomson: Wadsworth Publishing.

    Google Scholar 

  • Davis, J. A., Smith, T. W., Marsden, P. V. (2007). General social surveys, 1972–2006 [Cumulative File]. ICPSR Study Number 4697.

  • Dawid, A. P. (1984). Statistical theory: the prequential approach (with discussion). Journal of the Royal Statistical Society A, 147, 278–292.

    Article  Google Scholar 

  • DeSouza, C. M., Legedza, A. T. R., & Sankoh, A. J. (2009). An overview of practical approaches for handling missing data in clinical trials. Journal of Biopharmaceutical Statistics, 19, 1055–1073.

    Article  Google Scholar 

  • Ecklund, E. H. (2010). Science Vs. religion: What scientists really think. New York: Oxford University Press.

    Google Scholar 

  • Ecklund, E. H., & Scheitle, C. (2007). Religion among academic scientists: distinctions, disciplines, and demographics. Social Problems, 54(2), 289–307.

    Article  Google Scholar 

  • Ecklund, E. H., Park, J. Z., & Veliz, P. T. (2008). Secularization and religious change among elite scientists: a cross-cohort comparison. Social Forces, 86(4), 1805–1840.

    Article  Google Scholar 

  • Edgell, P., Gerteis, J., & Hartmann, D. (2006). Atheists as ‘other’: moral boundaries and cultural membership in American Society. American Sociological Review, 71, 211–234.

    Article  Google Scholar 

  • Evans, J. H., & Evans, M. S. (2008). Religion and science: beyond the epistemological conflict narrative. Annual Review of Sociology, 34, 87–105.

    Article  Google Scholar 

  • Garcia, R. I., Ibrahim, J. G., & Zhu, H. (2010). Variable selection in the cox regression model with covariates missing at random. Biometrics, 66, 97–104.

    Article  Google Scholar 

  • Gelman, A., Meng, X. L., & Stern, H. S. (1996). Posterior predictive assessment of model fitness via realized discrepancies (with discussion). Statistica Sinica, 6, 733–807.

    Google Scholar 

  • Gelman, A., Van Mechelen, I., Verbeke, G., Heitjan, D. F., & Meulders, M. (2005). Multiple imputation for model checking: completed data plots with missing and latent data. Biometrics, 61, 74–85.

    Article  Google Scholar 

  • Griliches, Z. (1986). Comment on Behrman and Taubman. Journal of Labor Economics, 4(3), S146–S150.

    Article  Google Scholar 

  • Groves, R. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675.

    Article  Google Scholar 

  • Groves, R., Couper, M., Presser, S., Singer, E., Tourangeau, R., Piani, G., & Nelson, L. (2006). Experiments in producing nonresponse bias. Public Opinion Quarterly, 70(5), 646–675.

    Article  Google Scholar 

  • Haung, R., Liang, Y., & Carrierre, K. C. (2005). The role of proxy information in missing data analysis. Statistical Methods in Medical Research, 14, 457–471.

    Article  Google Scholar 

  • Khoshgoftaar, T. M., Van Hulse, J., Seiffert, C., & Zhao, L. (2007). The multiple imputation quantitative noise corrector. Intelligent Data Analysis, 11, 245–263.

    Google Scholar 

  • Lenski, G. (1961). The religious factor. Garden City: Doubleday.

    Google Scholar 

  • Little, R. J. A. (1995). Modeling the drop-out mechanism in repeated measures studies. Journal of the American Statistical Association, 90, 1112–1121.

    Article  Google Scholar 

  • Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York: Wiley.

    Google Scholar 

  • Martinussen, T., Nord-Larsen, T., & Johannsen, V. K. (2008). Estimating forest cover in the presence of missing observations. Scandinavian Journal of Forest Research, 23, 266–271.

    Article  Google Scholar 

  • Maxim, P. (1999). Quantitative research methods in the social sciences. Oxford University Press.

  • Montiel-Overall, P. (2006). Implications of missing data in survey research. Canadian Journal of Information and Library Science, 30(3/4), 241–269.

    Google Scholar 

  • Neuman, L. (2003). Social research methods: qualitative and quantitative approaches (5th ed.). Allyn and Bacon Publishing.

  • Olson, K. (2006). Survey participation, nonresponse bias, measurement error bias, and total bias. Public Opinion Quarterly, 70(5), 737–758.

    Article  Google Scholar 

  • Paik, M. C. (2004). Nonignorable missingness in matched case–control data analysis. Biometrics, 60, 306–314.

    Article  Google Scholar 

  • PEW Research Center for the People and Press. (2004). Polls face growing resistance, but still representative. Survey reports (April 20th).

  • Porter, J. R., Cossman, R., & James, W. L. (2009). Research note: imputing large group averages for missing data, using rural–urban continuum codes for density driven industry sectors. Journal of Population Research, 26, 273–278.

    Article  Google Scholar 

  • Rose, R. A., & Fraser, M. W. (2008). A simplified framework for using multiple imputation in social work research. Social Work Research, 32(3), 171–178.

    Article  Google Scholar 

  • Rubin, D. B. (1976). Inference and missing data. Biometrika, 63, 581–592.

    Article  Google Scholar 

  • Rubin, D. B. (1984). Multiple imputation for nonresponse in surveys. New York: Wiley.

    Google Scholar 

  • Satten, G. A., & Carroll, R. J. (2000). Conditional and unconditional categorical regression models with missing covariates. Biometrics, 56, 384–388.

    Article  Google Scholar 

  • Schrecker, E. (2006). Worse than McCarthy. pp. B20, February 10, 2006, in The chronicle of higher education.

  • Southern, D. A., Norris, C. M., Quan, H., Shrive, F. M., Diane Galbraith, P., Humphries, K., Gao, M., Knudtson, M. L., & Ghali, W. A. (2008). An administrative data merging solution for dealing with missing data in a clinical registry: adaptation from ICD-9 to ICD-10. BMC Medical Research Methodology, 8(1), 1–9.

    Article  Google Scholar 

  • Verbeke, G., & Molenberghs, G. (2000). Linear mixed models for longitudinal data. New York: Springer.

    Google Scholar 

  • Verbeke, G., & Molenberghs, G. (2010). Arbitrariness of models for augmented and coarse data, with emphasis on incomplete data and random effects models. Statistical Modeling, 10(4), 391–419.

    Article  Google Scholar 

  • Wright, J. D., & Marsden, P. V. (2010). In P. V. Marsden & J. D. Wright (Eds.), The Handbook of Survey Research (2nd ed.). United Kingdom: Emerald Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. City University of New York, Brooklyn College and Graduate Center, 218 Whitehead Hall, Brooklyn College, 2900 Bedford Ave, Brooklyn, NY, 11220, USA

    Jeremy R. Porter

  2. Department of Sociology, Rice University, Houston, TX, USA

    Elaine Howard Ecklund

Authors
  1. Jeremy R. Porter
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elaine Howard Ecklund
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jeremy R. Porter.

Additional information

This research was supported by a grant from the John Templeton Foundation (grant #11299; Elaine Howard Ecklund, PI). The authors also wish to thanks and acknowledge that Kelsey Pedersen provided invaluable help with manuscript editing and formatting.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Porter, J.R., Ecklund, E.H. Missing Data in Sociological Research: An Overview of Recent Trends and an Illustration for Controversial Questions, Active Nonrespondents and Targeted Samples. Am Soc 43, 448–468 (2012). https://doi.org/10.1007/s12108-012-9161-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12108-012-9161-6

Keywords

Search

Navigation