Elsevier

Social Networks

Volume 29, Issue 1, January 2007, Pages 44-58
Social Networks

To tell the truth: Measuring concordance in multiply reported network data

https://doi.org/10.1016/j.socnet.2005.11.009Get rights and content

Abstract

Social network data must accurately reflect actors’ relationships to properly estimate network features. Here, we examine multiple reports of sexual, drug-sharing and social tie data on high-risk networks in Colorado Springs. By comparing multiple reports on the same ties, we can evaluate the reliability of this study's network data. Our findings suggest that these data have a high level of reporting agreement. From these findings, we discuss implications for analysis of these and similar data and provide suggestions for future social network data collection efforts.

Introduction

Network analysts are often interested in the potential flow of goods between nodes (Potterat et al., 2002, Woodhouse et al., 1994, Moody, 2002). Network measures, however, assume accurate data on the ties connecting actors (Liljeros et al., 2003, Marsden, 1990) and any misreported relationships can potentially result in biased estimates of relevant statistics (Butts, 2003, Rothenberg et al., 1995). For example, linking claims about connectivity in sex and needle-sharing networks to epidemic potential requires reliably reported sex and needle partnerships (Brewer et al., 1999, Brewer et al., 2006). Unfortunately, data limitations make reliability tests impossible for most studies, and researchers are forced to take such data at face value. With complex linked ego-network designs, this means that researchers are often forced to use single reports of individuals about their own behaviors and the behaviors of those to whom they are connected (Moody et al., 2005, Rothenberg et al., 2001, Burt, 1985). When one has multiple reports of the same tie, however, there is a unique opportunity to examine network reliability.

In this paper, we examine the data reliability in the Colorado Springs, Project 90 data. We compare agreement across multiple reports of the sexual, drug-sharing and social ties for reliability. We examine the consistency of an individual's reports of the same information over time, their agreement with others when reporting ties of which they are members, and when reporting ties in which they are not involved. Overall, the results from these data suggest remarkable reporting reliability. These results suggest general confidence in high-risk activity data and have specific implications for the data collection approach used to generate the Project 90 data that could prove useful for future studies (Moody et al., 2005, Rothenberg et al., 2001, Friedman et al., 1997).

Section snippets

Background

Past research has evaluated reliability for network reports for both general network data (e.g., McCarty et al., 1997, Marsden, 1993, Burt, 1986) and sexual and drug user data specifically (Bell et al., 2000, Brewer et al., 1999). The levels of reliability vary greatly across these studies and have not yet yielded generally accepted reliability standards.

Marsden (1990) summarized many of the previous evaluation studies, finding 40–60% accuracy of communication network reporting in the now

Methods

We first calculate frequencies and proportions of reporting concordance for each of the five comparisons described below by tie type. We make each comparison for social, sexual,6

Respondent–respondent contact ties

Table 1 presents the frequency of concordant reports for ties where both nodes are respondents for all tie types.11

Reliability of contact tie nominations

For the most part, when two individuals report on contact with each other they agree on the nature of the tie. This suggests that Project 90 respondents provided reliable information for contact ties. While needle-sharing ties are slightly less reliable than the other types of ties, all are significantly better than previous literature would suggest. The higher rates of agreement for sex ties than for drug or needle sharing relationships may indicate forgetting of such ties, which has been

Conclusion/implications

Project 90 respondents were remarkably reliable when reporting about the extent and timing of their own relationships. This is particularly relevant for processes such as disease diffusion, as it allows one to construct timed networks which can more accurately represent the epidemic potential of the network (Moody, 2002, Morris and Kretzchmar, 1995). While current uses of such data often ignore time, we suggest that researchers incorporate the timed data to more fully match available data to

References (39)

  • W.H. Batchelder et al.

    Consensus analysis of three-way social network data

    Journal of Mathematical Sociology

    (1997)
  • D.C. Bell et al.

    Partner concordance in reports of joint risk behaviors

    Journal of Acquired Immune Deficiency Syndromes

    (2000)
  • H.R. Bernard et al.

    Informant accuracy in social network data II

    Human Communications Research

    (1977)
  • P. Biernacki et al.

    Snowball sampling: problems and techniques in chain-referral sampling

    Social Methods and Research

    (1981)
  • Bignami-Van Assche, S., Watkins, S.C., 2004. Husband–wife disagreement in rural Malawi: a longitudinal analysis. Paper...
  • D.D. Brewer et al.

    Forgetting as a cause of incomplete reporting of sexual and drug injection partners

    Sexually Transmitted Diseases

    (1999)
  • Brewer, D.D., Rothenberg, R.B., Muth, S.Q., Roberts, J.M., Potterat, J.J., 2006. Agreement in reported sexual...
  • Burt, R.S., 1985. Network Items and the General Social Survey, GSS Technical Report No. 53, Chicago: NORC, Fall, 1984....
  • Burt, R.S., 1986. A Note on Sociometric Order in the General Social Survey Network Data, GSS Technical Report No. 60,...
  • Cited by (0)

    This work is supported by NIH grants DA12831 and HD41877. We would like to thank Steve Muth and Martina Morris for continued support in access to and understanding of these data. We also thank John Potterat, David C. Bell, Devon Brewer, the Social Structure Research Group at Ohio State University, and anonymous reviewers from Social Networks for helpful comments on previous drafts of this paper. Any mistakes remaining are the sole responsibility of the author(s). Please direct any correspondence to jimi adams, Department of Sociology, 300 Bricker Hall, 190 N. Oval Mall, Columbus, OH 43210 ([email protected]).

    View full text