To tell the truth: Measuring concordance in multiply reported network data☆
Introduction
Network analysts are often interested in the potential flow of goods between nodes (Potterat et al., 2002, Woodhouse et al., 1994, Moody, 2002). Network measures, however, assume accurate data on the ties connecting actors (Liljeros et al., 2003, Marsden, 1990) and any misreported relationships can potentially result in biased estimates of relevant statistics (Butts, 2003, Rothenberg et al., 1995). For example, linking claims about connectivity in sex and needle-sharing networks to epidemic potential requires reliably reported sex and needle partnerships (Brewer et al., 1999, Brewer et al., 2006). Unfortunately, data limitations make reliability tests impossible for most studies, and researchers are forced to take such data at face value. With complex linked ego-network designs, this means that researchers are often forced to use single reports of individuals about their own behaviors and the behaviors of those to whom they are connected (Moody et al., 2005, Rothenberg et al., 2001, Burt, 1985). When one has multiple reports of the same tie, however, there is a unique opportunity to examine network reliability.
In this paper, we examine the data reliability in the Colorado Springs, Project 90 data. We compare agreement across multiple reports of the sexual, drug-sharing and social ties for reliability. We examine the consistency of an individual's reports of the same information over time, their agreement with others when reporting ties of which they are members, and when reporting ties in which they are not involved. Overall, the results from these data suggest remarkable reporting reliability. These results suggest general confidence in high-risk activity data and have specific implications for the data collection approach used to generate the Project 90 data that could prove useful for future studies (Moody et al., 2005, Rothenberg et al., 2001, Friedman et al., 1997).
Section snippets
Background
Past research has evaluated reliability for network reports for both general network data (e.g., McCarty et al., 1997, Marsden, 1993, Burt, 1986) and sexual and drug user data specifically (Bell et al., 2000, Brewer et al., 1999). The levels of reliability vary greatly across these studies and have not yet yielded generally accepted reliability standards.
Marsden (1990) summarized many of the previous evaluation studies, finding 40–60% accuracy of communication network reporting in the now
Methods
We first calculate frequencies and proportions of reporting concordance for each of the five comparisons described below by tie type. We make each comparison for social, sexual,6
Respondent–respondent contact ties
Table 1 presents the frequency of concordant reports for ties where both nodes are respondents for all tie types.11
Reliability of contact tie nominations
For the most part, when two individuals report on contact with each other they agree on the nature of the tie. This suggests that Project 90 respondents provided reliable information for contact ties. While needle-sharing ties are slightly less reliable than the other types of ties, all are significantly better than previous literature would suggest. The higher rates of agreement for sex ties than for drug or needle sharing relationships may indicate forgetting of such ties, which has been
Conclusion/implications
Project 90 respondents were remarkably reliable when reporting about the extent and timing of their own relationships. This is particularly relevant for processes such as disease diffusion, as it allows one to construct timed networks which can more accurately represent the epidemic potential of the network (Moody, 2002, Morris and Kretzchmar, 1995). While current uses of such data often ignore time, we suggest that researchers incorporate the timed data to more fully match available data to
References (39)
- et al.
Informant accuracy in social network research IV: A comparison of clique-level structure in behavioral and cognitive data
Social Networks
(1980) Forgetting in the recall-based elicitation of personal and social networks
Social Networks
(2000)Network inference, error, and informant (in)accuracy: a Bayesian approach
Social Networks
(2003)Social networks and the spread of infectious diseases: the AIDS example
Social Science Medicine
(1985)- et al.
Sexual networks: implications for the transmission of sexually transmitted infections
Microbes and Infection
(2003) The reliability of network density and composition measures
Social Networks
(1993)- et al.
Eliciting representative samples of personal networks
Social Networks
(1997) - et al.
Concurrent partnerships and transmission dynamics in networks
Social Networks
(1995) - et al.
Choosing a centrality measure: epidemiologic correlates in the Colorado Springs study of social networks
Social Networks: Special Edition on Social Networks and Infectious Disease: HIV/AIDS
(1995) - et al.
The significance of sexual partner contact networks for the transmission dynamics of HIV
Journal of Acquired Immune Deficiency Syndrome
(1990)
Consensus analysis of three-way social network data
Journal of Mathematical Sociology
Partner concordance in reports of joint risk behaviors
Journal of Acquired Immune Deficiency Syndromes
Informant accuracy in social network data II
Human Communications Research
Snowball sampling: problems and techniques in chain-referral sampling
Social Methods and Research
Forgetting as a cause of incomplete reporting of sexual and drug injection partners
Sexually Transmitted Diseases
Cited by (0)
- ☆
This work is supported by NIH grants DA12831 and HD41877. We would like to thank Steve Muth and Martina Morris for continued support in access to and understanding of these data. We also thank John Potterat, David C. Bell, Devon Brewer, the Social Structure Research Group at Ohio State University, and anonymous reviewers from Social Networks for helpful comments on previous drafts of this paper. Any mistakes remaining are the sole responsibility of the author(s). Please direct any correspondence to jimi adams, Department of Sociology, 300 Bricker Hall, 190 N. Oval Mall, Columbus, OH 43210 ([email protected]).