Convenience sampling often is necessary when conducting research where there is no frame for a probability sample. This might include samples of people with rare diseases, persons recovering from compulsive behaviors, or those involved in some stigmatized activity. When there is no reliable list from which to draw a statistical sample, the best alternative may be to rely on sampling by referrals, sometimes called snowball sampling (Welch 1975), respondent-driven sampling (Heckathorn 2002), or multiplicity sampling (Rothbart, Fine, and Sudman 1982).
Referral sampling usually involves identifying individuals who meet inclusion criteria, gaining their cooperation, and then asking them to recruit additional respondents with the same conditions (Heckathorn 2002). However, professional referral sampling has received less notice as a methodology, but is used in health and mental health research. Professional referral sampling involves selecting respondents through an intermediary who provides professional services to the subject. Intermediaries might include pediatricians (Stille et al. 2007), school nurses (Lee et al. 2009), or substance abuse clinicians (Fals-Stewart, Birchler, and Kelley 2006).
We compare our experiences conducting two surveys depending on referral sampling. In each case, physicians acted as intermediaries, helping identify respondents with specific health conditions. Several related lessons are reported and presented for further hypothesis testing.
Methods
In both of our projects, obstetrician-gynecologists (OBGs) were members of the Collaborative Ambulatory Research Network (CARN). The CARN is an opt-in panel of OBGs willing to participate in research activities. This list is maintained by the American College of Obstetricians and Gynecologists (ACOG). As reported elsewhere, the CARN’s membership compares favorably to the population of OBGs practicing in the United States (Anderson et al. 2008). CARN members do not receive monetary incentives for participating in the panel, or specific research activities.
In both surveys, OBGs were asked to distribute questionnaires to eligible adult patients. Questionnaires in both surveys were self-administered, paper-and-pencil instruments. The OBGs were provided with written instructions describing the eligibility criteria and a summary of the purpose of the survey, the voluntary nature of participation, and confidentiality protections. All survey instruments and materials were reviewed by an Institutional Review Board. Both protocols instructed patients to complete the survey in the waiting room and return it to office staff in a sealed envelope. Below we describe distinguishing features.
Hormone Therapy Study In this 2004 study, OBGs distributed questionnaires to any visiting adult patient. A two-page questionnaire asked women about their attitudes concerning hormonal treatments. OBGs were asked to collect the forms and return the completed forms to ACOG within three weeks. Additional details of this study are reported elsewhere (Power and Schulkin 2006).
Prenatal Diabetes Study Conducted in 2008 and 2009, we asked OBGs to identify patients of child-bearing age who recently either had gestational diabetes (GDM) or pre-existing diabetes (PEDM). These patients – generally 40 year old and younger – were asked to complete a four-page questionnaire in the waiting room. Each physician was asked to recruit:
- One Caucasian woman who had GDM
- One African-American woman who had GDM,
- One Hispanic woman who had GDM,
- One Caucasian woman with (or who had) pre-existing diabetes during pregnancy, and
- Either one African-American or one Hispanic woman with (or who had) PEDM.
OBGs were asked to give each eligible patient a questionnaire. Over a three month period, at about four week intervals, we sent reminder mailings to OBGs asking for continued cooperation. After three months of data collection, a one-page survey was sent to OBGs who had themselves completed the survey, but who returned fewer than two patient surveys (n=291). We asked these OBGs if they thought they would be able to recruit a sufficient number of patients to meet our completion goals.
Results
The Hormone Therapy Study gathered 1,659 completed surveys during the three-week data collection period. Responses were received from approximately 14% of CARN members in 39 states. Because these results satisfied the study objectives, no attempt was made to recruit additional patients. Of the respondents, 332 were age 40 or younger, the target age of the Prenatal Diabetes Study target population.
By contrast, following a much longer three-month data collection period, the Prenatal Diabetes Study yielded 176 questionnaires (97 GDM and 48 PEDM) from 69 OBG practices, or about 7% of the eligible OBGs in the sample. Due to the low participation of OBGs and patients, a brief follow-up physician questionnaire was developed to determine whether it was advisable to extend the data collection period. Response to this mailed questionnaire (174 questionnaires) suggested that, even after three months in the field, 74% of OBGs who replied were willing to continue. However, on average, the responding physicians said they see only about 7 patients per month who experienced diabetes during pregnancy.
OBGs reported to us that they could have recruited more patients than they did were it not for the racial/ethnic quotas. More than one-third (36%) said they lacked eligible patients. The “flow” of Latina and African American women with particular diabetes types was a significant obstacle (see Table 1). For example, one physician said he could have recruited numerous Asian respondents, but no African Americans.
OBGs also indicated other obstacles hindered the process. Approximately 46% identified time as a barrier; 14% said they would require administrative approvals (i.e. Institutional Review Board) in order to distribute surveys; and others mentioned that they refer diabetic patients to other specialists, patient resistance, or their own forgetfulness.
Discussion
The Hormone Therapy Study received approximately twice as many completed surveys from women 40 and younger, from twice as many practices, in a three-week field period as compared to the Prenatal Diabetes Study’s three-month long field period. Using the OBGs reported expected patient flow of approximately 7 patients with pregnancy-related diabetes per month, we would have anticipated approximately 1,000 Prenatal Diabetes surveys. This would have been on par with the final number of patient forms received in the Hormone Therapy Study, and quite larger if comparing only those women 40 and younger.
Rather, the Prenatal Diabetes Study had a substantially lower number of completed surveys and required substantially greater effort. We recognize a notable distinction between the designs of these two projects, namely, a) subject matter and b) respondent burden of a two-page vs. a four-page questionnaire. Still, these results suggest that the patient sample design in the Prenatal Diabetes Study suffered from several factors as compared with the relative success of the Hormone Therapy Study.
First, asking professionals to identify subjects in rare populations is difficult and more so when additional sampling criteria are imposed. Perhaps we would have achieved a better result had we applied analytical rules for race and diabetes type after the data collection stage. Second, intermediaries may have a low “flow” of clientele with rare conditions like diabetes. Using the professional intermediary approach depends on the OBG remembering to offer the survey during rare visits with diabetic patients. Third, the demographic characteristics of the eligible respondents may not be distributed evenly across the OBGs’ practices. Adding selection criteria onto an already rare population may be an obstacle to gaining the desired yield within a limited time frame. Professional intermediaries reported that it would be easier to involve subjects if none of the racial/ethnic characteristics were imposed. We think the selection criteria contributed to the necessity for an extended field period, and added burden on the physicians. Fourth, survey length, as already has been shown, might have undermined patient cooperation.
Anecdotal evidence from physicians helps support our tentative conclusions. A more systematic study would help measure the degree to which the rarity of a particular disease and demographic quota requirements separately impede yield in professional-intermediary convenience sample surveys. Public health and survey researchers should carefully weigh the value of stringent quota requirements against the yield of completed surveys in deciding how to implement professional-referral design surveys.