Mail surveys : Obsolescent model or valuable instrument in general practice research ?

Question under study: Due to low response rates mail surveys have been called into question as research instruments in general practice. The most effective actions to reduce non-response, such as financial incentives and complex follow-up procedures, are costly. We investigated whether a good response rate is achievable with a less costly survey design, and examined the effect of increased response rates due to repeated follow-ups on survey results. Methods: In a mail survey on drug interactions among 2000 general practitioners in south-west Germany, most well-known criteria influencing response rates were met except financial incentives. A four stage design with two reminders was applied and the time course of response was recorded. Results after both reminders were calculated with 95% confidence intervals and compared with initial results using the Jonckheere-Terpstra test with correction for multiple testing. A p <0.01 was considered significant. Results: Although we did not provide financial incentives we achieved a response rate of 60.8% with our survey design. The first reminder with a simple postcard was almost three times less effective than the second reminder including another copy of the questionnaire. For only two survey questions, the answers of late respondents differed significantly from those of initial respondents (p <0.01). For these two questions, cumulative results after both reminders never differed from initial results by more than 3.7%. Conclusion: Even if financial incentives are not affordable, good response rates can be obtained among general practitioners when surveys are meticulously planned and implemented. Potential non-response bias introduced by those general practitioners who do not answer despite numerous reminders, cannot be tested by comparing early and late respondents. Therefore, we suggest that the impact of reminders on survey results should be assessed early. If no bias can be detected one further reminder with a copy of the questionnaire might result in estimates very similar to those after numerous reminders.

Question under study: Due to low response rates mail surveys have been called into question as research instruments in general practice.The most effective actions to reduce non-response, such as financial incentives and complex follow-up procedures, are costly.We investigated whether a good response rate is achievable with a less costly survey design, and examined the effect of increased response rates due to repeated follow-ups on survey results.
Methods: In a mail survey on drug interactions among 2000 general practitioners in south-west Germany, most well-known criteria influencing response rates were met except financial incentives.A four stage design with two reminders was applied and the time course of response was recorded.Results after both reminders were calculated with 95% confidence intervals and compared with initial results using the Jonckheere-Terpstra test with correction for multiple testing.A p <0.01 was considered significant.
Results: Although we did not provide financial incentives we achieved a response rate of 60.8% with our survey design.The first reminder with a simple postcard was almost three times less effec-tive than the second reminder including another copy of the questionnaire.For only two survey questions, the answers of late respondents differed significantly from those of initial respondents (p <0.01).For these two questions, cumulative results after both reminders never differed from initial results by more than 3.7%.
Conclusion: Even if financial incentives are not affordable, good response rates can be obtained among general practitioners when surveys are meticulously planned and implemented.Potential non-response bias introduced by those general practitioners who do not answer despite numerous reminders, cannot be tested by comparing early and late respondents.Therefore, we suggest that the impact of reminders on survey results should be assessed early.If no bias can be detected one further reminder with a copy of the questionnaire might result in estimates very similar to those after numerous reminders.

Key words: questionnaire; mail survey; nonresponse; general practice; postal reminder; drug interactions
Response rates in postal questionnaire surveys among general practitioners have been falling over the last years because of an increasing number of commercial requests and other reasons such as increasing paperwork and time constraints [1].However, research in general practice is important to identify common problems in our health care system and to develop practical solutions [2].
One of the most significant factors enhancing response rates is a monetary incentive [3].However, such incentives are often not affordable for adequately powered surveys.Another effective method to increase response rates is sending sev-eral reminders.Because four postal reminders (each including a copy of the questionnaire [1]) also raise substantial costs, it is worth investigating whether such expensive follow-up procedures will actually change the results obtained from initial respondents.
In the context of a mail survey among general practitioners [4] we investigated which response rate is achievable without financial incentives.Furthermore, we examined the effect of increased response rates due to repeated follow-ups on survey results.

Introduction
We conducted a questionnaire survey among 2000 randomly selected general practitioners in Baden-Württemberg (large federal state in south-west Germany) on perception and management of drug interactions (16 questions), which was described in detail elsewhere [4].We met most well-known criteria that increase response rates according to a meta-analysis [3], except financial incentives.Those criteria are a short questionnaire, personalised letters, pre-contact, follow-up contact, essential questions at the beginning of the questionnaire, and academic origin of the investigation.As an overall design we chose a four stage intervention: A letter of announcement was sent to explain the rationale of the study.One week later the questionnaire and a pre-addressed return envelope were sent.Non-respondents were sent a reminder postcard two weeks later.Another two weeks later a further reminder was sent to non-respondents including a copy of the questionnaire.
We recorded the time of response with respect to the interventions.The results of the first part of the questionnaire (questions on perception and current management of drug interactions) were calculated with 95% confidence intervals after each intervention using the SAS 8.02 software package (SAS Institute Inc., Cary, NC, USA).Item completeness is reported as mean (± standard deviation) for each group of respondents.The Jonckheere-Terpstra test for ordered differences was applied to determine differences between initial results and results obtained after postal reminders.Due to multiple testing, a p <0.01 was considered significant.

Results
The overall response rate of our mail survey was 60.8% (1216/2000) (figure 1).The initial response rate after the announcement and the first questionnaire was 33.0%.The first reminder with a simple postcard was almost three times less effective than the second reminder including another copy of the questionnaire.
The analysis of the results after both reminders revealed that for only two questions, the answers of late respondents differed significantly from those of initial respondents (table 1).However, because of the lower proportion of late respondents compared with initial respondents, overall results never differed from initial results by more than 3.7%.

Discussion
Despite falling response rates of mail surveys among general practitioners, our study has shown that it is possible to motivate more than 50% of contacted general practitioners in participating in a mail survey without financial incentives.Especially when other factors that enhance response rates [2], such as minimum workload to fill out the questionnaire and comprehensive explanation of the study rationale, are being considered.
Our study confirms that providing the questionnaire with the reminder yields higher response rates than a reminder postcard [5].We presume that many general practitioners dispose the questionnaire immediately [6], making it no longer available on arrival of the reminder.This aspect should be considered when only one reminder is planned.
Most studies compared early and late responding physicians only with respect to socio-demo-graphic characteristics.However, for decision making, the potential influence of reminders on the responses is more important.Because sending two reminders obviously did not alter the overall conclusion of the survey, we suggest that the impact of reminders on survey results should be assessed early.If no bias can be detected, one further reminder with a copy of the questionnaire instead of numerous reminders might result in estimates very similar to those after numerous reminders.
Two limitations of this observational study merit discussion.First, because we did not have a control group we could neither strictly separate the effects of individual interventions nor could we determine a potential influence of the perceived importance of the topic on the response rate.Second, we could not assess potential non-response bias.However, to correct a sample for non-response, few options exist (eg multiple imputation, propensity weighting [7]).All procedures require additional information on non-respondents which generally is not available in surveys.Time of postal return of a questionnaire is nearly always available, and therefore many attempts to use this variable as an indirect measure of response propensity have been published [8].The absence of a relationship between a dependent variable and time of response for respondents, does not guarantee that remaining non-respondents are similar to respondents.However, if there is no relationship between time of response and survey variables for those who respond, one can consider limiting the number of reminders.Hence, an early analysis of each wave of response might help to decide early in a survey whether additional measures are indeed promising or are likely not to add information to the overall conclusion.

Figure 1
Figure 1Cumulative response in a mailed questionnaire survey on drug interactions among 2000 randomly selected German general practitioners.∆ absolute increase in the number of general practitioners having responded within respective time period (% of selected sample of general practitioners).* Response to second reminder within the same time interval as passed by after the first reminder before the next intervention.** Response to second reminder in remaining study period.

Table 1
Prevalence of answers to seven questions on perception and current management of drug interactions among 1216 responding general practitioners in a mail survey in Germany, May 2003.Percentages calculated for the total number of respondents and separately for initial respondents and respondents to two consecutive interventions.Claude Martin, Marseille, France Prof. Josef Patsch, Innsbruck, Austria Prof. Luigi Tavazzi, Pavia, Italy We evaluate manuscripts of broad clinical interest from all specialities, including experimental medicine and clinical investigation.We look forward to receiving your paper!Guidelines for authors: http://www.smw.ch/set_authors.htmlAll manuscripts should be sent in electronic form, to: Official journal of the Swiss Society of Infectious disease the Swiss Society of Internal Medicine the Swiss Respiratory Society 6 Kaner EF, Haighton CA, McAvoy BR. "So much post, so busy with practice-so, no time!": a telephone survey of general practitioners' reasons for not participating in postal questionnaire surveys.Br J Gen Pract 1998;48:1067-9.7 Groves RM, Dillman DA, Eltinge JL, Little RJA.Survey Nonresponse.New York: John Wiley; 2002.8 Schnell R. Nonresponse in Bevölkerungsumfragen. Opladen: Leske + Budrich; 1997.