No Increase in Response Rate by Adding a Web Response Option to a Postal Population Survey: A Randomized Trial

Background There is substantial interest in use of the Internet for surveys, but there have been few health-oriented, large, randomized trials of general population surveys on the Internet. It is unclear whether providing the option to respond via Internet increases the response rate, and to what degree the results will differ. Objective The aim of the study was to evaluate changes in response rate and outcomes in a postal respiratory health survey by adding an optional Web response alternative. Methods This was a randomized trial of a random sample of 4213 permanent residents of Norway, aged 20-40 years. Participants were randomized into a traditional survey arm, where they were asked to return the survey by mail, and an arm where they were also offered the option to respond via a Web form. Results A total of 1928/4213 subjects responded, a response rate of 45.8% across both arms. The total response rate was 44.8% (944/2105) in the postal plus optional Internet response arm and 46.7% (984/2108) in the usual postal survey arm, with no statistically significant difference between the randomized groups (P = .24). In the optional Internet arm, 8.3% (175/2105) of the sample responded using the Internet and 36.5% (769/2105) responded by post. Thus, Internet response was chosen by 18.5% (175/944) of those who replied in the optional Internet arm. In the multivariate analysis, Internet response was associated with being male, frequency and type of Internet access (home users more likely to respond by Internet than work users), and smoking habit, with current smokers being more likely to be Internet responders. 57% preferred postal response (1102/1928), 38% preferred Internet response (733/1928), and 3% preferred telephone interview (54/1928), with no difference between randomization arms (P = .56). But among those who indicated that they preferred the Internet response and who were randomized to the optional Internet arm, only 47% actually chose the Internet response. Asthma prevalence was higher among participants choosing the Internet response mode (16.7% vs 12.4%). Conclusions We failed to increase survey response rates by adding an optional Internet response. Asthma diagnosis was higher in the Internet response group, suggesting nonresponse bias. Method comparison studies should be carried out before Internet studies are accepted in new populations or new subject matters.


Standard postal survey n=2105
Standard postal survey with optional internet response n=2108 Nonresponders n=1124 Responders n=984 Responders n=944 Nonresponders n=1161 Internet n=175 Postal n=769 CONSORT Checklist of items to include when reporting a randomized trial

INTRODUCTION Background
2 Scientific background and explanation of rationale. 4

3
Eligibility criteria for participants and the settings and locations where the data were collected.

5
Interventions 4 Precise details of the interventions intended for each group and how and when they were actually administered.

5-7
Objectives 5 Specific objectives and hypotheses. 4 Outcomes 6 Clearly defined primary and secondary outcome measures and, when applicable, any methods used to enhance the quality of measurements (e.g., multiple observations, training of assessors).

4
Sample size 7 How sample size was determined and, when applicable, explanation of any interim analyses and stopping rules.

5
Randomization --Sequence generation 8 Method used to generate the random allocation sequence, including details of any restrictions (e.g., blocking, stratification)

5
Randomization --Allocation concealment 9 Method used to implement the random allocation sequence (e.g., numbered containers or central telephone), clarifying whether the sequence was concealed until int erventions were assigned.

10
Who generated the allocation sequence, who enrolled participants, and who assigned participants to their groups.

5
Blinding (masking) 11 Whether or not participants, those administering the in terventions, and those assessing the outcomes were blinded to group assignment. When relevant, how the success of blinding was evaluated.

Statistical methods 12
Statistical methods used to compare groups for primary outcome(s); Methods for additional analyses, such as subgroup analyses and adjusted analyses.

13
Flow of participants through each stage (a diagram is strongly recommended). Specifically, for each group report the numbers of participants randomly assigned, receiving intended treatment, completing the study protocol, and analyzed for the primary outcome. Describe protocol deviations from study as planned, together with reasons.

Recruitment 14
Dates defining the periods of recruitment and follow-up. 5 Baseline data 15 Baseline demographic and clinical characteristics of each group. Table 1 Numbers analyzed 16 Number of participants (denominator) in each group included in each analysis and whether the analysis was by "intention-to-treat". State the results in absolute numbers when feasible (e.g., 10/20, not 50%).

17
For each primary and secondary outcome, a summary of results for each group, and the estimated effect size and its precision (e.g., 95% confidence interval). 7

Ancillary analyses 18
Address multiplicity by reporting any other analyses performed, including subgroup analyses and adjusted analyses, indicating those pre-specifi ed and those exploratory.

10-14
Adverse events 19 All important adverse events or side effects in each intervention group. n/a

20
Interpretation of the results, taking into account study hypotheses, 10-14 Interpretation sources of potential bias or imprecision and the dangers associated with multiplicity of analyses and outcomes. Generalizability 21 Generalizability (external validity) of the trial findings. 12 Overall evidence 22 General interpretation of the results in the context of current evidence.