A cross sectional survey of the UK public to understand use of online ratings and reviews of health services Patient Education and Counseling

Objectives: To identify the self-reported behaviour of the public in reading and writing online feedback in relation to health services. Methods: A face-to-face cross-sectional survey of a representative sample of the UK population. Descriptive and logistic regression analyses were undertaken to describe and explore the use of online feedback. Results: 2036 participants were surveyed, and of 1824 Internet users, 42% (n = 760) had read online health care feedback and 8% (n = 147) had provided this feedback in the last year. People more likely to read feedback were: younger, female, with higher income, experiencing a health condition, urban dwelling, and more frequent internet users. For providing feedback, the only signi ﬁ cant association was more frequent internet use. The most frequent reasons for reading feedback were: ﬁ nding out about a drug, treatment or test; and informing a choice of treatment or provider. For writing feedback they were to: inform other patients; praise a service; or improve standards of services. 94% had never been asked to leave online feedback. Conclusion: Many people read online feedback from others, and some write feedback, although few are encouraged to do so. Practice implications: This emerging phenomenon can support patient choice and quality improvement, but needs to be better harnessed.


Introduction
Online feedback from patients reporting their experiences of health services, health professionals, medical tests and treatments is an increasing phenomenon [1,2]. This is in line with online customer behaviour in many other sectors such as retail and travel, where an explosion in such feedback has been held up as an example of disruptive digital innovation, bringing transformative change to those sectors including service improvement [3]. A recent UK report on online consumer behaviour showed that three in four internet users read reviews before deciding to buy a product or service (not specifically health) and two in five write online reviews after the purchase [4]. In some ways, the health sector has been slow to harness this phenomenon, but there is much current interest in understanding the opportunities and challenges of online comments, reviews and ratings from people using health services. Also, the potential benefit of using these to measure quality, to inform patient choice, and to drive change, while acknowledging there are issues of digital inclusion and representativeness [5][6][7][8][9].
Current work in this area has shown that the use of online feedback by patients has, to date, been relatively limited [1,2,10,11]. Previous surveys found that those who are more likely to use online feedback of health services include people who: are younger [10,11], live in (sub)urban areas and have higher levels of education. (10) The last UK-survey was published in 2012, and conducted among a small non-representative sample of 200 people living in one borough in London showing that just 29 people (15%) were aware of doctor rating websites and only 6 people having used them [10]. In a US survey conducted in 2012, 65% of 2137 participants were aware of online patient feedback websites and 23% had used them [2]. Of 854 respondents in another US survey in 2013, 16% said they had previously visited a patient feedback website. (1) Whilst there are some caveats in the non-comparability of studies that have been conducted in different settings, using different questionnaires, it seems that the number of people using online feedback is rising rapidly from a very low baseline over time.
Currently, there is no up-to-date data on use of online feedback of UK health services, despite huge policy interest in this area in the UK and elsewhere [12]. Without such data, meaningful policy decisions, and practice change are not possible. We therefore undertook the first nationally representative UK survey on providing and using online patient feedback among the general population. Our aims were to identify the frequency of use, user characteristics, and self-reported behaviour of members of the public in reading and writing online feedback on health services, health professionals, and medical treatments or tests.

Methods
This study is reported in line with the STROBE statement [13].

Study design
A cross-sectional face-to-face questionnaire-based household survey was conducted with members of the UK public about their use of online ratings and reviews (see questionnaire in Supplementary file Appendix A). A market research agency, ICM Unlimited, conducted the fieldwork. ICM had previously conducted the Oxford Internet Survey which uses similarly methodology, and the authors collaborated with the Oxford Internet Institute in designing the survey and choosing the provider [14]. Similar to the Oxford Internet Surveys, a two-stage design was used for sampling. Firstly, a random sample of output areas stratified by region was selected. Secondly, within each selected output area a random selection of addresses was selected. ICM recruited and interviewed participants by sending interviewers to the homes of selected people in February 2017. The study received institutional ethics approval from the University of Oxford Central University Research Ethics Committee (CUREC, reference SSH_OII_C1A_074).

Participants and setting
We included adult members of the public who were willing and able to give informed consent for participation in the study, lived in the UK, able to speak and read English, and were aged at least 16 years. Participants were given information about the study and that they were free to withdraw from the study at any time for any reason, and with no obligation to give the reason for withdrawal. To select participants, a random location sampling system was used where we randomly selected Outputs Areas as the geographical sampling unit. Each output area consists of around 150 households and all properties are available to the interviewer to achieve the target number of interviews (usually 4-5 per point). Demographics quotas were applied to ensure the profile of achieved interviews in each sample point reflects the known population of the area. For an explanation of this method in a similar survey see Oxford Internet Surveys [14].

Variables
We collected data on participant's characteristics, including age, gender, ethnicity, annual household income, education level, living in an urban or rural area, health status and Internet use (see Supplementary file Appendix B). There were also 20 questions relating to online feedback (see questionnaire in Supplementary file Appendix A). These questions were principally designed based on items from previous surveys [10,11] and on policy documents and reports by online feedback organisations [15] and informed our concurrent survey of healthcare professionals (not yet published) in which we developed and piloted questions about professional use of, and attitudes towards online feedback. We piloted the questionnaire with a patient and public reference group and tested it using two rounds of cognitive interviews (also with the public). Questions were asked about whether, where and why participants read or wrote online ratings or reviews of health services, individuals, drugs, treatments or tests.

Data sources and study size
All data was obtained through face-to-face interviews with participants. Surveys were completed on a tablet and transferred to the study team in an excel spreadsheet. The names and any other identifying details of participants were not collected in any of the surveys. Direct access to study data was granted to authorised representatives from the University of Oxford and University of Warwick for monitoring and audit of the study to ensure compliance with regulations.
The survey was a fully representative sample of the population of Great Britain aged 16+. A sample size of 2000 with a margin of error percentage of two was chosen to maximise accuracy within reasonable resource constraints [14]. Data was weighted to the socio-demographic profile (Census data that included gender, age, socio-economic grade, region, and ACORN [A Classification Of Residential Neighbourhoods] group) of the target population (UK citizens aged at least 16 years).

Quantitative variables and statistical methods
All analyses were conducted using the statistical software package SPSS version 22 [16]. Descriptive analyses of participants' characteristics and the prevalence of providing and of reading online feedback were conducted. Non-internet users were excluded from these analyses as by default they would not be providing or reading or writing online content. We coded the outcome as binary: use any type of feedback vs none. Logistic regression was used to explain the use of online feedback (as the dependent variable), with the following independent variables that were considered to be potentially relevant: age, gender, education, income, living in rural or urban area, and frequency of internet use. These socio-demographic and Internet use variables have been shown to influence the uptake of a wide number of online activities, including health [17]. Ethnicity was not included in the logistic regression analyses because of the small number of participants in the ethnicity subgroups. In the results we present the model fit (%), Chi-square, P and R 2 (Nagelkerke) values. We used Binary Logistic Regression in SPSS and included all variables which were found to be statistically significant in univariate analysis in the model. Missing data were not imputed.

Patient involvement
This survey is part of the wider programme of work examining the phenomenon of online patient feedback (the INQUIRE study) [18]. The original design of this programme of work (including the current study) was informed by a workshop with patient organisations. Subsequently the further refinement of our research design was informed by our patient co-investigator on the INQUIRE project, as well as our public, patient and carer reference group. Both our patient co-investigator and the members of our reference group were involved in commenting on the survey questions and we presented them with a summary of our findings.

Participants and descriptive data
Our total sample included 2036 participants of whom 1824 used the Internet over the past year and were included in further analyses; their characteristics are shown in Table 1, as well as the characteristics of those who read and provided feedback.
Supplementary file Appendix C Table A1 and A2 show characteristics of the 10% of our sample who were non-users of the Internet (n = 212). Of 1824 Internet users, 42% (n = 760) had read feedback about health services, or about health professionals, or about medical tests or treatments in the past year, while 8% (n = 147) had written such feedback in the same period.

Education & household income
The highest proportion of readers and writers were also among those with degree level qualification and above (Table 1) and these people were significantly more likely to read online feedback than those with other qualifications (Table 2). People in the highest income bracket of >£100,000 were significantly more likely (OR = 1.784, 95% CI = 1.088-2.924, P = .022) to read online feedback than those with the lowest income (up to £24,999).

Health status
Of people with a long-term condition, health problem, or disability, 49% (n = 183) read and 10% (n = 39) wrote online feedback (Table 1) and they were significantly more likely (OR = 1.463, 95% CI = 1.164 to 1.839, P = .001) to read than those without such a health condition (Table 2).

Area and internet use
Of people living in urban areas, 48% (n = 240) read and 10% (n = 52) wrote online feedback (Table 1) and they were significantly more likely (OR = 1.697, 95% CI = 1.241 to 2.320, P = .001) to read compared to those living in rural areas. People accessing the Internet several times a day were significantly more likely to read (OR = 2.680, 95% CI = 1.808 to 3.974, P = .000) and write (OR = 3.206, 95% CI = 1.216-8.449, P = .018) compared to those who went online fewer than once a day ( Table 2).

Multi-variate regression
The initial multi-variate regression model for 'reading feedback' showed a model fit of 55%. When the following significant variables were included: age, gender, education, household  Fig. 1 and Table A5). Far fewer participants had written reviews, 6% (n = 105) about health care organisations, 4% (n = 69) about health professionals, and 4% (n = 69) about drugs, treatments or tests (Supplementary file Appendix D Fig. 2). Most participants who read or wrote feedback had done this once or every few months/monthly over the past year (Table 4 and Supplementary file Appendix Table A6).
Of the 760 participants who read feedback about a health care organisation, a health professional or a treatment or test, 42% (n = 320) read feedback about one of these, 29% (223) read feedback about two, and 28.6% (217) about three. Supplementary file Appendix D Fig. 2 and Table A7 shows that of the 147 participants who wrote feedback a health care organisation, a health professional or a treatment or test, 53% (n = 79) wrote feedback about one of them, 26% (n = 39) about two, and 20% (n = 29) about three.
Comparing readers and non-readers versus writers and nonwriters, we found that 7% of the whole sample of internet users (n = 128/1824) had both read and written a review. Of the 760 participants who read feedback, 83% (n = 633) had not written a review and of the 147 participants who wrote feedback had 13% reported not reading feedback. 57% of the whole sample of internet users (n = 1044/1824) had not read or written feedback over the past year.

Websites where online feedback of health services was read and written
The most frequently used formal review website for both reading and writing feedback was NHS Choices(19) (used by 49% of 'readers' and 35% of 'writers') followed by WebMD (15% and 5% respectively) and CareOpinion (formerly PatientOpinion) (6% and 9% respectively)(Supplementary file Appendix Table A8). The most frequently used social media outlets for reading and writing online feedback were Google reviews (31% and 14% respectively) and Facebook (25% and 23%, respectively). Table 5 shows the most frequently reasons among 760 'readers' for reading reviews: finding out about a drug, treatment or test (41%); choosing where to have treatment (19%); or choosing a healthcare professional (17%). The most common reasons for providing reviews were to: inform other patients (39%); praise a service (36%); or improve standards of NHS services (16%). Of the  total sample, only 112 (6%) of participants had been asked to write a review. Of those people who were asked to write a review, only 28 (25%) had written a review. The eight people who said they had often been asked to write a review had not done so. Writing a review to provide praise for a service was a far more common motivation for 147 'writers' (36%) than to complain about a service (6%), treatment (5%) or professional (4%).

Discussion
This paper reports the first representative UK population data on the use of online feedback about health care. As such it provides key baseline prevalence data for future engagement with online feedback by patients. About two in five people had read online feedback in the past year, while one in twelve had provided such feedback. The majority of the population had not used online feedback of health services over the past year. The least represented users of online feedback of health services were people: aged 65 and over, without formal qualifications, at lower social grades, accessing the Internet fewer than once a day, and living in rural areas.
The findings of this survey are representative of the general population of Internet users in the UK. Whilst the average member of the public visits a GP five times a year, not everyone in the general population uses health services in a one-year period, so it is not surprising that reading feedback is not universal. On average 42% of Internet users in our survey read online feedback on some aspect of health care in our study. This is higher than shown in  /every couple of days  14  3  1  1  9  3  3  5  11  2  1  2  Weekly/fortnightly  44  9  9  9  42  13  6  9  49  9  6  9  Monthly/every few months  230  45  29  27  149  45  22  32  335  58  30  43  Once last year  220  43  66  63  131  40  37  54  183  32  32  46 a Numbers do not add up because the data was weighted. a % do not add up to 100% because participants could have more than one reason to read a rating or review on more than one website. ** Numbers do not add up because the data was weighted.
previous studies described above [1,2]. For example the previous work in the UK, from 2012, had shown very low awareness (15%) and usage (3%) of doctor rating sites in a convenience sample survey of 200 people in London [10]. Overall, people are still far less likely to read and write reviews of health services compared to commercial services [4]. Our findings on age and gender concur with a study conducted in Germany which examined the characteristics of patients using a national public reporting instrument to leave feedback on their healthcare experiences. This study found that 60% of 107,148 patients rating physicians were female and 51% were aged 30-50 years [20]. Only 14% of writers in our study left feedback to complain, which is in line with another survey in the US where 9% of 854 patients provided an unfavourable review [1]. Findings from the German study that found that only 3% of 127,192 ratings of 53,585 physicians were rated with an insufficient and 5% with a deficient score in their overall performance [20], and in a UK study the NHS services received three times more positive (total: 223,439) than negative (total: 73,363) reviews [21]. About 1 in 10 people did not use the Internet in our study, which is in line with UK Ofcom data [4] and shows an increase in use of the Internet compared to the Oxford Internet Survey conducted in the UK in 2013 where about 2 in 10 people were non-Internet users [11]. In line with previous research, people with a lower level of education, income or social grade, older age, or living in rural areas were less likely to be regular Internet users [17]. We also found that these variables were associated with lower use of reading online feedback. It may be that those in urban areas use feedback more as they have more genuine choice in terms of health care provider in their locality.
We have conducted a parallel survey among healthcare professionals (currently under review), assessing their attitudes, behaviours and experiences in relation to online patient feedback. In that work we found that a low proportion of healthcare professionals were encouraging patients to leave feedback and that they viewed online feedback to be unrepresentative of the patient population. The findings in this present study of the public, support this view that the people who currently leave feedback are a minority and not representative of the general population. This is the largest and representative general population survey conducted across the UK. It provides an update in a fast-moving and under-researched area where no data in the UK had been published since 2012. While we believe our findings have relevance for other settings, they will (as with previous work undertaken in Germany and the US) be influenced by the nature of the health system in which this work was carried out. Even though it is a nationalised system, the NHS does encourage some level of choice and competition between its services, and initiatives such as ratings and reviews are seen as part of this, with the aim of driving quality improvement. However, the degree of choice exercised by patients in the NHS is less than found in US-style health care, or in social insurance-based systems. There may therefore be less motivation to use reviews and ratings (and less overall use), and perhaps UK consumers, in comparison with those elsewhere, may be relatively more motivated by 'voice' (articulating issues of concern), than by supporting 'choice' [22].
This survey method relies on participant self-report to a faceto-face questionnaire, as such it may be influenced by recall bias and presentation bias. Cognitive interviews with members of the public were conducted to optimise the design of questions with the aim of minimising other response bias caused by question wording or item order. As a result, we had a relatively low number of 'other' and 'don't know' responses. Data from cross-sectional surveys can only be used to investigate associations between variables, not causation, and the nature of quantitative findings mean that although we can identify prevalence of use, in this study we cannot provide any deeper, qualitative understanding of the phenomenon of using online feedback of health services.

Conclusions
We have provided the first UK-wide representative data on the use of online feedback which shows that while many people (more than 40% of internet users) read online feedback about health care, fewer currently provide it, and very few have been asked to provide it. Encouragingly, users are motivated to become more informed, to make choices, to provide praise, and to improve standards of care. Further work is needed to explore user behaviour in-depth, and to understand the relationship between the use of online feedback (both reading and writing) and the subsequent behaviour of patients, practitioners and health services.

Practice implications
Further work is needed to explore in depth the attitudes and motivations of the users of online feedback, to better understand why they choose to either read or write feedback, the context in which they do this (including which device they use and when and where), and how this might influence their subsequent health or consulting behaviour, alongside other online sources of information such as hospital report cards [23]. Longitudinal studies would also be valuable to track whether use of feedback (for example reading reviews) has a later influence on public attitudes and behaviour. In further work it would also be useful to identify which feedback platforms are used and why: some platforms are open, anonymous and can be used by all, others are linked to specific episodes of care and feedback is invited. It would also be valuable to explore the role of peer-to-peer comments made in social media [24]. Policymakers should note that this is now a widespread phenomenon and health services seeking to be more patientcentred and responsive to their users need to understand how best to harness feedback, and what the opportunities are to encourage it and engage with it, and investigate how it can be used for service improvement. Policymakers should incorporate online feedback with other sources of feedback to take appropriate action to improve quality of care and to monitor their strategies [25].
People living in suburban or rural areas, older people, and with lower levels of education were less likely to read or leave feedback and it is important that provision is made for these groups as the health service becomes more digital in its interactions with the public and the issue of digital inclusion (and exclusion) can be addressed. Further work could look at how these non-Internet users provide feedback to the health service as public services increasingly become 'digital by default'. Practitioners should note that while, in line with concerns expressed by some, those who currently provide feedback are not wholly representative of the general population, many more people read and potentially act on this feedback. They should also note that providing praise was a much more frequent motivation for patients than complaining. Our survey showed that very few people are asked to comment by their health professional and of those who were asked only few wrote a review as a result. Perhaps one approach to making feedback more representative in future would be for practitioners to find a more effective way to actively solicit feedback from all their patients, and it would be useful to understand what types of feedback are being encouraged by health services currently, and how this is being done (for example, using the NHS friends and family test).

Contributors
JP and HA conceptualized and designed the study, critically reviewed the results of analyses, and reviewed and revised the manuscript. MV designed the study, carried out the analyses, and drafted the initial manuscript. All authors approved the final manuscript as submitted. JP is the guarantor.