Elsevier

Computers in Human Behavior

Volume 48, July 2015, Pages 297-309
Computers in Human Behavior

When adults without university education search the Internet for health information: The roles of Internet-specific epistemic beliefs and a source evaluation intervention

https://doi.org/10.1016/j.chb.2015.01.045Get rights and content

Highlights

  • General Internet beliefs and two types of justification beliefs examined.

  • Differential relationships with navigation behavior and post-search decisions found.

  • Positive effects of source evaluation intervention on navigation behavior.

  • Positive effects of source evaluation intervention on post-search decisions.

  • The intervention strengthened beliefs concerning justification by multiple sources.

Abstract

In the present research, N = 48 adults without university education were tasked to conduct two web searches on unfamiliar health-related issues. Three research objectives were examined: the predictive value of individuals’ Internet-specific epistemic beliefs for their navigation behavior and post-search decisions in Web Search Task 1; the effects of a short source evaluation intervention on individuals’ navigation behavior and post-search decisions in Web Search Task 2; and the effects of the source evaluation intervention on individuals’ Internet-specific epistemic beliefs. Results showed that beliefs concerning the justification by multiple sources were positively related to the time spent on reliable objective web pages and to the likelihood to make a post-search decision that was in accordance with the objective pages. Beliefs that the Internet is a reliable knowledge resource were also positively related to the time spent on objective pages as well as to greater certainty in the post-search decision. Second, in Web Search Task 2, the intervention group spent more time on objective web pages, was more likely to make a decision that was in accordance with these pages, and was more certain of their decision than the control group. Third, one week after the intervention, individuals possessed stronger beliefs concerning the justification by multiple sources than before. In conclusion, the present research indicates that both adaptive Internet-specific beliefs and a short source evaluation intervention have positive effects on non-university educated adults’ online health information seeking and that their Internet-specific epistemic beliefs can be fostered through such an intervention.

Introduction

The Internet has evolved into a major knowledge resource for science-related content, offering easy access to billions of web pages. Particularly in the context of individuals’ personal concerns, such as medicine and health care, consulting the Internet increasingly supplements face-to-face interaction with experts (Fox, 2006, Stadtler et al., 2009). According to survey data, more than 70% of Internet users in the United States and Europe look online for health information (Andreassen et al., 2007, Fox and Duggan, 2013), with web searches for a certain medical treatment or certain diets or nutritional supplements being among the most popular topics (Fox, 2006, Vermaas and Wijngaert, 2005). About half of individuals’ health web searches are on behalf of someone else’s health situation (Fox, 2006, Fox and Duggan, 2013). Online health information seekers typically start their search with a general search engine (e.g., Fox, 2006, Fox and Duggan, 2013). They often focus on the first few search results presented by the search engine (e.g. Eysenbach and Köhler, 2002, Zhang, 2012) and visit between two and five websites during an average search session (Fox, 2006, Huang et al., 2012, Vermaas and Wijngaert, 2005, Zhang et al., 2014).

Since anyone can publish virtually any information on the Internet, the purpose and quality of web pages, however, can vary widely. Whereas the primary purpose of objective websites (e.g. from official health institutions) is presumably the provision of unbiased information that is based on scientific evidence, subjective websites (e.g. online health forums) primarily serve to exchange personal opinions and experiences, and commercial websites (e.g. provided by health or pharmaceutical companies or supplements shops) use information to promote or sell products or services (cf. Kammerer and Gerjets, 2012, Tate, 2010). Accordingly, unscientific and (intentionally or unintentionally) misleading health websites are as common as those providing factual, high-quality information (Bates, Romina, Ahmed, & Hopson, 2006). Thus, health information seekers themselves are responsible for identifying reliable websites and to distinguishing them from less reliable ones (e.g., Goldman, Braasch, Wiley, Graesser, & Brodowinska, 2012). Existing empirical evidence whether they actually engage in such evaluation behaviors, however, is inconsistent. While some studies indicate that health information seekers indeed prefer sites run by reputable organizations and distrust websites that sell products or that are sponsored by pharmaceutical companies (Sillence et al., 2007, Zhang, 2012), others report that many individuals fail to distinguish trustworthy from less trustworthy websites during their health web searches (Bates et al., 2006, Eysenbach and Köhler, 2002, Ivanitskaya et al., 2006).

Previous research provides first indications that sophisticated Internet-specific epistemic beliefs – i.e., individuals’ personal beliefs about what knowledge and knowing is like on the Internet – positively predict critical online search and evaluation behaviors (Kammerer et al., 2013, Kammerer and Gerjets, 2012). Another line of research shows that interventions about how to critically evaluate and use different information sources on the Internet also have the potential to improve Internet users’ information-seeking and source-evaluation strategies (e.g., Argelagós and Pifarré, 2012, Braasch et al., 2013, Gerjets and Hellenthal-Schorr, 2008, Mason et al., 2014, Walraven et al., 2010, Wiley et al., 2009). Furthermore, such source evaluation (or web search) interventions, in turn, might also have the potential to change individuals’ epistemic beliefs, as indicated by a third line of research (Tsai, 2008).

The present paper aims at expanding all three lines of research by examining a target group that has received little attention so far, despite representing a considerable percentage of people who gather health information online (cf. Fox & Duggan, 2013): adults without a university background (hereafter also referred to as non-university educated adults). This population was selected since their formal education has probably not provided them with the skills to use the Internet for science-related purposes, and they are likely to experience particular difficulties in gathering scientifically sound health information on the Internet. In Germany, where the present research was conducted, for example, less than 30% of the 25–64 year-olds have attained tertiary education (OECD, 2013). Thus, non-university educated adults can be considered a highly important target group.

When using the Internet to seek information about an unfamiliar topic, individuals have to make their own decisions about which information to access, evaluate its veracity, and consider whose authority to accept (Hofer, 2004). Hofer, 2004, Tsai, 2004 were among the first to postulate that such evaluation processes are associated with one’s epistemic beliefs – in other words, an individual’s personal conceptions about the nature of knowledge and knowing (Hofer & Pintrich, 1997). More recently, Bråten, Britt, Strømsø, and Rouet (2011) proposed a model that specifies how and why different epistemic belief dimensions (i.e., justification for knowing, source of knowledge, certainty of knowledge, and simplicity of knowledge, cf. Hofer & Pintrich, 1997) might facilitate or impair the processing and evaluation of multiple texts (e.g. multiple websites). For instance, the model postulates that individuals who believe knowledge claims must be justified through reason, rules of inquiry, and the evaluation and integration of multiple sources increasingly try to evaluate the documents’ sources and to corroborate information between different sources as compared to individuals who do not feel this need. The model also postulates that individuals who believe that knowledge is transmitted by experts rather than constructed by the self when learning about rather unfamiliar topics increasingly distinguish more and less trustworthy sources and give prominence to trustworthy information in their overall representation of the issue (Bråten et al., 2011).

When investigating the role of epistemic beliefs during web search, Bråten, Strømsø, and Samuelstuen (2005) suggested focusing specifically on epistemic beliefs related to the Internet as a particular type of knowledge resource due to its specific characteristics, such as the absence of editorial gatekeeping and the great heterogeneity of information sources. To measure Internet-specific epistemic beliefs, they developed the Internet-Specific Epistemological Questionnaire (ISEQ) that was designed to assess two dimensions concerning what knowledge on the Internet is like (i.e., certainty of knowledge and simplicity of knowledge) and two dimensions concerning how one comes to know something by using the Internet as a knowledge resource (i.e., source of knowledge and justification for knowing). However, Bråten et al. (2005) empirically found only two dimensions. The first dimension (labeled general Internet epistemology) addressed the degree to which students’ believed that the Internet is a reliable knowledge resource that contains correct and detailed expert information about what they are studying. The second dimension (labeled justification for knowing) addressed students’ beliefs concerning the need for critical evaluation of study-related knowledge claims encountered on the Internet through the use of multiple information sources, reasoning, and prior knowledge activation.

Bråten et al. (2005) correlated university students’ Internet-specific epistemic beliefs with self-reports on their search and evaluation behavior when searching for study-related issues. Results indicated that students with strong beliefs that the Internet is a reliable knowledge resource that contains correct and detailed information were reportedly more competent and experienced less problems when conducting web searches, evaluating information sources, and using the retrieved information in their course-work than were students who had doubts about this issue. In a similar study, Strømsø and Bråten (2010) found a positive relationship between beliefs that knowledge claims encountered on the Internet need to be checked against other information sources, reason, and prior knowledge on the one hand, and the reported use of self-regulatory strategies (such as planning, monitoring, and regulating cognition and performance) during web search on the other. Furthermore, Lee, Chiu, Liang, and Tsai (2014) showed that such beliefs were positively related to high-school students’ reported use of search engines and expert websites to solve academic problems. Beliefs that the Internet provides a wealth of detailed information were also positively related to such reported web search strategies (Lee et al., 2014).

Whereas the research outlined above correlated Internet-specific epistemic beliefs with self-report data, Kammerer and Gerjets, 2012, Kammerer et al., 2013 examined the relationship between Internet-specific epistemic beliefs and more direct measures of search and evaluation processes (i.e., by means of eye-tracking data, log files, and verbal protocols) while using a search engine to learn about a conflicting medical topic. Kammerer and Gerjets (2012) found that the stronger university freshmen’s epistemic beliefs that the Internet is a reliable knowledge resource, the more objective (i.e., scholarly, factual) websites they accessed during their web search. Kammerer et al. (2013) found that university students with stronger epistemic beliefs that the Internet is a reliable knowledge resource were more certain about their post-search decisions, that is, about which of two therapies to recommend to a fictitious friend. However, the results also indicated that students with such beliefs verbally reflected less on the nature or credibility of different kinds of information sources (e.g., forum websites, websites from official institutions, or websites from pharmaceutical companies) encountered in the search engine results pages (SERPs). While this could be an indication for a lower degree of source evaluation, another explanation for this finding might be that they mainly focused on reliable information sources while ignoring the other kinds of sources.

While prior research on Internet-specific epistemic beliefs has not yet demonstrated a relationship between beliefs concerning justification for knowing and source evaluation as proposed by Bråten et al.’s (2011) theoretical model, Strømsø, Bråten, and Britt (2011) found such a relationship for topic-specific beliefs concerning justification for knowing. The stronger university students’ beliefs about the necessity to critically evaluate knowledge claims about climate change, the more they relied on a popular science article written by a scientist (i.e., a reliable source) when informing themselves about conflicting issues of climate change. One potential reason for the lack of empirical evidence associating (Internet-specific) beliefs concerning justification for knowing and source evaluation processes might be that measures of justification for knowing beliefs such as in the ISEQ (Bråten et al., 2005) often include items that address the justification through external sources (i.e., justification by cross-checking multiple sources or by relying on external authorities) as well as items that address the justification through internal sources (i.e., justification through the use of prior knowledge or own reasoning). However, only beliefs regarding the former type of justification might be positively associated with source evaluation processes (cf. Ferguson et al., 2012, Greene et al., 2008). In line with this reasoning, Bråten, Ferguson, Strømsø, and Anmarkrud (2014) could show that, in a multiple-documents comprehension task, utterances concerning justification by multiple sources (as measured by verbal protocols) were related to an increased number of source references included in the essays participants wrote after reading; whereas utterances concerning justification by authority and personal justification (i.e., justification through own reasoning or the use of prior knowledge) were not.

Therefore, in the present research we further differentiated the items of the Internet-specific justification for knowing dimension into beliefs concerning the justification by multiple sources and beliefs concerning personal justification. Moreover, we aimed at expanding previous research, which has mostly focused on university students, to a different target group, namely adults without a university education. Specifically, we aimed at investigating the relationship between non-university educated adults’ Internet-specific epistemic beliefs and their navigation behavior (i.e., the time spent on different kinds of websites of varying reliability) and post-search decisions when seeking information on the Internet about an unfamiliar health-related issue.

Internet users’ information-seeking and source-evaluation strategies can also be fostered by interventions about how to critically evaluate and use different information sources on the Internet. In recent years several researchers have developed and tested such interventions for students in primary school (e.g. Kuiper, Volman, & Terwel, 2008), secondary school (e.g., Argelagós and Pifarré, 2012, Braasch et al., 2013, Gerjets and Hellenthal-Schorr, 2008, Mason et al., 2014, Walraven et al., 2010), or university (e.g., Graesser et al., 2007, Wiley et al., 2009). These interventions range from single, one-hour instructional units (Braasch et al., 2013, Graesser et al., 2007, Wiley et al., 2009) to a series of instructional units over several consecutive days (Gerjets and Hellenthal-Schorr, 2008, Kuiper et al., 2008), to comprehensive training programs embedded into regular classroom lessons lasting several months (Argelagós and Pifarré, 2012, Walraven et al., 2010). Most of the instructional methods turned out to be at least partly successful in improving students’ search and evaluation skills. One of the most successful interventions designed to critically evaluate science-related or health-related information has been the one-hour instructional unit SEEK (Source, Evidence, Explanation, and Knowledge) developed by Wiley et al. (2009). The instructional unit consists of three parts: (1) declarative information about criteria for evaluating online information (e.g., the type of website, potential motives and expertise of the information providers, and the scientific evidence of the content), (2) the application of these criteria to the evaluation of a set of example websites (i.e., websites from official institutions or specialist media, commercial websites, and a personal homepage) addressing a health-related topic, and (3) an exercise to rank the websites according to their reliability with subsequent feedback of an expert ranking. Wiley et al. (2009) could show that in a transfer task in which students were tasked to read six websites about a health-related topic, those who had received the SEEK intervention were better than controls at differentiating between reliable and unreliable information sources (as measured by ranking the websites according to their reliability after exploration). However, during reading no significant effect of the SEEK intervention on the time spent on reliable and less reliable websites was found. In contrast, Mason et al. (2014), who recently examined the beneficial effects of the SEEK intervention in secondary school students, found that the SEEK students in a transfer web search task spent more time on the most reliable websites (i.e., websites from official governmental or scientific institutions) than controls; they did not spend less time on the least reliable websites, i.e., forum and company websites, though.

In the present study, the effects of a similar intervention for non-university educated adults was tested with regard to its beneficial effects on their navigation behavior (i.e., the time spent on reliable over less reliable websites) and post-search decisions when seeking information about an unfamiliar health-related issue on the Internet. Whereas Wiley et al., 2009, Mason et al., 2014 in their analyses only differentiated between reliable web pages (i.e., official institutions and specialist media) and unreliable web pages (i.e., forum pages and commercial pages), in the present study with regard to unreliable web pages we further differentiated between subjective web pages (i.e., forum pages) and commercial web pages (cf. Kammerer & Gerjets, 2012).

Another question addressed in the present research is whether a source evaluation intervention in turn could change individuals’ beliefs about what nature and knowledge is like on the Internet. Bendixen and Rule (2004) proposed three interrelated components to be prerequisite for changing a person’s epistemic beliefs. These components are epistemic doubt (i.e., questioning the validity of one’s current beliefs), epistemic volition (i.e., the willingness to change one’s beliefs), and resolution strategies (i.e., reflecting on how or in which direction to change one’s beliefs). Ferguson et al. (2012) found empirical support for the existence of epistemic doubt and resolution strategies in university students’ verbal protocols while reading multiple conflicting documents on a scientific issue. Epistemic volition, in contrast, was hardly observed in students’ verbal protocols.

It is conceivable that a source evaluation intervention can trigger epistemic doubt about existing Internet-specific beliefs as well as provide resolution strategies to change them. First indications for this assumption are given by Tsai (2008), who examined high-school students’ beliefs concerning the evaluation of online information and their epistemic beliefs about science before and after having participated in a one-month science course. In this science course students were instructed to search the Internet to resolve controversial issues and to participate in some online discussions concerning those issues. Results indicated that after the intervention students deemed it more important to compare information from multiple sources in order to judge the accuracy of online information and viewed scientific knowledge as more theory-laden and more tentative and changing than before. Other studies have shown similar effects elicited by short-term or long-term interventions that stressed the uncertain and complex nature of knowledge on students’ beliefs about science or medicine (e.g., Conley et al., 2004, Kienhues et al., 2011).

The goals of the present research were threefold: First, we aimed at investigating the predictive value of Internet-specific epistemic beliefs for differences in navigation behavior (i.e., the time spent on different kinds of websites of varying reliability) and post-search decisions among non-university educated adults who searched the Internet for an unfamiliar health-related issue (Research Question 1). Second, we wanted to investigate the effects of a short source evaluation intervention on individuals’ navigation behavior and post-search decisions in a second web search on another health-related topic (Research Question 2). Third, we wanted to examine whether a short source evaluation intervention could change individuals’ Internet-specific epistemic beliefs (Research Question 3). In the following, we will outline the hypotheses regarding the three research questions.

With respect to Internet-specific epistemic beliefs, as in Bråten et al., 2005, Kammerer et al., 2013, we distinguished between general Internet beliefs, i.e., beliefs concerning the Internet as a knowledge resource, and beliefs concerning the justification for knowing. General Internet beliefs were defined as the degree to which individuals believe that the Internet is a reliable knowledge resource that contains correct and detailed expert information (Bråten et al., 2005). Beliefs concerning the justification for knowing were defined as the degree to which individuals believe that knowledge claims on the Internet require justification through reasoning, critical evaluation, and cross-checking (Bråten et al., 2005). With regard to the latter, however, in line with Bråten et al., 2014, Ferguson et al., 2012 we further differentiated between beliefs concerning justification by multiple sources (i.e., by cross-checking and corroborating across multiple web pages) and beliefs concerning personal justification (i.e., by reasoning and prior knowledge).

With regard to beliefs concerning justification by multiple sources, based on theoretical assumptions (Bråten et al., 2011) and preliminary empirical evidence outlined above (Strømsø et al., 2011), we hypothesized that stronger beliefs that Internet-based knowledge claims need to be critically evaluated by cross-checking multiple web pages would relate to an increased time spent on reliable, objective web pages hosted by official institutions and domain experts. These should be better suited for cross-checking information than other kinds of web pages and their knowledge claims should be better supported by evidence. In contrast, we assumed a negative relationship with the time spent on subjective web pages where laypersons exchange their personal opinions and experiences and on commercial web pages that promote vested interests (Hypothesis 1a). Second, we hypothesized that stronger beliefs concerning the justification by multiple sources would also relate to a post-search decision that is in accordance with the objective web pages (Hypothesis 1b).

With regard to beliefs concerning personal justification, in contrast, stronger beliefs that Internet-based knowledge claims need to be evaluated by reasoning and prior knowledge should not be or should even negatively be related to a critical and reflective navigation behavior or the quality of the post-search decisions (Hypothesis 1c), particularly when working on a rather unfamiliar topic as in the present study (cf. Bråten et al., 2014).

With regard to general Internet beliefs, based on findings by Kammerer and Gerjets (2012) we expected that stronger beliefs that the Internet is a reliable knowledge resource would relate to an increased time spent on objective web pages. In contrast, we assumed a negative relationship with the time spent on subjective web pages and commercial web pages (Hypothesis 1d). Finally, based on previous findings by Kammerer et al. (2013) we predicted that stronger beliefs that the Internet is a reliable knowledge resource would also relate to more certainty regarding the post-search decision (Hypothesis 1e).

We hypothesized that relative to a control condition participants who underwent a short source evaluation intervention about how to critically evaluate online information (cf. Mason et al., 2014, Wiley et al., 2009) in a transfer web search task would spent more time on objective web pages, and less time on subjective and commercial web pages (Hypothesis 2a). We further hypothesized that participants in the intervention group would consequently make a post-search decision that is in accordance with the objective web pages (Hypothesis 2b), and would also be more certain regarding their post-search decision than controls (Hypothesis 2c).

Based on preliminary empirical evidence by Tsai (2008), we hypothesized that a short source evaluation intervention would elicit changes in individuals’ justification beliefs, particularly in their beliefs concerning justification by multiple sources. Individuals’ beliefs that knowledge claims on the Internet require justification by critical evaluation and cross-checking of multiple information sources should become stronger after the intervention (Hypothesis 3a). As a source evaluation intervention should increase the general awareness of the importance of critically evaluating information on the Internet, also beliefs that Internet-based knowledge claims need to be evaluated by reasoning and prior knowledge were expected to become stronger (Hypothesis 3b). With respect to general Internet beliefs we expected no change, because our intervention was not supposed to convey the view that the Internet is a good (or a bad) knowledge resource but rather that it contains different kinds of information sources and how to identify them (Hypothesis 3c).

Section snippets

Participants

Participants were recruited via a newspaper advertisement and postings on noticeboards in public places in a mid-size town in southwestern Germany. The sampling criteria were as follows: (1) Only adult individuals (≥25 years) without a university or technical college education were allowed to participate. (2) At least basic computer and Internet skills were required for participation. Appointments were made with all participants who contacted us (via email or telephone) during the study period

Research Question 1: Relationships between Internet-specific epistemic beliefs and navigation behavior and post-search decisions

In the first research question, we aimed at investigating the predictive value of participants’ Internet-specific epistemic beliefs for navigation behavior and post-search decisions in Web Search Task 1. For one participant, the data regarding the post-search decision were missing. Descriptive statistics for the epistemic beliefs variables and the dependent variables, together with zero-order correlations, are reported in Table 2. Overall, participants on average accessed 4.81 of the 18 web

Discussion

As on the Internet anyone can publish almost anything, the quality of online information varies considerably. Hence, when using the Internet to find information about science-related topics, for example in the health domain, assessing the veracity of information found online and considering whose authority to accept are important evaluation processes (Hofer, 2004). Previous research has provided first indications that sophisticated Internet-specific beliefs are positively related to such

Acknowledgement

We thank André Klemke for his assistance in the programming of the web-based source evaluation intervention.

References (50)

  • E. Kuiper et al.

    Integrating critical Web skills and content knowledge: Development and evaluation of a 5th grade educational program

    Computers in Human Behavior

    (2008)
  • W.-C. Lee et al.

    Exploring the structural relationships between high school students’ Internet-specific epistemic beliefs and their utilization of online academic help seeking

    Computers in Human Behavior

    (2014)
  • L. Mason et al.

    Epistemic evaluation and comprehension of web-source information on controversial science-related topics: Effects of a short-term instructional intervention

    Computers & Education

    (2014)
  • L. Scharrer et al.

    The seduction of easiness: How science depictions influence laypeople’s reliance on their own evaluation of scientific information

    Learning and Instruction

    (2012)
  • E. Sillence et al.

    How do patients evaluate and make use of online health information?

    Social Science & Medicine

    (2007)
  • M. Stadtler et al.

    Effects of the metacognitive computer-tool met.a.ware on the web search of laypersons

    Computers in Human Behavior

    (2008)
  • A. Walraven et al.

    Fostering transfer of websearchers’ evaluation skills: A field test of two transfer theories

    Computers in Human Behavior

    (2010)
  • Andreassen, H., Bujnowska-Fedak, M. M., Chronaki, C. E., Dumitru, R. C., Pudule, I., Santana, S., Voss, H., & Wynn, R....
  • B.R. Bates et al.

    The effect of source credibility on consumers’ perceptions of the quality of health information on the Internet

    Medical Informatics and the Internet in Medicine

    (2006)
  • L.D. Bendixen et al.

    An integrative approach to personal epistemology: A guiding model

    Educational Psychologist

    (2004)
  • I. Bråten et al.

    The role of epistemic beliefs in the comprehension of multiple expository texts: Towards an integrated model

    Educational Psychologist

    (2011)
  • I. Bråten et al.

    Justification beliefs and multiple-documents comprehension

    European Journal of Psychology of Education

    (2014)
  • I. Bråten et al.

    The relationship between Internet-specific epistemological beliefs and learning within Internet technologies

    Journal of Educational Computing Research

    (2005)
  • M.A. Britt et al.

    Learning with multiple documents: Component skills and their acquisition

  • G. Eysenbach et al.

    How do consumers search for and appraise health information on the World Wide Web? Qualitative studies using focus groups, usability tests, and in-depth interviews

    British Medical Journal

    (2002)
  • Cited by (0)

    View full text