How primary care physicians’ attitudes toward risk and uncertainty affect their use of electronic information resources

Objective: The research sought to determine if primary care physicians’ attitudes toward risk taking uncertainty affected how they sought information and used electronic information resources when answering simulated clinical questions. ● Individual characteristics of clinicians affect how they search for information. Trainers should be aware of these differences and adjust their training methods to accommodate them. Differences seemed to center on ease of use, short cuts, and degree of control over searching and retrieval assessment. ● Clinician preferences should not be the only factor considered in providing information resources in clinical settings. Collection building should factor in content and strength of

• Physicians with varying attitudes toward risk taking and levels of stress from uncertainty searched differently, although with similar search outcomes. • Physicians who were risk avoiding or reported stress from uncertainty spent more time processing both the search and retrieval, while their counterparts spent more time considering additional searching options.

Implications
• Individual characteristics of clinicians affect how they search for information. Trainers should be aware of these differences and adjust their training methods to accommodate them. Differences seemed to center on ease of use, short cuts, and degree of control over searching and retrieval assessment. • Clinician preferences should not be the only factor considered in providing information resources in clinical settings. Collection building should factor in content and strength of evidence.

INTRODUCTION
Physicians typically seek external information several times per clinic day [1,2]. Whether clinicians seek answers from peers [3] or from print or electronic information resources [4][5][6], the answers they find are not always correct. The failure of information resources to help clinicians locate correct answers is troubling and merits further research. The inability of a resource to provide or lead a clinician to a correct answer could be the fault of the resource itself. However, almost all commonly used information resources are carefully designed to ensure content accuracy and ease of use. Characteristics of the clinicians themselves may contribute to their effectiveness with a resource and the correctness of the answers they receive.
Individual characteristics of users play a substantial role in effective use of computer-based resources and systems [7]. Some of the characteristics that affect use of information systems are addressed simply by good design (e.g., allowing movement to a larger font size for those with vision problems), and some are addressed by personalization options (e.g., change to a visual rather than text-based presentation, differing ordering of retrievals, or presentation of advanced rather than simple searching screens) [8]. With advances in computing power and sophistication of systems, it is increasingly possible to adapt resources to address user preferences and needs [9]. Successful adaptation of information resources takes into account user char-acteristics such as how individuals visualize information or innate spatial abilities that affect information retrieval or handling. In short, understanding the individual characteristics that affect use of computer resources can help improve an existing resource, produce effective updates, or develop new systems.
Research by Hersh et al. [10] and Bennett et al. [11] indicated a number of characteristics that affected search processes and performance. These characteristics include gender, age, standardized test scores, searching experience, and training background. More work needs to be done to fully understand the effect of other individual differences on the efficiencies of electronic information resources, especially identifying which factors affect resource use and outcomes.
Physicians, similar to members of the general population, vary in their attitudes toward risk taking and uncertainty. Risk and uncertainty are pervasive in health care and complicate decision making [12]. Jackson has developed a personality inventory scale, which defines attitude toward risk taking and divides people into being risk seeking, risk neutral, or risk avoiding [13]. Risk-seeking individuals make decisions easily and are not afraid to take risks. Jackson contends that risk seekers enjoy adventure and are unconcerned with danger, while those who are risk avoiders are cautious, hesitant, and security minded. In medical care, for example, a physician who is risk seeking will be quite willing to be the first among peers to order a new procedure, proceed with purposeful watching of a condition, or embrace a new technology [14]. Physicians who are risk avoiding are more comfortable with established procedures or are willing to use more resources, such as admitting a patient with chest pain to rule out a myocardial infarction [14].
Stress from uncertainty has been studied by Gerrity and colleagues [15][16][17], who define this stress as reporting anxiety, uneasiness, discomfort, or emotional turmoil in relation to uncertain situations. According to Clampitt and Williams [18], a person who reports stress from uncertainty is more dogmatic and rigid, avoids ambiguous stimuli, and relies more on authorities for opinions. In health care, physicians act (e.g., order more diagnostic tests or prescribe multiple medications) to reduce their feelings of uncertainty in clinical settings [19].
Physicians who report low levels of stress from uncertainty compared with their peers who report high stress levels have lower care costs for primary care internists [30], fewer ordered neuroimaging studies [19], lower prescribing rates [22], and lower self-reported rates of referrals and burnout [23]. Among family medicine residents, those with low stress from uncer-tainty also reported greater comfort in dealing with death and dying issues [26].
Most of these studies assessed process differences and did not seek differences in patient outcomes based on the risk attitude and stress from uncertainty variation. Those studies that sought differences in outcomes did not find any [14], perhaps because these studies were not designed to evaluate these outcomes or were underpowered to show differences. No data were available on how attitudes toward risk and uncertainty affect use of information resources.
If risk and uncertainty attitudes in physicians affect the use or outcomes of information resources, design of resources may need to take these characteristics into account. By modifying resources based on individual differences such as how risk and uncertainty are viewed, developers may be able to maximize the benefit of using information resources to answer clinical questions through tailoring or individualization.
This study attempted to ascertain whether primary care physicians with varying attitudes regarding risk taking and stress from uncertainty used electronic information resources differently. The study assessed both searching outcomes and processes; the largest differences were projected to be in the process outcomes.

Participant recruitment
This study included primary care physicians in Canada (family physicians) and the United States (family physicians and general internists) who reported seeing patients in clinic settings. Hospitalists and emergency physicians were excluded. The geographical area included southern Ontario and western New York and Pennsylvania. Potential participants were recruited from personal contacts and the mailing list of the Mc-Master Online Rating of Evidence (MORE) rating system. MORE raters are volunteer physicians who rate current studies and reviews for clinical relevance and newsworthiness in their disciplines [32].

Risk and uncertainty scales
After physicians volunteered for the project, they were emailed a link to two short questionnaires that measured their attitude toward risk (Pearson scale) [14] and their comfort with uncertainty (Gerrity scale) [16]. Both scales have been validated and used in studies of health care resource utilization, continuing education styles and preferences, and comfort with delivering bad news [14,17,[19][20][21][22][23][24][25][26][27][28]31]. The scales are included in Appendix A.
Pearson and Gerrity scale scores are normally distributed, and studies most often divide study populations into three categories based on standard deviation data. The current analysis used previous study data cut-points to form groups. The risk-seeking group had Pearson scale scores of eighteen or less, and the risk-avoiding group had scores of twenty or more. Physicians in the middle group (risk neutral with a score of nineteen) were not included in the risk anal-ysis, as the study considered participants in the extreme groups. Similarly, the uncertainty-unstressed group had Gerrity scale scores of fifteen or below, and the uncertainty-stressed group had scores of nineteen and above. The physicians who scored in the middle of the range (sixteen to eighteen; moderate stress group) were not included in the uncertainty analysis.
The Pearson and Gerrity scales are moderately correlated (Spearman rank correlation coefficient ϭ 0.37), showing that they are likely measuring somewhat but not completely different aspects of risk and uncertainty [14]. Because of this statistically significant correlation, the data in the study were analyzed in two separate analyses. Differences were sought comparing physicians in the risk-seeking and risk-avoiding groups and in the uncertainty-stressed and uncertainty-unstressed groups. Because physicians were scored on both scales, any one physician could be in more than one group (e.g., a member of both the risk-avoiding and the uncertainty-stressed groups).

Physician observations
Physicians whose risk or uncertainty scores put them into at least one of the four study groups were observed in their offices for an hour. One researcher (McKibbon) conducted all of the observations. Instructions were standardized and to a minimum. During the interview, physicians completed a questionnaire (Appendix B) about their computer skills and familiarity with commonly used information resources-Internet, MEDLINE, Physicians Information and Education Resources (PIER) from the American College of Physicians, Clinical Evidence, and UpToDate. They also answered 23 multiple-choice, clinically focused questions used by Hersh and colleagues in a previous study [5] and available online [33]. Questions were developed for primary care physicians with input from practicing clinicians [5]. Each question had a firm answer (yes, no, or evidence not available to support an answer). For example, one of the questions queried whether the majority of terminal AIDS patients (Ͼ50%) have clinical symptoms of cardiac involvement (correct answer: no). Physicians also listed their answer certainty (0 to 100%) on the paper forms. From this list of questions, they chose 2 for which they pursued an answer using their own information resources. Clinicians were not told if their answers were correct before choosing. Most choices were based on interest of the clinicians.
The participants were trained in think-aloud reporting of the searching steps they were taking by using standard methods developed by Ericsson and Simon [34]. Additional details of the observation sessions are available [33]. Prompting during searching was limited to asking the physicians to ''please keep talking.'' In addition to verbalizing their actions, the participants stated their answer to a question again, at the end of the searching session, after prompting from the observer. They also verbally reported their certainty of the correctness of the answer (0 to 100%) after searching.
Transcripts of the searching sessions were transcribed and checked by the lead researcher (Mc-Kibbon) within 24 hours of completion and verified 2 or more weeks later. Sentences and utterances were divided into discrete thought segments based on breaks in the verbal flow or grammatical operators such as verbs. Most sentences contained more than 1 segment. The segments were then coded for content analysis. Codes were developed starting with a comprehensive review of 10% of the segmented transcripts. Codes were expanded as needed to reflect additional ideas or themes during coding of all transcripts. Two complementary coding systems were designed to capture actions as well as the thought processes related to the searching actions. Coding categories included specific resources and classes of resources, searching actions (e.g., explode, index terms, or subheadings), and actions reported when reviewing retrievals and assessing evidence. The codes were developed based on the data in the transcriptions rather than established categorizations.
One of the major components of the thought process analysis is identifying and collecting search heuristics. Heuristics are rules of thumb or generalizations that facilitate complex decision making. Heuristics are often seen as overly simplistic rules or guides that work in most ordinary circumstances. Two examples of searching heuristics are, ''If a trusted organization published this (e.g., World Health Organization), it must be right so I will change my answer to agree with the resource'' and ''If I have twenty-five or fewer citations in my retrieval, I will stop searching and look at them.'' The process codes were also divided into ''active'' and ''inactive'' codes. An active code was one that described the action that the physician was taking (e.g., ''I'm putting the whole question into Google to see what it does with it''). Inactive codes described general issues or were explanations (e.g., ''I have not renewed my InfoPOEMs membership'' or ''I hate PDFs as they load too slowly'').
A full description of the process and codes used is available online, as well as samples of coded transcriptions [33]. The lead researcher coded all transcripts with input from a second researcher (Crowley). Intrarater reliability was checked once using 10% of the transcripts and found to be satisfactory (kappa Ͼ 75%).

Outcomes
Both the outcomes of searching and the searching processes were measured and compared across the groups of physicians (risk-seeking vs. risk-avoiding in one set of analyses and uncertainty-stressed vs. uncertainty-unstressed in other analyses). Time taken (total and per question based on timing embedded in the transcribed data), correctness of answer before and after searching, and certainty of the correctness of their answers constituted the searching outcomes. The correct answers were available [5] and reconfirmed by the lead researcher with clinician input the week that Physicians completed the two scales independently to determine both their attitude toward risk and their level of stress from uncertainty. The study analysis included two comparisons: risk-seeking versus risk-avoiding and uncertainty-stressed versus uncertainty-unstressed physicians. Most physicians were included in both sets of analyses.

Table 2
Searching outcomes for physicians in the risk-seeking, risk-avoiding, uncertainty-unstressed, and uncertainty-stressed groups  observations were started. Process outcomes included number of resources and search techniques used, resources and search techniques used, number of searching cycles (individual search attempts within one resource or change to a new resource), the number of segments and codes employed to categorize the transcripts, categories of codes, and proportion of codes across the risk and uncertainty groups. These outcomes were extracted from the segmented and coded transcriptions.

Statistical analysis
The group sizes were small and underpowered (Ͻ 20%) to detect differences. Statistical tests included ttests for means and chi-squared tests for proportions. Differences in distribution of resources across the groups were assessed using conditional logistic regression testing. No adjustments were done for multiple comparisons [35].

RESULTS
Invitations were sent to more than 260 clinicians. Forty-eight volunteered and 43 completed the Pearson and Gerrity scales. Twenty-five physicians (21 men and 4 women, 17 from Canada and 8 from the United States, 22 family physicians and 3 internists, and 24 board certified physicians) were observed in their offices. Eleven physicians were in the risk-seeking group, 11 were in the risk-avoiding group, 10 were in the uncertainty-unstressed group, and 11 were in the uncertainty-stressed group (Table 1). Risk attitude and stress from uncertainty were not associated with sophistication or frequency of use of computers or with familiarity or use of specific information resources-Internet, MEDLINE, PIER, Clinical Evidence, and UpToDate (t-tests using numerical values for scale scores). Full data are available [33].

Searching examples
Three examples illustrate the searching process. A woman in the uncertainty-unstressed group used Clinical Evidence to correctly answer the question, ''Is there benefit for ultrasound as physical therapy for sprained ankle?'' This search included just one cycle of searching, one set of terms in one database (Clinical Evidence). Second, a man (risk-seeking and uncertainty-unstressed) used MEDLINE and one Medical Subject Headings (MeSH) term (''Prostate Neoplasms'') and read abstracts for five minutes to address incorrectly the question, ''Is the elevation of alkaline phosphatase a better indicator of recurring prostate cancer than a rising prostate specific antigen?'' The answer was indeterminate. Third, a man (risk-avoiding and uncertainty-unstressed) correctly answered the question ''Are non-acytlated salicylates really safer in patients with nonsteroidal anti-inflammatory drug gastrointestinal intolerance?'' after seeking information from Google by searching on the majority of the words in the question.

Searching outcomes
Time taken for searching 2 questions, mean percentage of correct answers, and answer certainty are shown in Table 2. No differences were observed across the two Notes: Searching cycle ϭ a change or move to another method or approach in seeking information for a specific question (changing most of the searching terms or moving to another database); Technique ϭ searching techniques included formal limits, cache and highlight, control F to find, keyword (non-MeSH), MeSH, combine, explode, focus (majoring), subheadings, ''use whole question,'' table of contents, and spelling issues as taken from the coded segments data. An individual technique could be used more than once per question.

Table 4
Resource use in the four physician groups (data from two searches per physician combined) comparisons (risk-seeking vs. risk-avoiding and uncertainty-unstressed vs. uncertainty-stressed) for any outcome. Of note, however, was the very low level of correctness of the answers before and after searching, as well as some suggestive data that the uncertaintystressed group had a non-statistically significant improvement of an absolute 2% increase in their rate of correctness.

Resources
Participants were somewhat more certain of their answer if it was correct than if it was incorrect (P Յ 0.03). The increase in certainty was approximately 9% to 12% across the 4 groups. Attitude toward risk taking and stress from uncertainty were not associated with differences in search outcomes of time and correctness of answers for the primary care physicians in the study.

Process outcomes
Despite seeing no substantial differences for searching outcomes based on groups, differences were observed in the search processes used by the physicians in different risk and uncertainty groups. Table 3 shows data on physicians' actual searching processes, including the number of cycles in searching and the number and type of searching techniques used. Thirteen different searching technique categories were used: keywords, limits, combine, MeSH, subheadings, spelling issues, tables of contents, control F to find, starting again, use of whole question, cache and highlight terms, explode, and focus (major aspect).
Physicians in the four risk and uncertainty groups used all techniques except for cache and highlight (a technique used to highlight the search terms in a Web document in color for easier identification of the presence of the searching terms in a retrieved item); this technique was only used by physicians in the riskseeking and uncertainty-unstressed groups. In addition, more physicians in the risk-seeking group than the risk-avoiding group used the technique of control-F to find phrases or words in a retrieved document. Overall, there was a trend toward a statistically significant difference in the distribution of search methods (number and proportion of resources across groups) used by the physicians in the risk-seeking versus risk-avoiding groups (conditional logistic regression test, P ϭ 0.2). Table 4 indicates the number of resources used in the four physician groups. Ten resources were used eighty-two different times for the forty-six searches. The resources and number of times each resource was used for all study participants and per group are listed in the online only content of this article (Appendix C).
All searchers used multiple resources. No differences were seen across groups for overall resource use Table 5 Coding data for the searches done by the four groups of physicians (data combined across 2 searches per physician) 5% (3%) 8% (6%) 8% (5%) 5% (3%) P ϭ 0.04 P ϭ 0.04 * Segments ϭ separate idea or thought phrases in the transcribed descriptions of the searching processes. † Codes are assigned to each segment, often in a 1 to 1 matching although a segment could have more than 1 code. Codes represent both actions and thoughts during searching. ‡ Heuristics are useful ''short cuts'' or general rules of action that are not always accurate. Heuristics are useful in many situations for simplifying complex processes.
(number and proportion; conditional logistic regression, P Ͼ 0.5) and not for strongly evidence-based resources (Clinical Evidence, Cochrane Database of Systematic Reviews, and OVID Evidence Based Medicine Reviews) compared with other resources (chi-squared, P Ͼ 0.5). Physicians in the risk-avoiding group showed a trend toward less use of systems that provided answers or summaries of evidence (Clinical Evidence, Cochrane Database of Systematic Reviews, Ovid Evidence Based Medicine Reviews, UpToDate, and InfoPOEMs; chi-squared, P ϭ 0.07) than those in the risk-seeking group. Physicians in the uncertaintystressed group showed a trend toward less use of MEDLINE (Ovid MEDLINE and PubMed) than physicians who were in the uncertainty-unstressed group (chi squared, P ϭ 0.07). Analyses of the coded transcriptions of the searches showed differences in the searching processes for physicians in different risk and uncertainty groups (Table  5). There was a trend toward fewer segments (representing ideas or complete thoughts) in physicians in the uncertainty-stressed group (P ϭ 0.06). Similarly, there was a trend toward fewer codes represented in transcripted observations of the risk-avoiding group (P ϭ 0.07).
Physicians in the uncertainty-stressed group spent a higher proportion of their searching time on active searching and assessment of the retrievals (P ϭ 0.03) and less time discussing issues related to searching in general (P ϭ 0.04) than physicians in the uncertaintyunstressed group.
The most substantial difference among the risk and uncertainty stress groups was seen in the number of searching heuristics that the individuals used. Compared with their peers, physicians in the risk-avoiding (P ϭ 0.001) and uncertainty-stressed groups (P ϭ 0.03) used far fewer searching heuristics. The physicians in these two groups spent their searching energies concentrating on the searching and analyses of results processes rather than relying on heuristics to get their answers.

DISCUSSION
Studies have shown that physicians with differing attitudes toward risk taking and different levels of stress from uncertainty use health care resources differently across many disciplines and care areas [14][15][16][17][19][20][21][22][23][24][25][26][27][28][29][30]. In spite of these process differences, risk and uncertainty do not appear to relate to differences in outcomes. Because information resources are important to clinical care and are being integrated into electronic medical records systems, it is important to understand how they are used to enhance implementation. Based on previous research, the investigators expected to find differences in information searching processes and, hopefully, differences in searching outcomes. The analysis revealed differences in processes without accompanying differences in outcomes.
All of the study physicians, regardless of their attitude toward risk taking or uncertainty stress, took the same amount of time in searching and generally obtained the same number of correct answers when searching for answers to clinical questions using their own information resources, although the uncertaintystressed group showed some evidence of improved correctness. However, of note was the overall very low rate of correct answers. The questions were developed based on the suggestions of primary care physicians and were not designed to be difficult or obscure. Baseline rates of correct answers for this study and the study by Hersh et al. of medical and nurse practitioner students were similar [5]. The primary care physicians in this study did not substantially improve their scores after searching, while the medical students and nurse practitioner students in the original study examining these questions [5] obtained much higher rates (by approximately 10%) after MEDLINE search training. This finding of similar before-searching scores for participants in both studies with improvements only after training has implications for librarians for two issues. The first is related to the need for training for practicing clinicians or clinicians-in-training. The second re-lates to the choice of resources. The physicians studied here chose their resources, and these chosen resources did not improve their question-answering abilities. This reinforces the importance of providing high-quality information resources and training end users to employ them effectively.
All physicians reported higher certainty with their answers if the answers were initially truly correct than if they were incorrect. The difference in their certainty between correct and incorrect answers was approximately 10% (range 9% to 12%). Although this difference is statistically significant, a physician is probably not likely to be compelled to seek information more often depending on the 70% for a correct answer versus 61% for an incorrect answer certainty spread. The data suggest that clinicians do not know when they need to seek additional information. This fact, combined with the strikingly low number of correct answers after searching (less than 50% correct overall, with little improvement after searching, except for a trend toward improvement in the uncertainty-stressed group) adds weight to the need for further research into which resources and features of resources are associated with improving clinicians' abilities to obtain correct answers to their clinical questions.
An intriguing finding from Schaafsma's study of the accuracy of peers in answering questions was that if the person who answered the question could provide the evidence (a citation or copy of an article), their accuracy was 83% [3]. In contrast, only 35% of the answers were correct if no evidence was provided to back up the answer. This finding reinforces the importance of the role of evidence in external information resources that are used to support decisions.
It may be necessary to redesign information resources or training programs to address differences in riskseeking and risk-avoiding physicians and those who are stressed and unstressed by uncertainty. Compared with risk-seeking physicians, risk-avoiding physicians showed a trend toward fewer codes describing their searching processes and differences in their searching techniques. They also tended to use more resources that provide answers and summaries of evidence. In summary, risk-avoiders spent their searching time working with the system and its retrieval. They likely value search systems that provide guidance on what occurs during searching and allows easy evaluation of the results. Risk-avoiding physicians may also value search education based on in-depth instruction that explains processes. In contrast, tailoring a system for risk-seeking physicians would likely involve a transparent system allowing the searcher more control and resource choices as well as providing searching heuristics-quick fixes and fast accesses to resources.
Physicians who were stressed from uncertainty also showed differences in searching processes compared with their peers who were unstressed. They trended toward fewer segments in their transcripts and less use of MEDLINE. The uncertainty-stressed physicians also used far fewer searching heuristics, spent a higher proportion of their time in active searching, and spent less time on general discussions with a slight trend toward improved correctness with searching. Tailoring electronic information systems for those physicians who are stressed by uncertainty would include features that allowed for control of searching and choice of resources that provide direction as well as the evidence backing up that suggested action rather than a MEDLINE or MEDLINE-like resource. Training differences would also reflect differences in approach: quick-fix heuristics and speed for the uncertainty-unstressed versus in-depth concentration on function and presentation of retrievals for easy evaluation for uncertaintystressed physicians.
The study had several limitations. First, few participants were involved, with only 10 or 11 physicians in each risk and uncertainty group. This low number allowed for only about 10% to 20% power to detect actual differences in the groups. Physicians were volunteers, likely making them a nonrepresentative group who were comfortable with their searching abilities. The questions were simulated questions and not ones that the physicians had encountered in their daily practices. The act of observing the physicians likely changed behavior. In addition, the score cut-points chosen for the risk and uncertainty groups, as well as the exclusion of risk-and uncertainty-neutral physicians, also might have affected the findings. The clinicians used a variety of information resources, making comparisons of outcomes challenging, although knowledge of the range of resources employed was a useful outcome of the study. In spite of these limitations, the study still found some striking differences and provided insights into the process of seeking information using electronic information resources based on risk taking and uncertainty attitudes of primary care physicians.

CONCLUSION
Clinicians who access electronic information resources use a wide variety of resources and employ a variety of searching techniques. Participants in this study were not able to predict when their answer was correct, and the number of correct answers after using their own information resources was low. Research should be done to determine what resources or features of resources are associated with correct answers. Ely and colleagues have started this process by looking at obstacles that exist for physicians when they seek information and desired features of information resources [18,36,37]. The next step is to move beyond preferences to assess actual use of resources and searching outcomes.
Research also needs to be done on tailoring electronic information resources to address differences in searching process for physicians who are risk seeking or risk avoiding. The major difference found in this study is the need for those physicians who are risk seeking or unstressed from uncertainty to have fast systems and searching heuristics (quick-fix procedures), while the contrasting risk and uncertainty groups want to know how the searching systems work and have access that allows for evaluation and modi-fication of the searching processes and outcomes. Other individual characteristics of physicians should also be assessed to determine if they too affect how electronic information resources are used and answers found.