ENVIRONMENTAL HEALTH SCIENCES

Kim Tieu, Professor, Interim Chair Marcus S. Cooke, Professor Diana Azzam, Research Assistant Professor Jeremy W. Chambers, Associate Professor and Doctoral Program Director Alok Deoraj, Senior Instructor and Graduate Program Director (MPH) Quentin Felty, Associate Professor Tomás R. Guilarte, Professor and Dean, Robert Stempel College of Public Health and Social Work Jason Richardson, Professor and Associate Dean of Research Terrilyn A. Richardson, Clinical Associate Professor Deodutta Roy, Professor Stanislaw Wnuk, Professor and Associate Dean of Graduate Education Mohammad Hossain, Assistant Professor


Introduction
The increased prevalence of end-user searching has been a widely reported trend over the past decade, paralleling the trend towards a higher degree of computer literacy in general among students and professionals alike.Database producers and vendors, perhaps believing the library market to be saturated, have increasingly targeted end-users with a variety of products, pricing schemes, and interfaces to facilitate their adoption of searching.The simultaneous development of Web technology has made it attractive for database producers to offer their products via a Web interface that is readily accessible to end-users in a variety of locations without the need for installing specialized or proprietary client interface software: Dialog, Lexis-Nexis, Ovid, and the Institute for Scientific Information (ISI) are all prominent players in this field.
In response to the undeniable reality that a growing number of users are undertaking their own searches for information, many librarians have taken on responsibility for training.Certain assumptions generally underlie these training efforts-namely, that people who have undergone training, especially those who have volunteered for training, are more likely to continue to use the system and become more effective users than those who have not had training.It may also be assumed that the type of search carried out by end-users does not differ significantly from the type of search carried out by a librarian on the end-user's behalf, and that the same standards of retrieval, precision and recall, should apply.
Internet access to databases has become more prevalent, use of resources such as Medline and Toxline has increasingly shifted to the Web, notably through the National Library of Medicine (NLM)'s Internet Grateful Med and PubMed interfaces.Access to Chemical Abstracts and to the online version of ISI's Science Citation Index were restricted to mediated use through library staff; however, access to ISI's Web of Science became available to NIEHS researchers through a site license in February, 1998.Interest in end-user searching among NIEHS researchers has grown steadily.
Training sessions in Web of Science were developed and have been offered by reference staff twice each month.For this study, scientists who had completed one of these training sessions between May 1998, when they were first offered, and December 1998 were asked to complete a survey in an effort to determine the level of use of the database, the kinds of searches that were being conducted, and some of the factors influencing any variance in the scientists' level of use.
End-user training has long been an important responsibility of librarians, and its importance is likely to grow as database producers continue to provide access to information across a variety of user-friendly platforms.This study will build on previous work to examine the online searching habits and implementation levels of end-users who have completed at least one formal training session.By seeking to identify the factors that are most likely to influence user acceptance and continued use of online searching, this study may assist librarians and instructors in developing effective and successful approaches to end-user training.

Review of Relevant Literature
A study carried out in the late 1980s in Toronto set the framework for this study by examining the relationship between end-user training and the level of implementation of end-user searching among Canadian health professionals (Marshall, 1989).Marshall notes that an important first step in studying end-user behavior is to establish a method to evaluate or measure end-user searching; by what standard shall it be judged?Haines, studying end-user searching among research scientists at Kodak, also notes that it is "difficult, if not impossible, to determine the degree to which having access to a certain body of information improves the chemist's work in the laboratory" (Haines, 1982, p. 18).Yet without a valid measure of end-user searching it is difficult to evaluate training methods and their relative effectiveness.
Studies of end-user searching often focus on the "correctness" of the search, the number of procedural errors made, the rates of recall and precision, the cost per pertinent item, or simply the amount of time spent online (Lancaster, 1993, ch. 11).Yet while some types of search may lend themselves to this type of quantitative analysis, for others, such as online browsing, it may be less appropriate.Moreover, just as there are different types of search, there are different levels of evaluation.Lancaster (1993, ch. 12) and others distinguish between evaluating the learning acquired by participants and evaluating the behavioral change of participants, both valid measures of end-user effectiveness, as two of several approaches to evaluating bibliographic instruction.In discussing the evaluation of end-user searches, Cheney urges the integration of both procedural and conceptual skills into bibliographic instruction and evaluation.She specifically observes that such an instruction model allows end users "to use computerized sources at different stages of the research process for different purposes" (Cheney, 1991, p. 154).Marshall (1990) notes that since users vary in their information needs and subject domain, search time alone is an inadequate measure of end-user searching effectiveness.
She suggests a model for measuring the level of implementation of searching among endusers that avoids subjective, one-size-fits-all judgments of searching effectiveness.
Instead, Marshall measures thirteen objective, rather diverse items, such as the number of databases searched, number of locations where online databases could be accessed, number of others shown how to search; ownership of hardware, and frequency of searching.Scores for these items are added to provide a composite score that proves to be a reasonably reliable indicator of a person's level of implementation, a measure of searching behavior that avoids the problematic issue of judging the end-user's searching performance and placing a value on the information retrieved (Marshall, 1989).
Developing a similarly complex measure of implementation proved to be somewhat beyond the scope of this paper.Instead, amount and frequency of use are taken as indicators of the users' commitment and involvement with online database searching.In discussing the results, it will be noted that amount of use is only one facet of a person's end-user searching profile, but a significant one nevertheless.
Another issue that has been noted repeatedly by researchers studying end-user searching behavior is continuity, or persistence.Enthusiasm for searching can wear off-initial interest and use do not guarantee continued use (Marshall, 1989;Haines, 1982).
This issue is of considerable importance to a library in the process of deciding how much effort to devote to a training program for end-users, for whom searching may be a novelty soon to be abandoned.

The Diffusion of Innovations
Marshall's 1990 study developed a model for analyzing end-user searching that was drawn from diffusion of innovations theory pioneered by Everett Rogers in the 1960s and developed by him over several decades.Rogers identifies several attributes of an innovation (in Marshall's study, end-user searching) that are strong predictors of continuing personal commitment.These attributes are relative advantage over previous practice, compatibility with values and needs, complexity of the innovation, its trialability, and its observability.Potential adopters that perceive an innovation as having greater degrees of those attributes will adopt it more readily and with greater rates of continuance than those who do not.It is reasonable to believe that by assessing the enduser population's perceptions of these attributes with regard to online searching, a librarian may be able to better predict the long-term effectiveness of a training program, and justify the use of resources it requires.
The attribute found by Marshall (1990) to be the strongest predictor of end-user continuance is the perceived complexity of the database search system.This finding appears to have been taken to heart by the developers of database search interfaces, who have gone to some lengths to mask the complexities of the database structure with easyto-master search options.Database producers whose databases are especially complex face the challenge of creating a Web product that incorporates the best features of a highly structured database with the requirements of an inexperienced user population.
Databases in the health sciences, such as Medline and Chemical Abstracts, are among the most highly structured of the group, with complex thesauri and sophisticated tools for search and retrieval.For databases like these, some degree of complexity is probably unavoidable, making it all the more important for trainers to find ways to demystify the procedural complexities of the search.
The online science citation database that preceded Web of Science, Science Citation Index Expanded, was also notable for its complexity, both at the procedural and conceptual levels.In practical terms, the ability of researchers to use citation searching to its full potential, as a means of retrieving items of interest that are linked laterally, by associated citations rather than by applied indexing terms, has been sharply limited by this procedural complexity and by the technological capacity of the search interface (also, perhaps, by the rather high cost of per-search pricing structures).The use of hypertext linking with the Web of Science interface has made feasible the use of citation searching as a research tool.
A second attribute that is an important predictor in determining implementation level among end-users, according to Marshall (1990), is the degree of relative advantage of an online search system.Relative advantage is a broad concept that in the context of end-user searching can usefully be broken down into several components (Marshall, 1990).First, the advantage of personal access and control-a term that embraces both the convenience of individual access to the system and the element of personal control over the search process-and second, the perceived advantage of the innovation (online searching) over the earlier alternative.The importance of personal access and control as a factor in end-user acceptance has long been noted by researchers; Haines (1982) notes that the convenience of searching, the speed of the results, and ease of access to the terminal were important factors in the researchers' satisfaction with online searching.
Many of the searchers in her study appreciated the ability to conduct searches without having to go through an intermediary.Sewell (1986) found that the volume of searching in an academic health-sciences setting was directly related to the convenient placement of the terminal in the workplace.
Interestingly, a study at Hofstra University's Axinn Library led Barbuto and Cevallos (1991) to conclude that end-user online searching had taken hold to such an extent that patrons would go through almost any inconvenience rather than make use of the alternative, printed reference sources.In fact, this observation tends to highlight the importance of the second component of relative advantage-the advantageous comparison with previous or alternative methods.Marshall (1990) noted that the perceived degree of relative advantage of online searching of Medline over previous methods of finding information (e.g., searching printed indexes and abstracts) was a strong predictor both of implementation level and of continued personal commitment to searching.Where online searching is perceived as not providing a significant advantage over previous methods, such as a mediated search by a librarian, the end-user's commitment to searching is correspondingly weaker.In the experiences reported by Haines (1982) and by Sewell (1986), it appears that the relative advantage of personal control over the search was not enough to outweigh the inconvenience of lack of ready access to a terminal.Barbuto and Cevallos (1991) noted a considerable increase in the volume of enduser searching over the two years of their study compared to a period several years earlier.This increase far outnumbered the decline in librarian-mediated searches over the same period, a fact which the authors attribute to the Axinn Library's switch from a fee-based search system to CD-ROM searching via a local area network.They concluded that providing an unlimited, no-cost, and risk-free learning environment for searchers encourages greater use.This conclusion is not universally accepted; Marshall (1990) found that free access, termed "trialability" in the diffusion of innovation literature, ranked low on the list of important attributes in predicting level of use, and Sewell (1986) noted that cost did not seem to be a major factor in researchers' deciding whether to search for themselves.
In fact, the concept of "trialability" has two aspects, one related to cost and one more closely resembling convenience.As Marshall and Sewell suggest, cost may not be a significant factor in the end-user's mind influencing the adoption of online searching; the increase in end-user searches noted by Barbuto and Cevallos very likely owes more to the convenience of CD-ROM access over the relative inconvenience of setting up a mediated search with a librarian, than to any financial consideration.Similarly, Starr and Renford (1987) found that among persons who did not pursue online searching after taking a class, difficulty obtaining a password was the most commonly cited reason for the discontinuance; cost was also mentioned as a reason.Although cost and convenience can be differentiated from the user's standpoint, in actuality these two factors remain inextricably intertwined, at least from the librarian's perspective, because of the variety of pricing and access structures available from vendors.The library or institution must assume the cost associated with providing the trouble-free, low-barrier access for endusers that will facilitate their acceptance of the system.

Online Browsing and "Reinvention"
An issue raised by several researchers is the type of search that is facilitated by access to online databases, in particular the use of the system for "online browsing," or exploration and discovery of information that a researcher might not have known about otherwise.Haines (1982) notes that while many of the chemists in her study used the online system for "minisearches"-specific factual searches regarding a chemical compound-a significant proportion of the searches appeared to be concept searches, where such "browsing" might occur.Marshall notes that nearly three-quarters of her respondents agreed with the statement that "By using online databases, I often look for information that I would not otherwise have sought" (Marshall, 1990, p.66).Barbuto and Cevallos' (1991) findings also suggest that a large percentage of endusers are making use of "online browsing" as a way of exploring and generating ideasin particular, their findings that users are in general highly satisfied with simple searches that retrieve between one and twenty records suggest that there is something of value for these searchers that is perhaps unrelated to comprehensiveness, high recall and precision.
This possibility has been suggested repeatedly in recent years, with varying degrees of approval or dismay, depending on the viewpoint of the researcher.Cheney (1991), for example, notes that "end users may not expect or desire" (p.152) the same kind of search results achieved by intermediary searchers, with their emphasis on high precision and recall, but are satisfied rather with a reasonable number of relevant references.
Some research has suggested that end-users may possess better critical judgment and better search skills than librarians give them credit for.Kenny and Schroeder (1992) conducted a study in which experienced reference librarians were asked to rate the quality of 33 searches conducted by end-users who returned copies of their research question and a printout of their research strategies.Contrary to the opinions expressed by library staff in prior interviews, to the effect that users' searches were generally of low quality, the expert judgments showed "that the majority of searches evaluated were average to above average" in quality (p.45).In contrast to those who have expressed concern about the ability of end-users to evaluate their results, Sewell (1986) found that of the searchers she interviewed, "users who had failed to retrieve what was available were virtually always aware of the search deficiencies and had satisfied their needs in other ways before we reached them with a revised search" (p.239).She concludes, We should not be misled by the fact that end users are inexperienced with library-type tools.Our users know the primary literature of their fields very well-usually far better than their helpers do.They also know what they want and they will find it in a different way if the online search doesn't provide it.(Sewell, 1986, p. 243) Other researchers have a less sanguine view of end-users' ability to achieve satisfactory results.In his section on end-user searching, Lancaster (1993, ch. 11) cites a number of studies revealing a widespread ineffectiveness among end-users and a worrisome lack of critical judgment in assessing the value of the information retrieved.
Nash and Wilson's widely cited 1991 study, though based on a small sample, found that 80 percent of undergraduates were satisfied with the results of their searches although their search results were (in a majority of cases) quite irrelevant, even in their own estimation; the authors' estimation of the search results' relevance was even lower.Marshall (1989), and Starr and Renford (1987), like many information professionals, are somewhat ambivalent about the low rates of adoption among end-users, and frequent difficulties in using, such tools as MeSH headings and advanced search techniques.Sewell (1986) also notes significant difficulties with these tools, despite her conclusion that end-users can generally be trusted to compensate for their search deficiencies in other ways.In part these different conclusions and attitudes can be traced to the fact that the user populations of Marshall, Starr and Renford, and Sewell, unlike the undergraduate users studied by Nash and Wilson, and by Barbuto and Cevallos, work primarily with such highly structured scientific databases as Medline and Chemical Abstracts, the primary information resources for the medical and chemical sciences respectively.Marshall (1989) found that almost half of her respondents make little use of MeSH headings, yet adds that they seem to get along well enough, possibly because they were satisfied with simple searches in which the use of sophisticated tools for maximum retrieval and precision were less important.Similarly, as mentioned above, Sewell (1986) notes that problems with the incorrect use of MeSH headings persist among endusers, although these errors cause major loss of references in relatively few searches.Starr and Renford (1987) found that even trained end users experienced significant difficulties using MeSH vocabulary and searching techniques, and that some missed highly relevant citations.They conclude that end users and intermediaries may place a different value on information, with many end users "satisfying their information needs with only the basic search techniques" (p.201).They end with a warning, that "although end users are satisfied, they are also missing relevant literature that could be important" (p.201).
Thus a wide range of opinion exists among researchers as to the effectiveness of end user searching, relating in part to the nature and degree of sophistication of the end user community, the complexity of the database, and the purpose of the search as perceived by the researcher and (perhaps differently) by the user.It may be reasonable to suppose, as Sewell, Cheney, and Barbuto and Cevallos suggest, that for some types of search, involving online browsing and discovery, the use of these precision tools is of lesser importance, and possibly that emphasis on training end-users in their finer points may be misplaced.
In his discussion of the nature of innovations, Rogers (1995) introduces the concept of "reinvention": "the degree to which an innovation is changed or modified by a user in the process of its adoption and implementation" (p.174).He notes that, among other factors favoring reinvention, an innovation that is "an abstract concept or that is a tool (like a computer software program) with many possible applications is more likely to be reinvented" (p.178).Rogers also notes that innovations that are relatively complex are more likely to be reinvented (in the course of being simplified), and that reinvention can also occur as a result of an adopter's lack of full knowledge about the innovation (p. 178), both frequent occurrences in end-user online searching.
From this perspective, it is possible to understand that the innovation-online searching-may be adapted by end-users to suit their needs, and that online searching as it is practiced by end-users may serve a different purpose than when practiced by librarians.Interestingly, Rogers remarks that reinvention was slow to gain recognition within the scholarly community and is often resisted by diffusion agencies and change agents, who may feel that "they know best as to the form of the innovation that the users

Learning Methods
Another important aspect of end-user searching that is discussed by researchers is the range of methods of learning available.In discussing these various methods, it is helpful to note both the level of incidence of their use, and their perceived usefulness to end-users.Marshall (1989) identified four types of learning method: formal training, informal demonstrations (either by library staff or by colleagues), use of printed manuals, and online instruction or "help."Her study revealed a strong correlation between the number of training events or learning methods employed and the users' level of implementation.Printed manuals were the most common learning method employed, followed by informal demonstrations and formal training.
By contrast, Sewell (1986) found that learning from colleagues was ranked very high as a method by which people actually learned to search online; trial-and-error was also a very common method.When asked how they preferred to learn, however, Sewell's respondents ranked one-on-one training and hands-on experience the highest.
Marshall also notes that quite a lot of informal teaching takes place among end-users themselves; 73% of the respondents in her study know of another health professional doing his or her own searching.Her study did not differentiate between informal demonstrations from library staff or from colleagues, however, so her study is not able to address the issue of social networks as a learning resource.Barry (1997) discusses the question of methods of training in some depth, using a longitudinal, case-study approach to examine some of the social behavior characteristics Second, in contrast to Medline or Chemical Abstracts, the major indexes to the primary literature for scientists in this area, citation databases are relatively less familiar to researchers, and their use has hitherto been restricted to library staff at NIEHS.Hence, researchers' response to the online version can be expected to be uninfluenced by years of prior experience and habit with the database.Thus, as indicated earlier, relative advantage over previous methods becomes less perceptible as a factor influencing end-user acceptance of online searching, allowing the effects of other variables-complexity, compatibility, learning methods-to be seen more clearly.
Of the 93 NIEHS employees who attended one of the training sessions offered between May and December, 1998, an introductory letter was sent to 81 (Appendix A).
(The remaining 12 people had been determined no longer to be active employees.)The letter explained the nature of the study and invited participation.The survey questionnaire (Appendix B) was distributed the following week, and an e-mail reminder was sent out two weeks after the survey was distributed.54 surveys were returned, for a response rate of 67%.Survey data was analyzed using SPSS software.All reported percentages are "valid percentages," i.e., missing or invalid responses are omitted.
The objectives of the study are several: first, to explore several of Marshall's observations, in particular, the existence of a relationship between the number of training events, including formal, informal, print, and use of online help, and level of implementation or use; and the importance of perceived attributes such as complexity and compatibility with information needs as strong predictors of implementation level and continued use.Also, this study will consider methods of learning, as to whether formal training, printed material, or informal demonstrations among colleagues, or even simple experimentation, are viewed as most helpful.Finally, I hope to gain insight into the most common uses of the database system among these researchers, whether for simple searches for a known item, exhaustive searches of citations, or for associative searches for related subjects-online browsing.
The survey questionnaire developed for this study drew heavily upon the work of Dr. Joanne Marshall, using her 1987 survey and subsequent analysis as a model, and I would like here to express my appreciation for her generous assistance.I also wish to express my thanks to Larry Wright, NIEHS reference librarian, for his advice and unfailing encouragement.

Profile of Respondents
Of the 54 respondents, nearly three quarters (74.1%) were working scientists of varying grades; principal investigator, postdoctoral or staff fellow, visiting scientist, or contractor.Another 5% (n=3) identified themselves as "student" and the remaining 20.4% (n=11) identified themselves as "other NIEHS"-of these 11, however, six had attained advanced degrees.Of the three identified as students, one was at the predoctoral level.Thus, at least 74.1% of the respondents were scientific researchers, and in fact a higher percentage-85%-have post-baccalaureate education, a majority at the Ph.D.
level.Nearly 60% (n=32) of the respondents arrived at the institute within the last five years.Of the respondents, 87% had used the Web of Science database "little or not at all" before attending the training session, while the remaining 13% had used it to some extent.

Learning Methods
The first section of the survey was intended to discover the respondents' evaluations of their formal training-the workshop they attended-and of several other methods of learning online search skills as related to Web of Science.Respondents were asked to rate their satisfaction with particular aspects of the Web of Science workshop, and to rate the overall usefulness of the workshop and of five other learning methods: informal demonstrations or assistance from a) a colleague or b) library staff, c) online instruction or "help," d) printed materials and e) experimenting or "trial and error" (Fig. 1).
NIEHS researchers' satisfaction with the formal training they received was extremely high; 69.8 % of the respondents found it "extremely useful," and virtually all the rest-another 28.3%-found it "somewhat useful."None found it to be less than somewhat useful.Of the six learning methods listed, the formal training received by far the highest usefulness rating (98.1% overall).
When asked to rate their satisfaction with specific aspects of the workshop-the topics covered, the effectiveness of the presenter, and the workshop's printed materialsrespondents also reported high levels of satisfaction; over 90% considered both the structure of the workshop and the presenter to be "very good" or "excellent."These results correspond with an informal assessment of earlier workshop evaluations collected at the close of each workshop session-these also were generally highly favorable for all three aspects of the workshop.Interestingly, satisfaction with these printed materials was not as pronounced when respondents were asked to evaluate them as one of the components of the workshop, along with the effectiveness of the presenter and the topics covered.Although all three components received outstandingly high satisfaction ratings, both the presenter and the topics/overall structure of the workshop received somewhat higher ratings than did the printed materials.It may be that the respondents' opinion of the usefulness of the workbook increased over time as they sought to develop their search skills.
Two learning methods-using online help and receiving informal assistance or demonstrations from library staff-received similar rates of use among respondents: 39.6% and 42.3% respectively.Assistance from library staff rated a much higher level of usefulness, however-19.2% of the respondents considered this learning method extremely useful, while only 3.8% considered online help to be extremely useful.By contrast with Sewell's findings and with Barry's experience, the learning method least commonly used by respondents was informal assistance from a colleague-38% reported having received such assistance, with only 8% finding it extremely useful.As mentioned earlier, Marshall's study did not differentiate between informal demonstrations from colleagues or from library staff, ruling out a direct comparison with this study.
In summary, the learning methods considered most useful by respondents were first, the formal training session (69.8%); second, the printed materials distributed to workshop attendees (31.5%), and third, informal assistance from library staff (19.2%).
Experimenting on one's own followed at 14.8%.The most commonly used learning methods, after the formal training session, were experimenting (72.2%), followed by turning to the printed materials (57.4%).This may suggest a preference among these NIEHS end-users for independent learning to the extent possible.
In looking at the total number of learning methods considered to be useful to the respondents, a majority (58.5%) found three or four methods to be at least somewhat useful.A larger majority (66.7%) found just one or two types of learning method to be extremely useful.
Some correlation was found between those who found printed materials to be useful and those who found informal library staff assistance to be useful (r=.28).A similar correlation existed between those who liked the printed materials and those who found online help to be useful (r=.32), perhaps signifying that those who seek assistance with their searching are more likely to welcome it from a variety of sources.A moderately strong correlation (r=.40) existed between those who found experimenting or trial and error to be useful, and those who liked online help, two compatible approaches which are characterized by self-directed learning styles and possibly a greater degree of comfort with an electronic medium.

Complexity
The perceived complexity of the database was another important aspect of online searching that this study attempted to uncover.Respondents were asked to rank, as to ease or difficulty, eleven tasks related to searching Web of Science (Fig. 2).
The results showed a remarkably high degree of confidence among the respondents towards the variety of tasks listed.Among the tasks that were considered easiest, rather surprisingly, were "deciding what type of search to perform" and "learning system commands"; these were considered very or somewhat easy by 92.3% and 86.5% of the respondents, respectively.Deciding what type of search to perform had been expected to present more of a challenge than these results seem to indicate; distinguishing between a cited reference search, a citing reference search, and a topical search, for example, and deciding which is appropriate for a given circumstance are complexities that in the past have made online citation database searching a somewhat daunting prospect for end-users.While ISI has made significant progress in presenting the citation database to the end-user, with clearly explained options and simple menu selections, this degree of confidence with regard to the type of search remains striking; in fact, this finding may be related to the reasons that respondents search Web of Science, discussed in the next section.Other tasks found to be somewhat or very easy by respondents were displaying results (82.4%), following links between citations (84.7%) and downloading or printing results (74.5%).These results are not so surprising; the Web of Science interface is commendably straightforward with regard to the execution and other mechanics of a search.
Among the tasks that were considered at least somewhat difficult by respondents were searching complex topics (32.7%); narrowing a search (27.5%) and broadening a search (24%); and getting what you want out of the database (21.2%).That these aspects of searching Web of Science seemed challenging to a significant percentage of respondents might seem to call into question the higher levels of confidence described above.The first three of these tasks relate to performing a topical or subject search, perhaps combining concepts using Boolean operators and field limiters.As Web of Science is a citation index, rather than a subject index to the literature, it is perhaps understandable that searching complex topics could be challenging.
In general, looking in particular at the broader-based questions regarding complexity, such as "getting what you want out of the database" and "remembering how to use the system," fairly strong correlations were apparent among them.Especially striking were correlations for the variable "searching complex topics" in Web of Science.
Those who found this task more difficult also found it more difficult to learn system commands (r=.514); to narrow and to broaden a search when necessary (r=.589 and r=.516, respectively); and to remember how to use the system (r=.550).
An overall measure of complexity was compiled by adding the scores for each of the eleven items and dividing by the number of items considered applicable ("not applicable" was an option that added zero points to the total score); this overall measure of complexity achieved a reliability rating of 90.
Marshall 's study (1990) found that the perceived complexity of the database was a strong predictor of personal commitment to database searching, in particular the decision to continue searching (continuance).Interestingly, the NIEHS study found that very few of the respondents had discontinued using Web of Science; 92.5% of those who responded were still using the system (although 13.2% stated that they use it only rarely).
Thus continuance is not a helpful variable in this study, and no relationship with perceived complexity can be inferred.
Rather than continuance, frequency and amount of use can be examined as variables which may be related to perceived complexity.In addition to a simple variable measuring frequency of Web of Science use, a broader index of use was created of three variables: the frequency of use, estimated total time spent on Web of Science, and a measure of how recently the user had last accessed Web of Science.This use index achieved a reliability rating of 79.
Moderately strong correlations were apparent between several variables relating to level of use, perceived complexity, and perceived usefulness (Fig. 3).One variable relating to complexity recorded whether the user had found Web of Science to be easier (or more difficult) to use than expected.Another, relating to use, measured whether the researcher's use of Web of Science had increased, decreased, or remained the same since attending the training session.
Several correlations exist between respondents' level of use and whether they found the database as easy to use as expected.Those who found the database easier to use than expected were also more likely to report that their use of the database had increased (r=.58, p<.01).Similarly, using the overall measure of complexity described in the preceding section, a correlation was noted between the perceived overall complexity of the database and the frequency of its use among NIEHS scientists (r=.29, p < .05)(not shown).These findings all tend to confirm Marshall's observation that perceived complexity is a strong predictor of personal commitment to database use and can be a major barrier to end-user searching.A somewhat weaker correlation was noted between those who found the database more useful than expected, and those whose use of the database had increased (r=.34, p<.05).However, this indicator of usefulness had no significant correlation with either of the variables measuring perceived complexity (the complexity index, and the easier-thanexpected measure) or with the variables indicating overall amount of use (use index, and Web of Science frequency).Although usefulness can be inferred from frequency and continued commitment to database use, ease of use seems to be the factor more strongly associated with frequency and amount of searching.
The section of the survey that examined frequency of use also examined use of other database resources available to NIEHS researchers, in particular Medline (in CD-ROM or PubMed formats), Toxline, TOMES (a database of chemical toxicity information) and Current Contents Connect.This section of the survey yielded several interesting findings (Fig. 4).The most striking finding relates to the frequency of use; close to half-47.2%of the respondents state that they use Web of Science on a daily or weekly basis (most-39.6%-saidthey used it weekly).This contrasts with other databases whose use might also have been expected to be very high: Medline and PubMed were used on a daily or weekly basis by only 27% and 26% of the respondents.Toxline is as heavily used by only 15.5%, and Current Contents by 14.6%.Even allowing for some degree of response bias, these results show a surprisingly high level of use of this database.

Compatibility and Reasons for Searching
Another fundamental aspect of database use and of end-user satisfaction is compatibility with information needs-the degree to which the information retrieved meets the requirements of the user.Web of Science as a research tool is especially interesting in this regard, as in many ways it differs fundamentally from the other scientific databases mentioned above.Although it is possible to search Web of Science using a traditional keyword or subject phrase approach, support in the database for this approach is minimal.No thesaurus, index terms, or controlled vocabulary exist;3 items in the database are primarily linked by citation data.
Items of interest can be retrieved by following citation links to "parent" (cited) articles or to "child" (citing) articles; further, articles of interest can be retrieved using a proprietary algorithm that ranks related articles by the number of shared citations.Use of citation data as an alternative means to loosely associate related articles in a field is widely understood by information professionals to be an approach that augments, rather than replaces, traditional resources such as Medline, with its complex hierarchical indexing structure and rather daunting array of qualifiers and subheadings.
In light of these assumptions about the expectations with which one might turn to a citation database, a substantial section of the survey was devoted to exploring the reasons cited by NIEHS researchers for searching Web of Science (see Figure 5).Eight possible reasons for searching Web of Science were presented, and respondents were asked to rate them as to their relative importance with regard to their work.As with learning methods, it is helpful to assess both the level of incidence of their occurrence, and their perceived value or importance to the end-user.
Many of the reasons listed related to the type of searches that citation databases are explicitly designed for-keeping up with citations to one's own work, for example, or the work of a particular author-the sort of forward-and-backward searching described above.Another category of reasons for searching, or approach to the database, involved finding articles of interest-related articles-by citation linking: the kind of lateral searching made possible by the Web of Science interface.The last reason listed on the survey was searching by subject or keyword of interest.
Rather surprisingly, it is this last reason for searching-by subject or keywordthat is the most important to the greatest number of NIEHS survey respondents.This type of search was used by 87.8% of the respondents, and was rated very important by 75.5% of them.Another commonly used and favorably regarded reason for searching Web of Science was the first, simply to identify a specific article citation-this reason for searching was cited by 90.4% of the respondents, and rated as very important by 56.9% of them.
Other reasons for searching had slightly lower reported levels of use, in the 70% to 80% range.In terms of importance, finding related articles (either to an article of interest, one's own work, or the work of another author) was an activity that was regarded as very important by a large number of researchers.The fact that finding related articles is an important feature for many NIEHS researchers, coupled with the very high reported rates of use of the database-above and beyond rates of use of other databasessuggests that the database may be popular in large part due to its facility with online browsing, of the type that the "related articles" feature of Web of Science encourages.
Several of the added comments tend to confirm the database's usefulness to researchers as an exploratory tool: "I find it to be a good place to start a search"; "citation links extremely useful," and so on.The database's ease of use and relative lack of procedural complexity also seems to play a part in its widespread adoption among this group of researchers.
There are various possibilities that may help to explain why respondents attach difficulty with searching complex topics and other tasks raises interesting questions of whether these end-users have an adequate awareness of the capabilities and limitations of the system.This finding, combined with the high importance attached to subject and keyword searching, suggest that researchers may tend to regard Web of Science as a subject index to the literature, and may not fully appreciate the distinction between a subject index such as Medline and a citation index.This conclusion is also supported by the frequency of use data, which show that Web of Science has overtaken all other databases, including Medline, Toxline, and TOMES, for these researchers.
These final observations are not intended to detract from the importance and value of online browsing to end-users; as noted above, the system's widespread use among this group of researchers speaks for itself.This study's findings only serve to remind instructors and reference librarians of the need to emphasize the difference between browsing and an exhaustive subject search, and to recognize the appropriateness of each for a different stage of the research process.

Figure 5 :
Figure 5: Relative Importance of Reasons for Searching Web of Science………………………………………………31 among academic researchers at Kings College London.Interviews of both senior academics (professors and lecturers) and the doctoral students and junior researchers they supervise revealed that implicit, unspoken transfer of information-seeking behaviors and methods was far more common than explicit discussion of information-seeking practices or formal training in those skills.1These studies and others clearly suggest the importance of peer acceptance of online searching as an innovation and of social interactions as learning opportunities.MethodThis study will draw on survey data to examine the characteristics of end-users of a specific database, ISI's Web of Science, among NIEHS researchers who have completed a workshop offered by the NIEHS library in its use.Web of Science offers a number of advantages for the purposes of this study.First, with its Web interface, it is clearly intended for an end-user population.The element of complexity, identified by Marshall as a major predictor of user continuity and level of use, has been minimized; users are presented with fill-in forms and menu-based search options rather than command-line syntax.Moreover, the ISI citation databases do not rely on a complicated thesaurus such as MeSH, relying instead on simple indexing by author and title for citation searches, and on computer-based algorithms that are hidden from the user to retrieve relevant citations for more complex, concept searches.The user's need to master a baffling system of commands or procedures is minimal.

Figure
Figure 4: Percentage of Respondents Who Search on a Daily or Weekly Basis Earlier data, taken from brief surveys completed by researchers upon completion of the Web of Science training session, can be used to compare their use of different databases over the period of time since attending the session.When asked what at present, 2 58 of the 77 respondents to the earlier survey (75%) named Medline (networked CD-ROM version); 36 (47%) named PubMed; 16 (21%) named TOMES; 17 (22%) named Toxline; 20 (26%) named Web of Science; and 7 (9%) chose Current Contents.Clearly use of Web of Science has increased dramatically since respondents attended the training session in 1998.
such importance to searching by subject or keyword of interest.ISI's KeyWords Plus retrieval tool may function reasonably well as a means of retrieving articles of interest by topic.Quite possibly, the fact that the Web of Science database licensed to NIEHS (covering the years 1987 forward) represents only a fraction of the number of articles indexed in Medline, for example, makes retrieving articles by keyword a much more manageable proposition.Other evidence, both anecdotal and research-based, exists to confirm the longstanding popularity of subject and keyword searching among end-users.Upon completing the Web of Science training session, the attendees were asked to complete a quick survey containing a few questions about their present database use and search habits, as mentioned earlier.That survey also revealed that subject searching was by far the most common approach for these researchers at that time as well, used by 60 of the 71 respondents (84%) with prior search experience.4

Figure 1: Perceived Usefulness of Learning Methods Learning method Used Not used Extremely useful Somewhat useful Not very or Not at all useful
learning method most highly used and also favorably regarded by respondents was the printed workbook that each received at the training session.Developed by the reference librarian, this workbook contained several sections: a lengthy introductory training guide provided by ISI and customized for the NIEHS community by the reference librarian; a printout of the online help screens; step-by-step guides to selecting the appropriate database for a search, for downloading/exporting records, for creating saved search profiles and so forth; and a page of sample search questions, with suggestions and search tips.