Tenure-Track Science Faculty and the 'Open Access Citation Effect'

INTRODUCTION The observation that open access (OA) articles receive more citations than subscription-based articles is known as the OA citation effect (OACE). Implicit in many OACE studies is the belief that authors are heavily invested in the number of citations their articles receive. This study seeks to determine what influence the OACE has on the decision-making process of tenure-track science faculty when they consider where to submit a manuscript for publication. METHODS Fifteen tenure-track faculty members in the Departments of Biology and Chemistry at the University of North Carolina at Chapel Hill participated in semi-structured interviews employing a variation of the critical incident tecnique. RESULTS Seven of the fifteen faculty members said they would consider making a future article freely-available based on the OACE. Due to dramatically different expectations with respect to the size of the OACE, however, only one of them is likely to seriously consider the OACE when deciding where to submit their next manuscript for publication. DISCUSSION Journal reputation and audience, and the quality of the editorial and review process are the most important factors in deciding where to submit a manuscript for publication. Once a subset of journals has satisfied these criteria, financial and access issues compete with the OACE in making a final decision. CONCLUSION In order to increase the number of OA materials, librarians should continue to emphasize depositing preand post-prints in disciplinary and institutional repositories and retaining the author rights prior to publication in order to make it possible to do so. © 2013 Doty. This open access article is distributed under a Creative Commons Attribution 3.0 Unported License, which allows unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. RESEARCH jlsc-pub.org | Journal of Librarianship and Scholarly Communication Implications for Practice: • The promise of increased citation counts is unlikely to entice many tenure-track science faculty who are not already convinced of the benefits of OA, particularly if participation will cost them thousands of dollars. • When talking to research faculty about OA, greater emphasis should be placed on trying to convince them that access to the literature is not as universal as they assume it to be. • Efforts directed toward the deposition of preand post-prints into institutional and discipline-specific repositories should be prioritized over those trying to convince faculty to publish in OA journals that do not match the reputation and audience of the subscription-based journals in which they normally publish. Received: 08/01/2012 Accepted: 09/18/2012


INTRODUCTION
In 2001, Nature published a correspondence from Steve Lawrence in which he detailed a citation analysis of conference articles in computer science. Lawrence found that computer science articles that were "freely available on the web" had a citation advantage of 336% (median 158%) compared to offline articles. In an attempt to control for article quality, Lawrence also restricted his analysis to the top 20 publication venues by average citation rate. The citation advantage for online articles was determined to be 286% (median 284%). Lawrence ends his correspondence with a request: "To maximize impact, minimize redundancy and speed scientific progress, authors and publishers should aim to make research easy to access" (p. 521).
Making peer-reviewed scholarly research easy to access is one of the goals of the Open Access (OA) movement. There are two roads to OA: the "gold" road is defined as publishing an article in an OA journal, and the "green" road is defined as publishing an article in a subscriptionbased journal and self-archiving a version of it in an OA repository (Harnad et al., 2004;Harnad et al., 2008). The most recent development is a hybrid publishing model in which an author publishing in a subscriptionbased journal has the opportunity to make the article instantly and freely available online for a fee, generally between $500 and $2500 (Schroter, Tite, and Kassem, 2006;Shieber, 2009).
Librarians and information scientists have been inspired by Lawrence's analysis to conduct additional studies examining how publishing venues impact the number of citations to scholarship freely available online. Research on the possible benefits authors receive upon making their work freely available online is on-going and, at times, controversial. A great deal of the controversy is centered around the issue of whether or not OA articles receive more citations than subscription-based articles, hereafter referred to as the OA citation effect (OACE) (n.b., it is often called the "OA citation advantage"). Kurtz et al. (2005) proposed three non-exclusive explanations for the OACE described in the literature:

LITERATURE REVIEW
[1] the OA postulate, [2] the early access (EA) postulate, and [3] the self-selection bias (SB) postulate. The OA postulate states that articles with unrestricted access allow authors to read them more easily and, hence, cite them more frequently. The EA postulate states that articles deposited in a pre-print server (or posted on the web prior to publication) are seen earlier than articles that are not available prior to publication and, as a result, have more time to accumulate citations. The SB postulate states that authors preferentially deposit their most important work in pre-print servers or on the web and, hence, have greater potential to garner more citations. With these postulates in mind, Craig et al. (2007) critically reviewed the OACE literature, asserting that previous studies could only claim correlation and not causation between citation rates and OA. In particular, the methods used in these early studies were heavily criticized for a lack of randomization in article selection, which would control for selection bias and "the skewness of science" (Seglen, 1992), and failure to use fixed windows of time for citation counting-the EA postulate.
Research supporting (Antelman, 2004;Eysenbach, 2006;Norris, Oppenheim, and Rowland, 2008;Wagner, 2010Wagner, , 2011 and casting doubt (Davis, 2006;Davis, 2009;Davis, 2011;Davis & Fromerth, 2007;Davis, Lewenstein, Simon, Booth, and Connolly, 2008;Gaulé & Maystre, 2011;McCabe & Snyder, 2011;Moed, 2007) on the OACE continues to be published. The literature surrounding this topic is inconclusive, and the debate is unlikely to end soon. It is entirely possible that the growth of altmetrics (Priem, Piwowar, and Hemminger, 2012) will only add to the controversy. Consensus seems to be forming, however, around the idea that if an OACE exists in the STEM literature (i.e., it cannot be explained by the EA and SB postulates), its effect is small, or at least significantly smaller than originally reported in the literature.
What is missing from the discussion is the opinion of the authors who publish articles in OA and subscription-based journals: almost all of the OACE studies published to date are based on the underlying assumption that authors care a great deal about the number of citations their articles receive. It is not an unreasonable assumption, but it is one that has not been verified. McCabe & Snyder (2011) broach the possibility of author ambivalence towards the OACE: Even if publishing in an open-access journal were generally associated with a 10% boost in citations, it is not clear that authors in economics and business would be willing to pay several thousand dollars for this benefit, at least in lieu of subsidies. (pp. 37-38) Aksnes & Rip (2009) investigated the perceptions of citations among highly-cited Norwegian scientists. Although issues pertaining to OA did not enter into the discussion, the views of citations expressed by faculty in the fields of chemistry, earth sciences, life sciences, mathematics, physics, and engineering are still relevant to this study. Aksnes & Rip did not find a disciplinedependent or age-dependent view of citations among their respondents. They found that the respondents were "quite knowledgeable" of citations and the many reasons that they may not perfectly reflect scientific contribution or importance. Overall, the general attitude toward citations was conflicted: Citations are sought after because they are perceived as part of the reward system of science but also criticized for not reflecting actual scientific contribution. This then leads to further ambivalences, where high citation counts are accepted as reflecting quality, and low citation counts are explained as a result of visibility and obliteration dynamics. (p. 904)

METHODS
Guided by the following research questions, this study explores the assumption that authors are heavily invested in the number of citations their articles receive: Semi-structured interviews employing a variation of the critical incident technique (Flanagan, 1954) were conducted in each faculty member's office. The length of the interviews varied between 30 and 45 minutes. Detailed field notes were taken during the session. Interviews began with a short discussion of the faculty member's most recent publications-the "critical incidents." After establishing the scope of the session in this manner, questions were limited to those that directly pertained to the research questions under investigation in this study (representative questions are available in the Appendix).

RQ1
These included questions about the author's perception of the role of post-publication metrics in the evaluation of his or her work, the kind of post-publication metrics with which the faculty member is most concerned, and the quantity of post-publication metrics the interviewee expects an article to receive. If the faculty member demonstrated an awareness of OA options, follow-up questions about OA were interjected in an attempt to determine if the faculty member had a qualitatively favorable or unfavorable opinion about OA initiatives. These questions helped determine if there was a bias within the sample in terms of awareness of OA options and attitude toward OA publishing. The qualitative data generated from the interviews was coded and grouped into categories. Coding and category formation occurred via an inductive and iterative process.

RESULTS
Fifteen faculty members participated in this study: eleven were members of the Department of Biology (11 of 48 = 23% of the department), and four were members of the Department of Chemistry (4 of 48 = 8% of the department). Overall, there were five assistant professors (tenure-track), four associate professors (tenured), and six full professors (tenured). Six of the faculty members interviewed for this study have at least one article that is freely-available online, and five of those six have published in a Public Library of Science (PLoS) journal. All six are faculty members in the Department of Biology. It should also be noted that the faculty members interviewed for this study may be considered "successful" researchers at a research-extensive institution. All participants have at least four papers with more than 25 citations. All but two participants have at least one paper with over 100 citations and at least four papers with over 50 citations. Ten of the fifteen participants have at least four papers with over 100 citations, and three participants have published a paper with over 2500 citations.

Open Access Citation Effect
Of the fifteen biology and chemistry faculty members interviewed, seven said that they would consider making their article OA based on the OACE. A closer look at these seven, however, reveals dramatically different expectations with respect to the OACE (see Table 1). One faculty member was not willing to pay more than $1000 per article, and for this amount would require the expected number of citations to double. The other six faculty members discussed the OACE under the assumption that they would be paying PLoS-level prices ($2000-$2500 per article). At this price, one faculty member would require the citations to double in order to

Table 1. Faculty expectations and opinions regarding the open access citation effect.
Expected OACE refers to the relative citation advantage the faculty member believes exists for open access materials in their discipline. Required OACE refers to the relative citation advantage above which the faculty member is willing to participate in an author-pays model of OA publishing at the price per article listed in the subsequent column. increase. Two others would require citations to increase 25%. One of these two, however, does not believe an OACE exists for their field, and the other suspects it is closer to 10%. The remaining two faculty members expect a 10% OACE in their field and are willing to participate in an OA publishing model at this level. One of these two, however, is already an enthusiastic supporter of the OA publishing model and publishes in OA journals as often as possible. Hence, only one of the fifteen faculty members interviewed for this study is likely to seriously consider the OACE when deciding where to submit their next manuscript for publication.

Rank
The remaining eight faculty members were not interested in increasing the number of citations to their work via the OACE. Five of the eight faculty members expressed the opinion that the number of citations is not important. Three of these five are tenured faculty members, and two of them specifically mentioned being at the stage of their career when citations are not important. A sixth faculty member stated that a "combination of factors", not just citation counts, is needed to evaluate the quality of a scientist's work. The remaining two faculty members were much more concerned that their papers be read than cited.
In order to more clearly understand the importance, or lack thereof, of the OACE on a scholar's decision-making process, the participants' views on three areas that overlap with the OACE were examined: (1) What factors do they take into account when deciding where to submit a manuscript for publication?; (2) What are their attitudes concerning post-publication recognition?; and (3) What is their opinion of the OA publishing model?

Publication Decisions
The factors that influence where to submit a manuscript for publication were determined via questions about two or three of the faculty member's most recent publications (see Figure 1). Responses were grouped into four categories: Reputation, Audience, Editorial and Review Process, and Format.
Reputation was mentioned by every faculty member. Twelve of the fifteen faculty members specifically mentioned "reputation of journal." The remaining three faculty members are full professors. Two of them, both members of the Biology Department, described a somewhat complex, multi-step process to publication: "appropriate place for the topic." The responses "high profile for the audience" and "high impact but will not get bogged down in the editorial process" indicate the difficulty involved in trying to deconstruct the thought process that goes into deciding where to submit a manuscript: Reputation, Audience, and the Editorial and Review Process are intertwined.
Audience was also mentioned by every faculty member. Twelve of the fifteen faculty members are represented by the responses "journal has a particular audience" and "specialization of journal." A thirteenth member responded with the previously mentioned "high profile for the audience." The remaining two faculty members replied with "you want your papers to be read." In particular, one of them mentioned publishing in a journal because people still subscribe to the print copy and keep it lying around the lab [Associate Professor, Biology]. The importance of this category in deciding where to submit a manuscript for publication was emphasized by an assistant professor in Biology who stated that the specialization and audience of a journal are more important than the Impact Factor.
The Editorial and Review Process was mentioned by eleven of the fifteen faculty members. Eight of the nine early career faculty members mentioned the editorial and review process as a factor in deciding where to submit a manuscript whereas only three of the six full professors mentioned it. While the majority of the responses referred to various aspects of the editorial process, the responses corresponding to the review process focused exclusively on the quality and timeliness of the reviews. The responses pertaining to the Editorial Process indicate that a good editor running a well-oiled machine may attract manuscripts that could have been submitted to "better" journals. Likewise, authors will avoid sending quality manuscripts to a journal overseen by a poor editor who requires authors to endure a miserable experience.
Format was mentioned by six of the fifteen faculty members (five biologists and one chemist). Interestingly, and perhaps not surprisingly, the two responses favoring a long format for detailed, descriptive work were from full professors, and the three responses favoring a short format were from the early career faculty members. In addition to article length, format also includes the structure of the article (e.g., no methods section) and the distribution mechanism (print vs. electronic). Electronic articles allow the inclusion of color images, video, sound, and other multimedia formats. While an article is usually tailored to the format required by the journal chosen for submission, these responses indicate that occasionally a journal is chosen primarily for its format.

Post-Publication Recognition
A number of questions were asked about post-publication recognition in order to understand the faculty members' attitudes concerning citations to their work. From the responses to these questions, three categories stood out: Peer Recognition, Importance of Citations, and Other Metrics.
Recognition from scientific peers can come in many different ways. A total of twenty-two different kinds of peer recognition were mentioned during the interviews (see Table 2, following page). The responses indicate recognition both at the article level and career level. The most common responses include invitations to give talks at conferences and departmental seminars and to review manuscripts and grants.
The Importance of Citations was discussed by almost all of the faculty members (fourteen out of fifteen). Seven of them believe that "citations are not important," with four of those responses coming from full professors. The majority of this subset (six out of seven) also expressed disinterest in the OACE. The seventh faculty member, while having previously published in an OA journal, requires an OACE sufficiently high that it is unlikely to influence future participation in an OA publishing model. Nonetheless, citations are referenced and considered in tenure decisions. While citations may not be openly discussed during a tenure committee meeting, several faculty members indicated that they and their colleagues check the publication record and citation counts of the candidate prior to the meeting. As a full professor in the Department of Chemistry stated, "citations are good for judging a candidate when you don't know the field." Or put another way, citations "are not a perfect metric but they are good for showing how someone's work is impacting the field." [Associate Professor, Biology] Many of the responses pertaining to tenure had to do with things that are more important than citations, namely the quantity and quality of publications and outside letters of reference.
Seven faculty members (six from biology, one from chemistry) made statements that were categorized under Other Metrics. This category includes topics such as pageviews and pdf downloads, Top 10 or Top 20 lists, Faculty of 1000, and the h-index. With the exception of the h-index, which is a career citation metric, the other metrics are at the article level. Only three faculty members expressed interest in the pageview and pdf download statistics for their articles (an assistant, associate, and full professor, all in the Department of Biology). One of these three has published in a PLoS journal where this information is made public for every article; another had access to this information as the editor of a journal. They felt that these other article-level metrics are interesting but that no one really knows how they correlate with impact. As one faculty member stated, pageviews and pdf downloads "are useful but you have to think about them carefully" [Professor, Biology]. The implication was that at the current level of understanding, pageviews and pdf downloads are more useful to publishers and editors than scholars.

Open Access
For faculty members with no previous experience with an OA publishing model, several questions were asked in order to elicit their opinions on the matter. The six faculty members who already had experience with an OA publishing model were asked how their interest began and why they chose the particular publication venue (see Figure 2, following page). From the responses to these questions, two categories stood out: Financial Issues and Access Issues.
Financial Issues were mentioned by thirteen of the fifteen faculty members. Every mention of financial issues by chemistry faculty members was in the context of not The challenge is to bring these two costs closer together. As one faculty member said, "There is a big difference between $500 and $2500." [Professor, Biology] Access Issues were mentioned by ten of the fifteen faculty members. All four of the chemistry faculty members felt strongly that access to the literature is not a problemthey can get any article they need, and so can their colleagues. Full professors (five out of six) mentioned access issues with more regularity than early career faculty members (five out of nine). This category is most clearly summarized with the following two responses: "anyone anywhere can access the literature" and "the people who matter have access." Only one respondent provided a more tempered response that "most institutions have access to the journals." [Professor, Chemistry] Woven into the belief that access to the literature is not a problem is the assumption that an email to a colleague or a corresponding author will quickly result in access to otherwise inaccessible articles. The reluctance of many faculty members to publish via an OA model becomes a little easier to understand when their views about access to the literature are taken into account.

DISCUSSION
Based on interviews with fifteen tenured and tenure-track biology and chemistry faculty members at the University of North Carolina-Chapel Hill, the OACE has very little influence on the decision-making process used to determine where to submit a manuscript for publication. For the cost of publication in highly-reputable journals like those published by PLoS ($2000-$2500 per article), the seven faculty members who expressed an interest in Responses mentioned by fewer than three faculty members are not displayed.
the OACE expect the total number of citations to increase by either 100%, 50%, 25%, or 10%. As a result, only one of the fifteen faculty members interviewed is likely to take the OACE into consideration the next time they are deciding where to submit a manuscript for publication. This ambivalence about the OACE is also partly due to each one of the interviewed faculty members being a heavily cited research scientist. While sampling bias is introduced through the small sample size, the differences in the scholarly communication practices of the various scientific disciplines, and participant self-selection, the generalizability of these findings is most restricted by the quality of the interviewed faculty members and the quality of the departments in which they reside. Preliminary work with biology and biomedical engineering faculty members at the Georgia Institute of Technology and Emory University has not contradicted this assertion. Additional interviews with faculty members at institutions in each category of the Carnegie Classification (http://classifications.carnegiefoundation. org) would be required in order to determine the extent to which the opinions expressed by the faculty members in this study are universal.
The responses of the faculty members interested in the OACE were practically indistinguishable from those of the faculty members ambivalent about the OACE. One of the more interesting differences concerned access to the literature: seven of the eight members of the group ambivalent about the OACE believe that they and their colleagues have sufficient access to the literature; only four of the seven members of the group interested in the OACE agree. In addition, six of the eight members of the ambivalent group stated that citations are not important while only one member of the interested group concurred. One possible explanation for the lack of interest in citations is that the faculty member conducts research in a slowly moving field, where tenure is frequently achieved before a paper has had time to accumulate a significant number of citations. Four members of the ambivalent group, and three of the six who stated that citations are not important, identified their fields in this way.
The ambivalence surrounding the OACE does not appear to stem from a negative opinion of OA publishing. Four of the seven faculty members who expressed an interest in the OACE have published an article via an OA publishing model, and two more are interested in doing so in the future. Two of the eight members in the group ambivalent about the OACE have published via an OA publishing model, and three additional members expressed an interest in the author-pay model for future publications. Amongst both the faculty members who are interested in the OACE and those who are not, journal reputation and audience are the most important factors when deciding where to publish an article. A lesser, but nonetheless important, third consideration is how well the journal handles its editorial and review process. For some faculty members, the format of the article is also important. A combination of these factors influences their decision on where to submit a manuscript for publication. It is an iterative process that requires "weighing what's the best way to convey your research results" [Professor, Biology]: "You want to find the right audience. You want a high impact but you also don't want to get bogged down in the editorial process." [Professor, Biology] "Highest tiered, broadest readership journal we could get the article into." [Associate Professor, Biology] For many of the faculty members, there are also elements of time and effort that go into the decision: "You target the best journal you can for what you think you've got. Maybe you could do better but there is an amount of time you want to invest in getting it there, too. You have to balance these things, impact and the time required" [Professor, Biology] "We have a paper that we could submit to Macromolecules today and get accepted. Maybe we could do more and submit it to a better journal like JACS. But maybe it takes a year for us to do that work. It just isn't worth it. Ultimately, it doesn't matter if it is JACS or Macromolecules-it will get cited if it is good science, regardless of where it is."

JL SC
Once a faculty member has identified a subset of journals that satisfy these criteria, financial and access issues compete with the OACE in making a final decision. If the price to participate is too high or there is no money available, the faculty member will not choose an OA option. Similarly, if the faculty member believes that everyone (who matters) already has access to the subscription-based journal, an OA option will probably not be chosen.
There is also an underlying belief, similar to opinions expressed by the Norwegian scientists in Aksnes & Rip (2009), that citations are not that important. The following quotes given by untenured faculty members exemplify many of the opinions expressed during the interviews: "Citations are important but not that important… The time lag makes them not terribly useful for tenure, and once you've got tenure it doesn't matter how many times you are cited." [Assistant Professor, Chemistry] "You're in a bad place if citations matter for tenure." [Assistant Professor, Biology] To elaborate upon what the Assistant Professor of Biology has stated so bluntly, the number of citations to one's publications should pale in significance to the quantity and quality of those publications and the amount of research dollars brought in to the university. Also implicit in this statement is the belief that citations are not a good proxy for article quality. Many of the faculty members, in fact, did not view citations as being independent of journal quality.

CONCLUSION
In order to increase the number of OA materials, librarians should continue to emphasize depositing pre-and postprints in disciplinary and institutional repositories and retaining the author rights prior to publication in order to make it possible to do so. This effort is currently aided by OA mandates at both the federal and institutional level. After an extensive study of faculty self-archiving behavior, Jihyun Kim (2010Kim ( , 2011 argues that IR outreach efforts need to focus on digital preservation and copyright. Action items in these areas include "understand[ing] the preservation needs of an institution and [its] stakeholders" and "provid [ing] services that give guidance to faculty on publishers' self-archiving policies and authors' rights" (2011, p.252).
In order to prioritize the action items above and gain a better understanding of the particulars they encompass, librarians need to investigate the motivations and barriers to self-archiving at their institutions (Troll Covey, 2011). This is also a great opportunity for subject librarians to gauge faculty interest and participation in OA issues. How many faculty members in your department have published in an OA journal? How many have paid to make an article in a subscription-based journal freely available? Are any of them editors for OA journals? Do any of them self-archive in discipline-specific or institutional repositories? What issues are most important to them or concern them the most, e.g., maintaining control of copyright, the OACE, increased readership, a change in the scholarly communication landscape, etc.? Are any of them willing to advocate on behalf of the Library?: "By supporting OA myself that encourages my colleagues to do the same thing." [Associate Professor, Biology] Finally, this study points the way to further work. The interviews focused on the gold road to OA-publication in OA journals and paying a fee to make an article in a traditional, subscription-based journal freely available. There was no mention of the green road to OA-making an article freely-available by self-archiving it in an OA repository. It is possible that because the University of North Carolina at Chapel Hill did not at the time have a digital repository and self-archiving was not being advocated across campus, faculty members were unfamiliar with the concept. It is also possible that the faculty members were not interested in self-archiving, or that they did not want multiple versions of a manuscript accessible via the Internet. Because a question specific to self-archiving was not asked during the interviews, one can only speculate about the silence surrounding this topic.
Perhaps the most interesting questions for future work were suggested by one of the faculty members during the discussion of the OACE. First, "does the OACE increase the variance in the kinds of institutions?" [Professor, Biology] Namely, what percentage of citations to OA articles come from first-, second-, and third-tier institutions? How does this compare to traditional articles locked behind a subscription wall? Second, "does the OACE impact the hit level of major scientists?" [Professor, Biology] In other words, who is citing OA articles compared to non-OA articles? Also, how do the answers to these questions vary by discipline? These issues get to the heart of the ambivalence about the OACE displayed by the fifteen faculty members interviewed for this study, echoed in the following statement: "Modern tools are expensive. It takes rich countries and rich universities [to advance the field]." [Professor, Chemistry] A direct extension of this study would be to expand the sample to tenure-track science faculty who are not located at research-extensive institutions. It is possible that less prolific, less cited faculty members have different opinions concerning the OACE than their more elite colleagues at top-tier institutions. In a 10-year longitudinal study of physics faculty at different types of universities (elite, pluralist, and communitarian) and at different stages of their careers (early to mid, mid to late, and late to retired), Joseph Hermanowicz (1998Hermanowicz ( , 2009 found widely varying differences in career focus, professional aspirations, recognition sought, definition of success, perceptions of the reward system of science, orientation to work, work/ family focus, attribution of place, and overall satisfaction. It is a fascinating exploration of the different paths a career can take based on when and where a faculty member receives his/her first position and becomes tenured, and a reminder of the heterogeneity inherent in the academic profession.