Problematizing Peer Review: Academic Librarians’ Pedagogical Approaches to Peer Review

INTRODUCTION This study is the first to consider how academic librarians’ understanding of and participation in the peer review process influences their information literacy pedagogy and practice. METHODS This mixed-methods study uses a modified sequential explanatory design, beginning with a survey of academic librarians in the United States and Canada, followed by interviews with interested study participants. RESULTS & DISCUSSION The researchers find that academic librarians frequently teach about peer review, but approaches vary widely, and though some have adapted the Framework to fit their instruction about peer review, there are no best practices. Instructor demands, the length of instructional sessions, and student level influence whether and how academic librarians contextualize the peer review process. While some academic librarians draw from their personal experience in the peer review process as authors, reviewers, and/or editors in their teaching, academic librarians do not consistently report their personal experience as an influence on their teaching of the peer review process to students. CONCLUSION This article argues that academic librarians should consider the place of peer review in information literacy instruction, including interrogating how scaffolding instruction about peer review may provide a disservice to students from an equity perspective. The authors urge academic librarians who have it to draw on personal experience to contextualize their instruction about peer review.


INTRODUCTION
Peer review is often held up as the arbiter of quality research, frequently referred to as the "gold standard" of academic publishing. This term obscures the complex issues that intertwine with the peer review process, such as power imbalances, unequal access, and academic pressures and constraints. College students are frequently introduced to the concept of peer review in an abstract and decontextualized manner, by way of an instructor simply requiring that sources for an assignment need to be peer reviewed, often as a proxy for quality and credibility. Academic librarians are then tasked with teaching students about peer reviewed sources; however, due to a number of factors addressed in this study, instruction on this complex topic may be as limited as showing students how to filter their results by clicking a "peer-reviewed" box on a database results page.
In this study, the authors explore what kind of context academic librarians include in their instruction about the peer review process and what factors influence their discussion of peer review. Furthermore, as academic librarians are also often involved in the peer review process as authors, editors, and reviewers, the authors of this study seek to understand how personal experiences with scholarly publishing influence academic librarians teaching about peer review.
The authors' interests in this topic stem in part from personal experiences as participants within the academic peer review process as well as experiences teaching peer review to students in a university context. Both authors have served as peer reviewers for scholarly journals and authors of peer reviewed articles as part of designated professional responsibilities as tenure-line faculty members at current and previous institutions. Both authors have observed personal disconnections between their work as authors and reviewers versus in their practice of teaching students about the peer review process in order to facilitate students' understanding of scholarly knowledge production. The authors also began to think through how both convention around information literacy instruction and guiding documents in the profession shaped and constrained their teaching of peer review.

LITERATURE REVIEW
This study is the first to consider how librarians' understanding of and participation in the academic peer review process influences their information literacy pedagogy and practice. It is also part of an ongoing discussion in academic publishing and in librarianship about the value of traditional peer review, as well as the potential benefits offered by newer alternatives such as open peer review. For the purposes of this study, the authors consulted the literature on academic peer review in two main categories: the history of peer review in academic publishing and the treatment of peer review within information literacy philosophy and instruction.
Peer review has a long history in Western scholarly publishing. While there is some debate about the history (and longevity) of peer review (Baldwin, 2017;Fyfe, 2015;Fyfe et al., 2017), some form of editorial review of academic research has been in place since the mid-1600s (Lipscombe, 2016) and it slowly morphed into what is now considered "peer review" (Smith, 2006) by the 1970s (Zuckerman & Merton, 1971). Peer review is now ubiquitous as a tool of quality control in academic publishing, and as such is necessarily the subject of questions around its effectiveness at improving manuscripts (Ma et al., 2013) and the continuing role of traditional peer review in scholarly inquiry in the digital age (Nicholas et al., 2015;Solomon, 2007). Bias has been proven to exist in peer review (Resch, Ernst & Garrow, 2000;Lee, Sugimoto, Zhang, & Cronin 2013;, including the tendency of the traditional peer review process to reproduce wider cultural inequities, including sexism and racism (Demarest, Freeman, & Sugimoto, 2014;Bornmann, Mutz, & Daniel, 2009) by privileging white men's voices in academe. In recent years, there has also been increasing discussion of the ethics of peer review (Souder, 2011) and the circumstances under which peer review fails at its primary mission of quality control (McCook, 2018;Patel, 2014) as well as the ways in which the traditional peer review process can sometimes be an impediment to timely information sharing (Mrowinski et al., 2016).
Scholars in many disciplines have written extensively about the utility and reliability of peer review for their purposes (Fisher & Parisis, 2015;Morley, 2016;Gosden, 2003), and some have offered suggestions for revising the process as well as replacing it with alternative methods (Cosgrove & Flintoft, 2017). Open peer review, which has been discussed intensively by Emily Ford and others, is a popular alternative choice in LIS. Other alternatives include crowdsourcing, re-review opt-out, portable, and cascade review (Kovanis, Trinquart, Ravaud, & Porcher, 2017), none of which have yet to replace the traditional double-blind approach employed by the majority of scholarly journals.
Library literature discusses peer review under the auspices of open access publishing (especially notable is Emily Ford's (2013Ford's ( , 2016aFord's ( , 2016bFord's ( , 2018 work on open peer review), as a peer-to-peer editing activity for improving student writing (Wheeler, 2011;Zwicky & Hands, 2016), or in terms of information evaluation (Rosenzweig, Thill, & Lambert, 2019;Shotton, 2012;Warren & Duckett, 2010), but rarely in terms of critically analyzing the process itself (Potvin, 2017). Articles discussing the value and role of peer review in academic publishing (Kaspar, 2016;Weller, 1995) are most likely to address critiques of the peer review process, while there are a growing number that directly confront bias in the peer review process (García, Rodriguez-Sánchez, & Fdez-Valdivia, 2015) and the moments in which peer review fails at its own stated goals (Leon, 2014).
The critical librarianship literature queries where the critical analysis of peer review fits within the ACRL Framework (Battista et al., 2015;Beilin, 2015). The literature is also beginning to reflect the increased attention to the area of scholarly communication more generally as an area of librarian expertise (Ball, 2011;Association of College & Research Libraries 2013), including some explorations of librarian faculty status and the role and value of librarians publishing in peer reviewed journals (Francis, 2013;Lamothe, 2012). LIS journal publishers, particularly those who create open access journals, contribute to the discussion about peer review by publishing rationales for adopting their specific peer review processes (Brower, 2008).
The LIS sources we found that describe how librarians teach peer review to students overwhelmingly reflect a tendency toward teaching peer review instrumentally rather than critically, emphasizing it's utility as a tool, describing the process through which articles achieve a peer reviewed designation, and ultimately giving instruction on how to find peer reviewed articles on a given topic. Unfortunately, this means that students do not receive the full wealth of knowledge possessed by librarians about the complexities of peer review, including its limits and alternatives intended to address them. Instructional materials that address the information literacy implications of peer review tend to define it as it is traditionally regarded: as scholarly quality control (Bekius, 2017;Repplinger, 2017;Sung, 2017), as do studies examining the use of pedagogical tools, such as formative assessment (Dunaway & Orblych 2011), in information literacy instruction. However, articles about information access and siloing (Booth, 2014), and the scholarly value of non-peer reviewed information (Deitering & Gronemyer, 2011;Mark, 2011a) are likely to mention the ways in which the reliance on peer reviewed information within academia can become a barrier for students (Donovan & O'Donnell, 2013;Mark, 2011b).
Faculty in other disciplines recognize the value of teaching peer review as a complex evaluation tool as well as an integral piece of scholarly discourse. For example, in a phenomenolog-ical study examining the perceptions of faculty teaching information literacy, Dawes (2017) found that, "while teaching students how to use information to communicate within their discipline, faculty introduce the concept of peer review and its place in scholarly conversations" in order to support students' in their ability to critically evaluate sources of evidence. A 2018 study of undergraduates' information behavior and heuristic decision-making confirms that students tend to rank sources by their "ability to advance a predetermined argument," leading the authors to reflect that there is "a need to help students understand the various means by which information is created, and to develop a deeper understanding of processes such as peer review. Such an understanding is important both for the democratic value in teaching students to find the best information to support their own arguments and for equipping students with the ability to evaluate competing claims by weighing the quality of the information used to support them" (McGeough & Rudick).
While no in-depth study of librarians' instructional behavior around peer review could be found, critical information literacy scholars offer context for how librarians can support students deeper engagement with peer review under guided by the Framework and as part of the larger information ecosystem (Elmborg, 2006;Seale, 2016). Jacobs (2008) argues that reflective pedagogical praxis, or the process of examining the ways that your lived experience and personal perspectives influence your teaching and the ways that your teaching does or does not support students in critically engaging with the sociopolitical context of the information universe, is crucial to information literacy instruction. Pedagogical reflexivity around peer review as an instructional topic would entail reflecting on how to align what we teach about peer review with what we, as critical-minded information professionals, know through our experience as academics and as participants in the peer review process.

METHODS
The study 1 uses a modified sequential explanatory design (Creswell, Plano Clark, Gutmann, & Hanson, 2003). This approach involves the collection and analysis of quantitative data, followed by the collection and analysis of qualitative data, with the two methods being integrated in the interpretation of the study. This mixed methods design is used to allow for further explanation and interpretation of findings than with solely a qualitative or quantitative method (Creswell, Plano Clark, Gutmann, & Hanson, 2003). The authors refer to this study as employing a modified version of this design because the first research instrument, a survey, is designed to produce both quantitative and qualitative data, however the second research instrument, semi-structured interviews, is designed to produce only qualitative data.

SURVEY
The survey (Appendix A) consists of closed and open-ended questions meant to gather information on how academic librarians teach peer review and what their personal experiences with the peer review process are. The survey was administered using Qualtrics, a survey software platform, and quantitative analysis was also performed using Qualtrics. Prior to mass distribution, the survey was sent out to a select group of people to pilot the survey and provide feedback. The comments and suggestions provided by pilot users helped the researchers modify the survey in order to improve the validity of the instrument. The survey was then distributed using purposive sampling, which is a nonprobability sample meant to be representative of the population of interest (i.e., academic librarians who teach), by posting the invitation to participate on targeted listservs for instructional librarians (Battaglia, 2008). Criteria for participation consisted of academic librarians employed in Canada or the United States, however because invitations were posted on instruction librarian listservs, the authors sought to encourage participation by academic librarians interested in library instruction. The survey was open between August 15-September 9, 2016, and during this period 216 complete surveys were recorded, of which 202 met the survey criteria. On average, the survey took 31.75 minutes to complete, with a standard deviation of 20.24 minutes.
Survey participants were asked to fill out profession specific demographic information. Of the 202 survey participants, slightly more than half (51.5%) had been in the profession for 0-10 years; with the remainder split between 11-20 years (26.2%), and more than 20 years (22.3%). The majority of participants worked in doctorate or master's granting universities (41.6%, 29.2%, respectively); with the remaining in Associate's college (17.3%), Baccalaureate college (8.9%), and the rest in other categories of the Carnegie classifications. More than half (53%) are in tenure-track, tenured, or tenure-equivalent positions. Almost all (95.5%) survey participants work in the United States; the authors decided to keep the Canadian participants in the study analysis because of the shared guiding documents used in both countries.

INTERVIEWS
An interview invitation was included as the last question on the survey. Survey participants self-identified if they were interested in a follow-up phone or video interview on this subject. All of the 39 survey participants who self-identified as interested in an interview were contacted individually via email to sign up for an interview slot with either of the researchers. In total, 25 interviews were completed. The interviews were semi-structured with a set question script (Appendix B), however, as is typical with the method, both authors could follow topical trajectories in the conversation that may stray from the question script when appropriate (Gubrium, Holstein, Marvasti, & McKinney, 2012).

Qualitative Data Coding
The qualitative data in the survey questions and interviews were analyzed using grounded theory, which employs content analysis using open coding, axial coding, and theoretical coding (Flick, 2018). First, each author independently coded the qualitative data to identify as many ideas and concepts as possible; each open coding was then compared to one another to check for consistency and bias. Once open coding was complete, the authors worked together on axial coding in order to systematically analyze the relationship or interaction found between variables, and formulate the theoretical findings with a smaller set of higher-level concepts (Flick, 2018).

RESULTS
Traditional peer review is largely held in high regard by study participants, with the majority of survey participants (79.1%) agreeing or strongly agreeing that traditional peer review is valuable in the scholarly knowledge production process. Almost all (98.5%) of the survey participants who had taught in the 12-months prior to taking the survey said the topic of peer review came up in their information literacy instruction over the last year. This reverence for traditional peer review is also seen in the study participants' responses to defining the concept and function of peer review. where terms such as "expert" and "quality" were frequent inputs. When asked to define peer review, "expert" and "field" came up frequently, as well as survey participants referring to peer review as a "process," with many describing the steps in a typical traditional peer review process as a way of defining the concept. When asked to describe the function of peer review, most survey participants wrote about peer review ensuring quality in published research.

Providing Context
There are multiple ways that academic librarians provide context about peer review ( Figure  3). While teaching about peer review, nearly all of the survey participants incorporate discussion of scholarly knowledge production (87%) and the publishing industry (61%). Issues of power in academia and the publishing industry were discussed by multiple interview participants, particularly as it relates to race, class, and access to publishing avenues. One participant said, "I talk about who is doing the peer review, and it's normally people who are faculty. And then the demographics of faculty are mostly white and male." Fewer survey participants bring in explicit critiques of peer review (37%), the role that the tenure process has on peer review (29%), and funding (19%) while teaching about peer review. Some of the interview participants bring up discussions of free labor in the publication industry, often as a way to engage students in the lesson, since it is surprising to many students that faculty members are not compensated to write and review research articles. One participant states, "I think it's a lot of academics, doing a lot of work, for publisher's profit." A few interview participants discussed the economics of peer review as a way to contextualize the often-long period of time it takes for an article to go through the process and be published, as well as to contextualize that articles are not always checked with great diligence because of the other demands on academics' time. Some interview participants mentioned finding ways to take peer review out of the academic context, presenting parallel examples that students could more easily relate to. One example to illustrate expertise compares the critique contestants get from professional celebrity chefs on a show like Iron Chef versus culinary feedback from a relative. A few interview participants spoke about their desires for students to think about peer review as more than just a checkbox on the databases; one participant elaborated this is in order to "[critique] the system rather than just the content of it." Survey respondents were asked to rank the influence of external demands on their teaching of peer review on a five-point Likert frequency scale, with the numeric values represented with the following phrases: 1 = none at all, 2 = a little, 3 = a moderate amount, 4 = a lot, and 5 = a great deal (Table 1). The responses that had a mean score of 3 or higher included nonlibrary faculty demands (x = 3.44), length of the information literacy session (x = 3.32), ACRL's Framework for Information Literacy for Higher Education (x = 3.22), and ACRL's Information Literacy Standards for Higher Education (x = 3.01). Survey participants included additional free-text responses in the "other" category which were explored in greater detail during the interviews; these included: academic discipline and student level.

Non-Library Faculty Demands
What academic librarians cover in course-integrated instruction is most often influenced by the instructor of the class, especially their interests and assignments. Study participants' relationships with classroom instructors vary widely; survey and interview participants reported a wide range of autonomy to cover material related to peer review, based on the course instructor. Study participants report that they craft instruction sessions based on the needs of the course and the interests of the classroom instructors, whether or not they are given specific directives about information literacy content. In some situations, study participants are able to influence the structure of course assignments and lectures through their relationships with course instructors. One participant spoke about collaboratively discussing the lesson plan to draw attention to instruction on peer review, saying, "I'll ask, 'Hey, you know, I noticed that [peer review] is one of the requirements, do you feel like your students are comfortable with that or should we spend a couple minutes on that?'" One participant noted the value of the librarian's perspective in discussions around peer review, particularly in clarifying how student work is a part of the process of scholarly knowledge production, stating, "I find that I'm often the only person talking about peer review as part of the ongoing conversation, like the process of scholarly production, and I think it's 'cause most faculty don't think of it that way. And they don't think of it that way when they're looking at student work, which is not the production of a journal article, but it's just like meeting a kind of set of arbitrary assignment requirements that were probably the assignment requirements they had when they were in college."

Length of IL Session
Study participants expressed not having enough time to cover the peer review process in detail, particularly in "one-shot" information literacy instruction or workshops, with an interview participant stating, "time is always a factor." By contrast, interview participants discussed having enough time to adequately contextualize peer review when presented with longer term instructional opportunities such as standalone information literacy courses or embedded instruction that meets multiple times in a term. Some mentioned scaffolding the discussion about peer review so different pieces are discussed at different student levels to help mitigate the lack of time in a single instructional session, and in an effort to ensure that students have a deeper understanding of scholarly knowledge production by the time they graduate.  , 2020). Given that the survey was administered shortly after the Framework was officially adopted, the authors sought to understand how both guiding documents influenced academic librarians teaching of peer review. Not surprisingly, the Framework was more influential than the Standards, however a higher number of participants commented on the difficulty of applying the Framework to their instructional, particularly as it is not as prescriptive as the Standards.

ACRL has two primary guiding documents for information literacy instruction, the Framework for Information Literacy for Higher Education (Framework) which was adopted in
Interview participants cited the following frames from the Framework as particularly influential in thinking about peer review: "Authority Is Constructed and Contextual," "Information Has Value," and "Scholarship as Conversation" (ACRL, 2016). For those who use Framework to teach or at least plan for teaching, these frames provide guiding points and a shared vocabulary for discussing the peer review process. One interview participant speaks about how peer review concepts map to the Framework, "in terms of peer-review, myself and another librarian in information literacy, have tried to match the outcomes more to what the desired outcomes of the framework are."

Academic Discipline
To some degree, the peer review process varies by discipline (Rowley, 2018), and academic librarians' instruction reflects this by discussing the discipline's common publication standards and processes for scholarly knowledge production. Interview participants who taught in the sciences, especially health and medical sciences, discussed the peer review process in greater detail than those teaching in other fields such as arts and humanities. One participant spoke to this in saying, "having [students] recognize that peer review has different values in different disciplines is… a very useful conversation to have, establishes authority of the librarian, if nothing else."

Student Level
Study participants report tailoring their instruction sessions to the academic level and experience of the students involved. Many interview participants who work with first-year students feel that a nuanced discussion of peer review is not possible until the students have a grounding in basic information literacy concepts. An interview participant asked, "why would I talk about that if they don't even know the basics of what the difference is between really basic information sources, you know what I mean?" Interview participants who work with upper level undergraduate and graduate students report introducing critical discussion of peer review at this level, such as this participant who said, "I look at this as a process of building all over throughout their academic career. And hopefully, when they get to be a senior, they know what a scholarly article is. I'm hoping they do." However, with all student levels, study participants push back on the simplified concept that peer reviewed sources are somehow better than other sources, instead talking about different information formats and needs. A participant spoke to this simplification in saying "I don't like the default notion that peer-reviewed research are good sources, and not peer-reviewed material are bad sources; that it's a format of information, and it's a method of information getting published. And sometimes it's the right and appropriate stuff to look at, sometimes it's not."

Participation in the Peer Review Process
More than three-quarters (77.9%) of survey participants said they participated in the peer review process, as an author, editor, and/or reviewer. The influence of this participation was further explored in the interviews. Some interview participants discussed how they draw on their personal experience with peer review as a way to contextualize their instruction. This can manifest as personal stories, or stories from their colleagues, about their experiences as authors, reviewers, and/or editors of peer reviewed publications. One participant says, "Everyone has a bad day or a bad week. And sometimes, [that] attitude can come through in peer review... And so, I guess what it's done is it's giving me a more realistic opinion of what peer review can do. I maybe don't have that glossy received knowledge that I might have gotten from library school." Some participants with peer review experience report being able to share their insights with students and answer questions about how long the process takes, author compensation, and other topics that may be less clear if one does not have first-hand experience.
However, bringing personal experience into one's teaching of peer review is not universal. While many of the interview participants shared both negative and positive experiences with various parts of the peer review process, only some discussed how they either bring explicit personal examples of this experience into their teaching or that their general experience frames their teaching about peer review. Some interview participants discussed how their lack of negative experiences with peer review make it hard to bring their personal experience with peer review into their instruction while others questioned if it would be valuable to bring these examples into the classroom.

DISCUSSION
Almost all survey respondents teach about peer review in their information literacy instruction. As such, the topic of peer review is the domain of academic librarians to cover, whether because departmental faculty expect or request it be covered, because academic librarians see it as an important part of their instruction, or for other reasons. However, while academic librarians almost universally instruct on peer review, there are no best practices for how to talk about and contextualize peer review. Documents like the Framework can loosely inform this instruction, but do not take the place of more tailored guiding documents.
Critically examining the peer review process exposes numerous inequalities in academia, such as gender and racial biases (Resch, Ernst & Garrow 2000;Lee, Sugimoto, Zhang, & Cronin, 2013;Roh, 2016), publishers' exploitation of academics' labor (Bergstrom, 2001;Lehner & Ziegler, 2019;Williams, 2020), and the impeding of timely sharing of information (Mrowinski, Fronczak, Fronczak, Medic, & Ausloos, 2016). Thus, instruction on peer review provides an excellent entry point to discuss inequality in academia with students. However, just slightly more than one in three respondents bring in explicit critiques of peer review in their instruction, meaning most academic librarians are not leveraging this opportunity to engage students in critical inquiry about a significant player in scholarly communication. This lack of context and critique continues to hold up traditional peer review as an unfettered good in academia.
Contextualizing peer review is often seen as important primarily for higher level students, with lower-level information literacy instruction focused more on basic research and evaluation concepts. In an ideal world, academic librarians will be able to scaffold their instruction on peer review to work up to discussing issues of power, privilege, and bias in academic publishing, however this is only possible when academic librarians have sustained access to students throughout a student's entire college trajectory. The authors both work at a state university with a large retention gap between students' first and second years at the university, and it brings up the question of whether introducing these concepts earlier in a student's studies can help contextualize the imperfections in the academic literature they are often asked to read, and provide a better critical lens for lifelong learning regardless of whether the students return to their studies at a later date. Thus, information literacy programs must re-consider the scaffolding of peer review instruction and think of ways to introduce it in lower-division courses.
Academic librarians are overwhelmingly engaged in the peer review process, with experience in roles such as author, reviewer, and editor. This personal experience, combined with personal stories shared from colleagues, often influences how academic librarians think about peer review, but does not consistently influence how academic librarians teach about peer review. Bringing in these personal stories to illustrate flaws in the system, such as efficacy, timeliness, bias, and reliability of traditional peer review will help to better contextualize academic publishing. This is particularly important given that students are often told that peer review is the "gold standard" in academic publishing, without a critical analysis of the issues and power at play in the traditional peer review process.
Using these examples may also help students better understand the need for critical evaluation of their sources, including peer reviewed literature. Furthermore, this disconnect between personal experiences and pedagogy may help to explain why traditional peer review is held in such a high regard in librarianship, and undermine attempts to change to alternative forms of peer review that are more open, transparent, and collaborative.

STUDY LIMITATIONS
The survey was open to academic librarians in the United States and Canada, however there was a low response rate by Canadian librarians, perhaps because of the listservs it was distributed on. It would be beneficial to do a subsequent study focused on peer review instruction by academic librarians in Canada. When designing the survey, the authors did not include whether personal involvement in the peer review process influences academic librarians' teaching; however, this emerged as a question of interest based on the findings from the survey and was explored in the subsequent interviews with study participants. Had the authors included participation in the peer review process as a scale to rank in the survey would have provided interesting insight. While the authors gathered data on academic librarians' use and preference for alternative forms of peer review, it was not the intended focus of this study; further research in this area is warranted.

CONCLUSIONS
While academic librarians are overwhelmingly responsible for teaching about peer review, there are varying degrees of practice for whether and how peer review is contextualized. External constraints such as instructor demands, time, and student level influence academic librarians' decisions on what and how much to cover in the classroom about peer review. Academic librarians employ strategies to make the complexities of the peer review process more relatable to students, by finding parallel examples that connect to students' lived experiences, as well as creative activities that illustrate the process. Still, although many academic librarians are involved in the various stages of the peer review process, there appears to be a disconnect between the personal experience and the pedagogy around peer review. Following Jacobs (2018), who asserts that, "as a form of literacy, information literacy also operates within a sociopolitical context and is thus politically charged" (p. 258), the authors argue that academic librarians can apply reflective pedagogical praxis to support students' critical engagement with, or cultivating a practice of self-reflection about how what one knows, what one teaches, and the space one holds for student learning are interrelated, can help academic librarians integrate their lived experience of peer review with what they say about it, thus helping academic librarians to support students' critical engagement with scholarly discourse.
This study is the first to consider how academic librarians' understanding of and participation in the peer review process influences their information literacy pedagogy and practice. Future studies are encouraged to continue to look at how academic librarians contextualize scholarly communication processes in their instruction. While not the intention of this study, the authors welcome the development of best practices on teaching about peer review and other related scholarly communication topics to college students. Finally, the authors hope that this study will encourage academic librarians to more intentionally incorporate their personal experiences with the peer review process in their instruction about it.

APPENDIX A Information Literacy & Peer Review Survey
Implied Consent to Participate in Research: Data collected from this confidential survey will be used for a study about how academic librarians in Canada and the United States think about and teach peer review.
It is anticipated that the survey questions will take about 15-20 minutes to complete. You have been invited to participate because you are an academic librarian with teaching responsibilities working in Canada or the United States.
You must be 18 years of age or older to participate.
There are no risks or benefits to you in participating in this survey. You may choose to participate or not. You may answer only the questions you feel comfortable answering, and you may stop at any time. If you do not wish to participate, you may simply return the blank survey, with no penalty to yourself. Please do not put your name in the survey.  (2) □ 3rd year or higher undergraduates (3) □ Master's level or Credential graduate students (4) □ Doctoral level graduate students (5) □ Other (Please explain): (6) ____________________ Q4 Have you ever done any of the following peer review related activities? Check all that apply. □ Authored a journal article or book chapter in a peer reviewed publication (1) □ Been a reviewer (2) □ Edited a peer reviewed publication (3) □ Other peer reviewed activity (Please explain): (4) ____________________

Belief Attitude / Belief Questions
Q5 What is your working definition of peer review?
Q6 What do you think the function of peer review is?
Q7 Please read the following definition and answer the corresponding question: Traditional peer review is a process whereby editors of publications send submitted scholarly work out to other scholars in the field for feedback. These experts scrutinize the scholarly work, research, or ideas, and ultimately provide guidance on whether the work should be published by the journal. Traditional peer review can be double-blind or single-blind (where the reviewers know the identity of the author, but the author does not know the identities of the reviewers). (Definition based on http://blogs.scientificamerican.com/information-culture/ the-birth-of-modern-peer-review/ and https://en.wikipedia.org/wiki/Peer_review)Based on the information about peer review contained in this definition and in your personal working definition, how strongly do you agree with the following statement: Traditional peer review is valuable in the scholarly knowledge production process. □ Strongly agree (1) □ Agree (2) □ Somewhat agree (3) □ Neither agree nor disagree (4) □ Somewhat disagree (5) □ Disagree (6) □ Strongly disagree (7) □ Comments: (8) ____________________ Q8 Do you think any of the following are better alternatives to traditional peer review? Check all that apply. □ Open peer review Emphasizes transparency in the peer review process; "Sometimes this involves publicly naming reviewers and/or editors. Other journals publish some or all reviewer comments" (http://blog.f1000research.com/2014/05/21/what-isopen-peer-review/). (1) □ Post-publication peer review Reviews by formally invited or volunteer reviewers or feedback from commenters after the piece has been published (http://blog.f1000research.com/2014/07/08/what-is-post-publication-peer-review/). (2) □ Collaborative peer review Reviewers interact with the pre-published document and with each other using collaborative editing software (https://www.elsevier.com/reviewers-update/story/innovation-in-publishing/experimenting-with-collaborativepeer-review) (3) □ Not sure (Comments optional): (5) ____________________ □ No (Comments optional): (6) (3) Screen Resolution (4) Flash Version (5) Java Support (6) User Agent (7)