Metric competencies for biomedical librarians: results of a survey developed by the EAHIL Evaluation and Metrics group

Introduction Bibliometric methodologies are of increasing importance for universities, research institutes, private sector companies, policymakers, and government administrators to assess research performance. For library and information specialists (LIS), bibliometrics has been a research focus for decades [1]; however, libraries have recently incorporated bibliometrics support as a standard service in research performance evaluation [2, 3]. With the development of the web 2.0 social media era and the rapid increase of open access journals, novel alternative analytic metrics, known as altmetrics, have emerged [4]. Altmetrics are not substitutes to traditional bibliometrics, but rather function as complementary additions to access both impact and influence of a research project, a particular researcher, or a group of researchers [5]. Similar to traditional bibliometrics, altmetrics could be organized with respect to their focus [6]. However, the measurements for impact and influence of a scientific contribution include more than just citations. Other items such as clicks and views, downloads, bookmarks, saves, mentions in blog spots, comments, reviews, attributions, likes, shares and tweets are also of value. Additionally, individual contributors have particular profiles, and their accounts are linked to their contributions using unique identifiers for the contributing authors’ IDs, and for the DOIs, URLs or PMIDs of their works. What about altmetrics and their incorporation in the health library routine? Are librarians prepared to accomplish these new challenges? What are their needs? To give an informed answer to those questions, we decided to explore how the health LIS view themselves in terms of knowledge and hands-on operations of bibliometrics and altmetrics, through a survey based on the competencies survey developed by Karen Rowlett [7] and developed by the EAHIL Evaluation and Metrics group. Abstract The library profession continues to evolve and respond to user demands with both tools and services that support instruction and research. These changes typically lead to a need for increased librarian understanding and training. One such example is seen in the growing interest surrounding bibliometrics, altmetrics, and personal identifiers. Each of these serve as indicators of impact, and are becoming increasingly important in research. To more clearly identify and measure the current prevalence of each in the librarian profession, the EAHIL Metrics Group developed and disseminated a survey designed to capture current demand, and identify potential knowledge gaps where training would be beneficial. This publication presents the results of the survey and discusses pathways to attaining and providing increased expertise.


Objectives
The main objective of this paper is to first determine the bibliometric competencies of health librarians and information specialists with basic metrics and resources. In addition, we also wanted to explore the feelings around librarians' needs, and difficulties acquiring these new competencies.

Methods
During the Dublin conference in 2017, which saw the birth of the SIG Metrics Group, we realized the need to better investigate the current inventory of librarian skills, knowledge, and use of the main traditional bibliometric databases and alternative metrics. With that focus, an online questionnaire was developed to assess the bibliometric skills and knowledge of health information specialists. Emphasis was given to the utilization of novel Altmetric tools, author identifiers, and citation databases. For the construction of the questionnaire we used a software called REDCap (Research Electronic Data as Capture), which is a secure web application for building and managing online surveys and databases. The reasons to choose REDCap include that it provides real time data access, as well as the ability to export the collected data in formats useful for the statistical elaborations. The construction of the survey required to simple and short, but comprehensive questions to obtain the requested data. We summarized 15 questions preceded by a brief in-troduction to the questionnaire. Initially, the first draft of the survey was shared with members of the Metric Group for comments and corrections. After the implementation of the suggestions, the questionnaire was reviewed by the President of EAHIL. The questions were divided into 4 sections: -Section 1 ( Figure 1): Professional data. This section collected information regarding the respondent's type of institution, the function carried out in the library, the country of provenance, the enrolment in EAHIL.  Metric competencies for biomedical librarians -Section 2 ( Figure 2): Understanding of metrics competencies. This was aimed at capturing the level of librarian knowledge with common citation databases, the traditional and alternative metrics, and the main tools for management of author identifiers.
-Section 3 ( Figure 3): Metrics in practice. This had the aim to explore the use of metrics in the working environment at the library level.
-Section 4 ( Figure 4): Training needs for librarians. The primary goal was to capture which specific areas librarians would like to see included in training programs, and how libraries could best support a librarian led bibliometric service.
The questionnaire was launched on the 13th of June 2018, until the 5th of July 2018, when an e-mail message was sent to all EAHIL members through a link straight from RedCap. Multiple answers were allowed in some question of section 3 and 4. Percentages are calculated on total of respondents.

Results
The first results from the questionnaire were presented during the meeting of the 2018 Metrics Special Interest Group in Cardiff. The questionnaire received 173 responses from 33 different countries; most of them were from Italy, Spain and the UK, but there representatives from all over Europe. 77% of the participants were EAHIL members; 23% of the individuals were not EAHIL registered, but employed in biomedical libraries who requested to take part in the questionnaire.
Regarding the professional categories, 35.8% of the respondents were working at a hospital, 46.8% at universities, 16.8% at research institutions, 1.2% at health technology assessment agencies, 2.9% at private companies, 1 person was a freelance professional (0.6%), and 6 were at other types of institutions (3.5%). In regards to the function in the library or the department in which the respondents were working, the majority indicated more than one competence. 66 working as director/coordinator (38.2%), 26 in document delivery (15.0%), 65 in teaching and reference (37.6%), 44 as bibliographic researcher, (25.4%), 46 in research and scholarly communication (mainly areas of metrics and open science) (26.6%), and 26 had other functions (15.0%). Please note that the percentages do not sum up to 100, as 13 librarians indicated that they work on both, university and hospital. The heart of the questionnaire was about the knowledge of the main metric indicators and databases and tools for finding metrics related information. Question 5 was "How would you rate your knowledge of the following metrics?" Undoubtedly the ISI journal impact factor (JIF) was the most commonly known metric: only 1.2% had no knowledge, 1.7% would know where to find it, 29.1% could find and explain what it is at basic level, and 68.0% could find and would be able to explain  Alicia Fátima Gómez, Valeria Scotti, Annalisa De Silvestri, Giuse Ardita, T. Derek Halling and Artemis Chaleplioglou strengths and weaknesses of this metric. Similar results for Citations. Only 0.6% had no knowledge about, 4.1% could find the data, 29.4% could find and explain what it is at basic level, and 65.9% could find and would be able to explain strengths and weaknesses. As for the H-index knowledge: 3.5% did not know it, 6.5% would be able to find it, 32.2% could find and explain what is at basic level, and 58.8% could find and would be able to explain it in depth. The less known metric was the Field Weighted Citation Impact (FWCI), which is a Snowball Metric that shows the ratio between the actual citations received by a publication and the average number of citations received by all other similar publications. 46.4% did not know that metric, 23.2% would know where to find it, 17.3 could find and explain it at a basic level, and only 13.1% would be able to explain its strengths and weaknesses. Regarding CiteScore, a metric similar to the JIF but from Scopus and based on a 3-year range: 26.6% showed no knowledge, 32.0% could find it, 28.4% explain it at a basic level, and only 13% would be able to explain it in depth. Concerning the performance of altmetrics, results were positive: 42.9% of the respondents could find and explain what it is at basic level, and 32.4% could also explain their strengths and weaknesses, only 7.6% did not know about it, and 17.1% would know where to find it. Question 6 was about the awareness of specific databases and tools for finding metrics related information. The results show that the best known are Web of Science, Journal Citation Reports, Scopus Journal Metrics, and Google Scholar, all in terms of basic and advanced use. On the contrary, the new citation tools were less known. For example Dimensions -a tool by Digital Science that provides access to research through grants, publications, citations, clinical studies and patents in one place -was not known by 60.5%, 21.0% did know but had not used it, 8.4% did use it at a basic level and only 3.0% used at advanced level. The software designed by Microsoft -Academic was also not well known; despite being a free public search engine for academic pub-lications and literature; 51.8% were not aware of it, 30.6% heard about it but did not used it, 5.9% could use it, 8.8% did use it and could show how to access at the basic level, and only 2.9% used advanced features and could show to others at an advanced level. There was a similar fate for LENS, a global resource that allows the search for academic publications and patents at once in an inclusive way; 77.6% were not aware, 17.6% were aware but without using it, 2.9% had use it, 1.8% have used and felt comfortable explaining about it at a basic level, and no one had advanced knowledge of it. With Publish or Perish, a tool to assist with academic publishing and the assessment of research and journal quality, as well as software to conduct citation analysis; 33.3% were not aware and 29.8% had heard but it but never used, and 36.9% did use it at a basic or advanced level. Regarding the use of alternative metrics, data revealed that their knowledge should still be explored. Starting with Altmetric.com, 38.2% did not know it, 29.4% knew it but never used, and 23.6% did used it at a basic level, and 8.8% shown an advance knowledge. Most widely known was Plum Analytics, recently purchased by Elsevier and included in Scopus, in that 25.3% were not aware, 40.6% had some knowledge but without using it, 24.2% used it at a basic level, and 5.9% were able to use and show to users advanced features. Question 7 was about tools for management of author identifiers. Researcher ID 1 , Scopus author ID, and Google Scholar Profile show a good knowledge both at a basic level, as well as at an advanced level (using them, explaining the benefits, and giving help about them). ORCID showed an excellent performance, reaching almost 100% of advanced knowledge (to set up the profile, explain the benefits, and give help about that). Section 3 was about metrics in practice, including questions around the contexts where librarians use metrics to advise users or processes, the metrics demanded by users, and the role the library plays in providing training to users or advising researchers or managers.
Question 8 wanted to probe in which context the li-Metric competencies for biomedical librarians brarians use metrics to advise users or processes. As Figure 5 shows, metrics play a central role in a biomedical library, and are used for several tasks.
In this scenario, 75.7% answered that the library or information service does play an active role in training researchers, with active participation of librarians. Question 10 explores which metrics are included in the training courses the library provides, the data of the respondents is striking: 64.9% give training on how to find journal IF, 57.6% on how to calculate the h-index, 32.5% how to use alternative metrics, 54.3% on how to create ORCID profiles, 37.7% on how to create Researcher IDs, 32.5% on how to create Scopus IDs, and 19.9% cover all of them. Additionally 74.3% replied that the library or department plays an active role in advising researchers or senior management. The final section of the questionnaire was dedicated to training needs for librarians. The need for courses emerges in all the aspects taken into consideration by the questionnaire, in particular with an in-depth analysis for alternative metrics (85.5%). Nevertheless, they also show a need for courses that increase basic knowledge of traditional metrics (75.9%), as well as for author identifiers (79.5%). The last question centers on understanding which additional resources would a library need to improve the support for metrics; 77.2% want training for librarians, 54.5% highlighted the need for tools and resources, and 55.1% would like more participation in meetings and conferences.

Discussion
Health LIS professionals have been facing many changes over the years as a result of technological advances, and their end-users' needs. We found the results of the survey quite interesting in that it was intended to detect bibliometric skills and knowledge of health librarians and information specialist (LIS), in order to outline the real situation of bibliometric issues among the EAHIL members. The responses obtained, which were based on a self-evaluation of individual abilities and needs, also gave us a clear image of how health LIS view themselves in terms of theoretical knowledge and practical use of bibliometrics and altmetrics, bringing to light some gaps and topics to be strengthened. A great number of survey participants indicated engagement in traditional library activities, while only a quarter of the participants reported involvement in research and scholarly communication, such as metrics, open science, and new technologies. The majority reported themselves able to understand, explain, and retrieve the traditional bibliometric indicators (IF, h-index, and citation counts), even if those skills were not part of their daily work, while others were less familiar with the more recent Snowball Metrics used in Scopus. A large percentage of participants reported to being able to find and explain altmetrics, at least at a basic level. A very high share of librarians declared their practical involvement in all support activities related to research evaluation and publication strategies for their users' benefit. However, those who feel to have only a basic level or any knowledge of bibliometric resources and platforms expressed their desire to increase their understanding of citation databases, as well as bibliometric and Altmetric indicators. Respondents also indicated the need of professional training for librarians, and of more tools and resources for their libraries. One critical issue to be addressed is the library strategic plan for budget distribution, which has to be redesigned over time, according to end-user changing needs.

Conclusions
The survey results and findings lead to the conclusion that there is an increasing interest among librarians in bibliometric issues and research evaluation. For basic users and for those who are less familiar with some of the platforms, there is an emerging desire to deepen the knowledge and understanding Alicia Fátima Gómez, Valeria Scotti, Annalisa De Silvestri, Giuse Ardita, T. Derek Halling and Artemis Chaleplioglou with the use of databases and metrics. The most immediate need regarding alternative metrics is additional knowledge about indicators, tools and their application. Clearly there is a demand for the involvement of librarians in all phases of Altmetric support for their users and administrators. The need to discuss and resolve these demands are becoming more evident, and coincide with the increased requests of more advanced training, and the desire for more frequent participation at meetings, workshops, and conferences. The vast majority of health LIS participants in this study recognize the need to incorporate the new metrics technologies into their daily library routine. Regarding the self-evaluation of their abilities and their needs for additional training, we observed a demand for more financial resources being devoted to continuing education courses, and opportunities for experienced librarians to mentor the less experienced ones across the European Health libraries. What better coach than a librarian who uses these tools on a daily basis, and can explain to colleagues how they work, potentials, flaws, and tricks?