How to Select Better Topics and Design Better Bibliometrics and Scientometrics Studies: A Perspective

Copyright © The Author(s). 2021 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/ licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. Brady D Lund

This paper highlights some of the weaknesses of many bibliometrics and scientometrics studies published today and offers guidance on how to better select topics and analyses to perform in these studies. Bibliometricians should focus on topics with which they already are familiar or for which they have specialized knowledge. Analyses that are performed should be selected based on the knowledge that will be most relevant to the researchers in the subject areas in informing their work. Bibliometricians should always ensure that their work aligns with the stated aims and scope of the journal to which they are submitting and ensure that the manuscript is properly formatted according to the journal's specifications. By taking a more measured approach to the design of their studies, the quality and reputation of scientometrics research may be greatly improved on the global scale.
Scientometrics, defined as the quantitative study of scientific and scholarly literature, and bibliometrics, the quantitative study of literature, have gained substantial popularity over the past several decades. These studies are popular among many researchers, especially (but not exclusively) those within the discipline of library and information science. The journal Scientometrics is considered by many of these researchers to be the most prestigious in which to be published and national and regional journals -particularly in Southeast Asian countries like Iran, India, and Bangladesh -are full of scientometrics studies. Specialized journals, like Collnet Journal of Scientometrics and Information Management, Journal of Scientometric Research, and the Caspian Journal of Scientometrics are publications within these countries that are dedicated specifically to publishing scientometrics studies. Even Library Philosophy and Practice, a journal that is published in the United States but whose authors are largely from developing countries in the Global South, publishes dozens, if not hundreds, of scientometrics studies each year. By any measure, scientometrics is a major area of inquiry across academia.
Well-performed, systematic scientometric studies can be very useful to researchers within a topic area. With emerging research areas, in particular, scientometrics studies inform researchers about research trends, important researchers whose work should be studied, and appropriate publication venues for manuscripts related to the research area. However, there are many issues with much of the scientometrics studies published today. The quality can vary greatly, the relevance/importance of the findings is not always clear, and it is evident that the expertise of the researchers in the area being examined is questionable. This not only harms the quality of scientometrics studies, degrading the reputation of these studies in developed countries like the United States and United Kingdom (where scientometrics studies are often considered insignificant or less rigorous), but also harms the researchers' own reputations and their ability to have a substantial impact on scholarly discourse. In order to improve the quality of scientometrics studies published today, it is important that researchers pay better attention to the topics they select as the focus of their studies, the study's design, and the specific analyses that are performed. These choices must be informed by the knowledge and expertise of the researchers themselves, rather than selected at random (as is often the case today). In this opinion paper, I present a series of recommended guidelines for scientometrics researchers to follow when developing a study, with the objective of helping this field mature and gain greater respect on the global scale.

Selecting the Right Topic
Often, it is not clear why researchers select the topics they do for scientometrics studies. Why is an information systems instead have generalized librarians who are not qualified to conduct scientometrics studies on topics like health science and physics.
Part of the issue with topic selection is that some researchers, I believe, see "scientometrics" as the same type of "topic" as "social psychology" and "information behavior". However, scientometrics is better viewed as a collection of methods than a topic area. It is more similar to a "survey" or "experiment" than it is to the "information behavior" topic. Information science researchers would not conduct an experiment to examine whether a cancer treatment is effective -they do not have the expertise in cancer research -and they should have a similar approach to scientometrics studies in these areas. If one would not feel comfortable taking part in other, non-scientometrics studies in this area, then perhaps they should not be the person to conduct a scientometrics study in this area either. The fact that many non-experts perform scientometrics studies on topics about which they have very limited actual knowledge is largely responsible for the influx of low-quality scientometrics studies that have harmed the reputation of this area of research. Figure 1 is a diagram that illustrates the procedure I would recommend for selecting the topic of a scientometrics study. By discussing how one followed this procedure to select an appropriate topic for their study, it may demonstrate greater expertise and trust of them as the researcher in these studies and the importance or novelty of their study.

Designing a Scientometrics Study
A scientometrics approach should never be selected because it is "easiest." Often this seems to be the case with many of the low-quality articles -that the author selected this approach and topic because it was easy. In these cases, the drive to "publish or perish" has negatively impacted the quality of our scholarly efforts. In the long term, it may harm career prospects to have these low-quality studies published. Building a large resume of low-quality articles published in sub-par journals, while it may be acceptable for attaining tenure and promotion in some countries, would be considered a very poor resume by United States' standards. These low-tier journals are not considered to be serious journals by U.S. researchers, and even a journal like Scientometrics is considered less prestigious than several dozen other information science-centric journals like the Journal of the Association for Information Science and Technology, Journal of Information Science, and Journal of Documentation. It is possible to publish a scientometrics study in these prestigious journals, but researchers much first select topics that are relevant to these journals (information science topics, rather than random topics from other fields) and then design the study to actually cover content that is original and helpful to potential readers.
researcher, who has no background in medical sciences, performing a scientometrics analysis of cancer research? There is no way this researcher has sufficient knowledge of cancer research to really understand what they are analyzing. They can make a mapping of keywords in VosViewer, and point to themes, but they cannot possibly understand the significance of these findings -so how is it possible for the study to actual make any impact for cancer researchers? What, in theory, makes scientometrics studies valuable is that they can reveal underlying themes or patterns in the literature. They can tell readers something they did not already know about their field. A listing of topics and keywords, without any linkage of these lists to actual developments in the field (which requires specialized knowledge of the field) is worthless.
When selecting a topic for a scientometrics study, researchers should look to topics where they already have knowledge and experience. Perhaps, if one has personally experienced cancer or has advanced knowledge of the literature through a degree or coursework in this area, they would be qualified to conduct a study in this area. But one should not assume that they can simply dive into a research area that they know nothing about and conduct a useful study. Think about "do I have adequate knowledge in this area to put together a study that will actually be helpful to readers?" If the best one can do is make a list of the top authors, journals, and keywords, they are contributing very little that would be helpful to researchers, especially if the field/research area is already well-defined. Cancer researchers already know the top cancer researchers and top cancer journals. Perhaps, if there is a new category or type of cancer (as we saw with coronavirus research in 2020), then these studies are valuable, because the area of study is so new that it may not be clear to researchers who the top researchers and journals are. In general, though, these descriptive studies are just stating the obvious (what every expert in the field already knows). One must have enough knowledge to be able to use their findings to reveal something new in the discussion (emerging areas of research, new trends).
To that end, it is important when selecting a topic for one to choose a topic that aligns with their expertise, which in most cases will be within one's "native" discipline (for instance, "social psychology" for psychology and "information behavior" for information sciences). Only researchers with specialized knowledge of topic areas should conduct those studies. In some countries, this "specialized knowledge" may also be possessed by a subject, or "liaison," librarian, who works directly with subject matter faculty (e.g., physics, biology, literature) to find appropriate literature for collection development and reviews of literature. These subject librarians are required to become experts in the subject to which they liaise, which satisfies the specialized knowledge criteria. [1] However, many libraries in the Global South, unfortunately lack subject librarians and Often, when I read scientometrics studies, I am left wondering "why did these researchers select these particular analyses to perform?" Generally, there is no clear rationale other than that the researcher simply used every analysis they knew how to perform without asking whether the analysis actually adds value to the study. Lotka's law is an interesting theory, but it is not necessary to calculate it for a study of "Top Information Seeking Researchers." It is not actually something that the target readership (information seeking researchers) will care about. They want to know things like "what are important new topics emerging in the field that I could expand my research into?" or "what journals are most likely to accept papers on the topic of information needs?" When choosing what analyses to include in a scientometrics study, researchers should ask "what do researchers in this topic that I am analyzing want to know?" If the researcher cannot answer that question, then they probably do not have the expertise to conduct a scientometrics study on that topic. They likely do not want to know about whether certain scientometrics laws apply to the literature of the field (they are not themselves scientometrics researchers), they want to know things that directly relate to their work.
When working primarily with data, as scientometrics researchers do, it is easy to forget the human element behind who the readers are. If the topic of the paper is anything other than scientometrics itself (i.e., if the topic is "a scientometrics study of cancer research" or "information needs research"), then the readers are likely not bibliometricians themselves. Thus, the focus should not be on performing complex analyses that the readers will not understand. The focus should be on simplifying findings and explaining their significance to readers who lack expertise in scientometrics. It is helpful to look at some very early scientometrics studies to clarify this point. For instance, note how in Donohue [2] and Lawson, Kostrewski, and Oppenheim [3] much of the articles are dedicated to the identification of keywords/topic areas and top journals for the purpose of informing search terms for information retrieval and collection development. The art of explaining how the findings of a scientometrics study can or should be used is lost today and should be rediscovered.
Often, the easiest path is not the best one. Certainly, collecting data from a database like Web of Science and analyzing it on VosViewer and Excel is straightforward, but since when is "straightforward" the mark of good research? In his 1987 dissertation at Florida State University, Imad Al-Sabbagh performed a manual analysis of references included at the end of articles published in the Journal of the Association for Information Science and Technology, [4] an approach that was replicated by Lund in 2019 for the journal Information Technology and Libraries. [5] These analyses categorized each reference in the articles' reference lists based on the discipline of origin, with the aim of calculating the interdisciplinarity (citation to articles from other disciplines) of the journal. This type of study can take hundreds of hours, but it actually tells the reader something new about their discipline, and in a much more accurate way than if some type of automated analysis was performed. Speed should never receive favor over getting the best, most interesting and novel analyses and results. Again, the emphasis on quick studies has caused substantial harm to the reputation of scientometrics around the world.

Why Do You Love Scientometrics Studies?
One of the most important questions to ask oneself is "why am I doing this study?" Answers like "because scientometrics is what most professors in my department do," "because it is easy," or "because it is not time-consuming," are not particularly strong ones if one truly believes in research as a noble, scholastic pursuit rather than just some job that must be done. If one truly enjoys scientometrics studies, then they should perform them, and do them well, thinking always of their intended audience. But if they do them only because it is what others do, or because it is easy, then they should stop doing them. Poor quality work from an unmotivated researcher only harms the reputation of bibliometrics and scientometrics further and gives elites in the Global North more excuse to exclude researchers from the Global South in important, serious conversations about research opportunity and equity.

CONCLUSION
This paper presents factors that should be considered when selecting the topic and approach to a bibliometrics or scientometrics study. These factors, informed by my experience with and knowledge of scientometrics research, are outlined to help improve the quality of scientometrics research published today and re-establish the value of these types of studies on a global scale. By considering the rationale behind why they conduct scientometrics studies, and why they focus on specific topics and analyses, researchers may expand the impact of their own studies.
One of the most important things to remember when composing a scientometrics study is simply to understand the audience and the journal to which you plan to submit the manuscript. Always read several articles published within the journal first to determine whether your scientometrics study would be a good fit. The title of the journal alone is insufficient to determine the appropriateness of a manuscript. Further, always be sure to follow journal guidelines! Scientometrics study authors, in my editing and reviewing experience, are some of the biggest offenders in submitting manuscripts that are not properly formatted to the journal's specifications, because the authors simply submit their manuscript to a new journal following rejection without paying any attention to the new journal's style guidelines. This can be seen as a sign of disrespect to the journal that you are unwilling to take a few minutes to properly format the manuscript before submitting it, but still expect the editors and reviewers to dedicate hours reviewing it. If one follows the basic principles of selecting a topic relevant to their existing knowledge, performing appropriate analyses, choosing an appropriate journal, and following the author guidelines, their work stands a considerably better chance of being accepted for publication.