THE IMPACT FACTOR AS A LEGITIMATOR OF THE SCIENTIFIC KNOWLEDGE PRODUCED: A REVIEW OF THE LITERATURE

The impact factor (IF) is perceived by all researchers and scientists as the main criterion in their publication strategies, in order to respond to the demands of the academic reward system that defines career progression and the allocation of funds for research. Through a review of published literature we aim to explore different limits and ranges associated with impact factor, according to academia. We also try to understand in which way the state of the art is evolving, as well as the features that point to ambiguous criteria inherent to the construct on which impact factor was established as the legitimising criteria of scientific knowledge production.


Introduction
At a time when the main challenges to the Open Science model (Priem, 2012;Bare, 2014;Quintanilha, 2015;Berghmans, 2015) are being discussed, or the derivation of a phenomenon of scientific res publica (Cardoso et al, 2009) for skewing logics of the Open Access movement from its appropriation by the parallel and parasitic industry (Quintanilha, 2015), it is important to reflect on one of the crucial points that are part of this derivation.
Within a framework of "certification independent of the means of registration" (Borges, 2006: 72), where belonging to the main indexers (Thomson Reuters, Scopus) relates to the most important criteria for accreditation of scientific journals, impact factor (IF) (Garfield, 1955;Borges, 2006;Johnstone, 2007;Saarela, 2016;Seglen, 1997;Greenwood, 2007) -appropriately named as factor of influence by Sygocki and Korzeniewska (2018) -is assumed as the great criterion defining the panorama of the worldwide scientific publication. Other authors, such as Muller (2018), call it the tyranny of metrics, whereas Garfield (2006), five decades after the concept was created, uses the neologisms "scienciometry" and "journalology" to define the time. Impact factor is a sort of filter that separates almost indiscriminately the relevant knowledge produced from the rest. Thus, it constitutes itself as the great and current legitimating factor of the scientific knowledge produced at the same time and, directly or indirectly, contributes to the imprisonment of the Open Science model. Thomson Reuters/Web of Science/Clarivate Analytics -through its journal citation reports -and Scopus Elsevier -through its SJR (SCImago Journal Rank) and SNIP (Source Normalised per Paper) -control the great metrics associated with the publication of scientific knowledge. Due to the way they were established, these metrics end up contributing to factors of higher impact and influence 1 The translation of this article was funded by national funds through FCT -Fundação para a Ciência e a Tecnologia -as part of OBSERVARE project with the reference UID/CPO/04155/2013, with the aim of publishing Janus.net. Text translated by Thomas Rickard. JANUS.NET, e-journal of International Relations e-ISSN: 1647-7251 Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32- 44 The impact factor as a legitimator of the scientific knowledge produced: a review of the literature Tiago Lima Quintanilha, Gustavo Cardoso in scientific journals which are mostly of Anglophone origin and controlled by the major publishers, such as Sage, Wiley, Taylor & Francis, Routledge, etc. These publishers impose pay-per-query dynamics and help to pervert and strongly compromise the idea of democratising a more inclusive Open Science model, forcing an almost tacit acceptance of legitimating structures in a large number of highly requested journals which antagonise the Open Science model due to their close access and how they operate. An article in Science Magazine 2 considers these as exorbitant amounts applied by the major publishers, which own the scientific journals with the greatest impact factor, a sort of disruptive body to the continuity of the Open Access model in the sense that they create tremendous pressure on the future of academic publishing.
Impact factor, depending to a large extent on the number of citations generated by an article or publication, is determined beforehand by the hegemonic position of the Anglosphere and the holders of greater symbolic capital in the academia -an adaptation of Bourdieu's (1994) concept. In other words, an article published in English will have, for example, a greater possibility of being cited, in a context of cumulative scientific construction (Quintanilha, 2015), when linking texts with other texts, blocks of signification and units of reading or lexia (Barthes, 1972).
Similarly, Saarela (2016: 699) noted that the top journals or core journals of large disciplines typically have more citations, embodied in larger impact factors, compared to the ones of smaller disciplines (Saarela, 2016). Whilst Howard (2009) calls them the "Alist" of academic publication, Adler, Ewing and Taylor (2009) remind us that in some scientific fields, such as biomedical sciences, most articles are often cited shortly after their publication; and in other disciplines, such as math, most citations begin to emerge two years after the articles are published. Some institutions, such as the University of Western Australia 3 , warn of the fact that the metrics could be beneficial for validating knowledge produced by journals in natural sciences, medicine and social sciences, and totally irrelevant in the arts and humanities.
On the other hand, entry in this reading system of the impact factors is only possible, therefore, by the aggregation of the journals to the main global indexes, some of them controlled by the biggest publishers in the world -such as the case of Scopus, owned by Elsevier. Thus, the process that defines the impact factor of a publication depends, in the first instance, on the entry and association of it with the two main world indexes (Thomson Reuters' Web of Science and Elsevier's Scopus). The most evident sign of this bias associated with the organisations that have the main impact factors (i.e. Reuters -Web of Science/Clarity Analytics -Journal Citation Reports) is given when the journals themselves apply to the main indexers. One of the great criteria used by the Web of Science, for example, in accepting journals is the publication of articles exclusively in English, to the detriment of the quality that may be associated with articles published by the proponent journal. The reason seems obvious: articles exclusively published in English have a superior scope and potential for citation than an article published, for example in Portuguese, and are restricted to the speakers of this language.
This controversial process, which regulates the panorama of the world scientific The impact factor as a legitimator of the scientific knowledge produced: a review of the literature Tiago Lima Quintanilha, Gustavo Cardoso publication, inaugurates a certain institutionalised way of looking at a scientific journal based on its legitimation or delegitimation according to the impact factor attributed to it. Therefore, it is important to reflect on the construction of the concept itself, observing the criteria for the evolution and redefinition of the dimensions that have been on the basis of the construction of this concept in systems of academic reward, which significantly contributes to access to entry exams and progression in teaching and research careers. It is also fundamental to think about how these criteria have been decisive for the granting of postdoc fellowships and, to a certain extent, for PhD fellowships as well as for the financing of research projects based on the merit of the candidates and their research centres to which they are associated. Currently, impact factors have increasingly played a crucial role in the allocation of resources for scientific research purposes (Saarela, 2016) in many countries and constitute the primary criterion for academics' prestige and survival (Pirmez, Brandão & Momen, 2016: 543).

Framework of the concept
Maria Manuela Borges (2006) introduces the historical roots of the term "impact factor", first used by Eugene Garfield (1955). This quantitative concept "seeks to express the intellectual influence or the contribution of a work in a given area of knowledge", and "other functions can be added to these ones, such as the increasing degree of communication between authors or bringing up for them all the references related to their work (Borges, 2006: 55).
Others, such as Seglen (1997), use the term "citation rate" as a complement to the term impact factor in order to define the average of citations contained in a given journal and in a certain period of time. Impact factor is usually defined by calculating the number of citations in a given year in relation to the number of items published in that year (Seglen, 1997). Following the same reasoning, Greenwood (2007) states that "impact factor is thus calculated as the number of citations a journal has received in the last complete year for articles published in the two preceding years, divided by the total number of articles the journal published in the two preceding years". According to the author, this calculation expresses the average number of citations of published articles, without discriminating positively or negatively the larger or more frequently published journals (Greenwood, 2007: 1).
Traditionally, impact factor was used to determine the most important sources of knowledge to be acquired by universities (Saarela, 2016). Currently, impact factors are carefully consulted by researchers from around the world, who thus define the scientific journals to which they submit their articles (Greenwood, 2007). Johnstone (2007) argues that impact factor is being used as an absolute measure to define the quality of scientific journals. In addition, she points out that impact factor is also increasingly used as a tool to measure the academic performance of researchers and to evaluate the feasibility of departments and universities (Johnstone, 2007).
For Garfield (1955), the calculation of impact factor was done based on the number of citations received by the scientific journals in a period of two years. "If one intends, for example, to calculate the impact factor of a journal in 2005, one will use the data   It should be noted that, parallel to the two-year criterion, there is also the five-year impact factor, which is identical to the two-year impact factor, but with a naturally greater time interval, allowing us to observe much subtler variations in citation counts.

The cumulative construction of the term
The enormous plasticity of meanings attributed to the universe of the concepts involved in the phenomenon of open science collaborates for some parallel definitions, which also determine variations within the interpretation of the impact factor itself, giving it subdimensions that, together, allow us to look to the concept in a more maximalist and structured way, as if it were layered.
As an example, Bauer and Bakkalbasi (2005) introduce two concepts: co-citation analysis, which is the number of times two documents are cited simultaneously in later publications; and bibliographic coupling, which consists in a forecast that two articles that cite a previous work may have something in common (Borges, 2005: 55).
Other secondary indicators, such as the "h-index", for example, already quantify scientific productivity and the apparent scientific impact of a scientist or researcher through the number of citations that their cited articles reach. "The h-index measures the total number of papers a scientist has authored and the number of citations those papers have received, may be more acceptable to some. If, over a lifetime of a research career, you have authored 50 papers that have been cited 50 or more times, your h-index equals 50" (Kupiec-Weglinski, 2015: 482). In this way, the h-index can more accurately measure the micro side associated with individual scientific production.
The "g-factor" quantifies, on the other hand, the scientific productivity of a researcher and its calculation is based on the distribution of the citations received in the publications of this researcher. The g-factor was firstly developed to respond to the underrepresentation of European scientific journals in the bibliographic databases of Thomson ISI. The "y-factor" results from a simple combination of the impact factor available on the ISI databases and the weighted "page rank" in order to consider and adjust the impact factor according to the greater or lesser popularity of scientific journals (Satyanarayama, 2008).
Thomson also publishes the influence index of the article and the immediacy index, which appears as a measure of the time (speed) that elapses between the moment a given content is acquired and the moment of its reference/citation.
Eugene Garfield (2006) acknowledges that the creation of the impact factor in 1955 was based on the need to select additional journals and sources of research. With the legitimacy conferred upon him as the creator of the term impact factor, Garfield argues that the "term 'impact factor' has gradually evolved to describe both journal and author impact" (Garfield, 2006: 1). He also states the impact factor of scientific journals usually involves large amounts of articles and citations, and individual authors generally produce fewer articles, although some of them have published an admirable number. Garfield gives the example of a transplant surgeon named Tom Starzl, author of more than 2,000 scientific articles; and Carl Djerassi, who invented oral contraceptives and published more than 1,300 articles.

The ineffectiveness of the indicator and its repercussion in the skewing of the evaluation of the scientific knowledge produced
One of the problems associated with impact factor has to do with a kind of appropriation of the indicator by major indexers, such as Scopus 4 or ISI Web of Science, resulting in evidence of the most prominent journals indexed in them, to the detriment of others, or even of other documents, such as monographs and dissertations (Borges, 2006). This evidence leads us to a snowball effect with repercussions on the motivations and strategies of researchers and scientists whose annual evaluation and career progression depends to a large extent on the volume of publications and their impact factor.
New researchers need to publish in the most prestigious scientific journals as well as senior and associate researchers, who need to do so to maintain their fellowships and/or research funding or to progress in teaching careers. "The researchers submit the manuscripts to prestigious journals" (Borges, 2006: 275), entering the databases of the great world indexes, which institutionalise the calculation of the impact factor. "In the case of the scientific article, recognition of its quality is linked to the journal in which it is published" (Borges, 2006: 36). It depended previously on the editorial board and today depends on the impact factors of core journals.
However, "Ever since its discovery in 1960s, impact factor continues to be in the news, often for the wrong reasons" (Satyanarayama & Sharma, 2008: 4). Some authors point out that there are significant dangers and limitations associated with the calculation of impact factors, such as the lack of quality of citations and the existence of journal selfcitation (Archambault & Larivière 2009;Falagas et al, 2008;Vanclay, 2012) -scientific journals that tend to value proposals for the publication of articles that refer to other articles published by them -as well as the informal combinations of researchers who take the modus operandi of citing each other, raising the impact factor of their articles.
For Seglen (1997), assessing scientific quality without a universal solution is an extremely complicated task. The author argues that, ideally, all processes to validate scientific knowledge produced should be defined by academic certification based on the scrutiny and verification of real experts in a given area. In practice, the so-called "peer-review", usually carried out by committees that have too generalist competences, conditions everything else, from the simplistic verification of the knowledge produced to the processes that legitimise journals (Seglen, 1997). This author, who is a critic of the model of legitimating scientific knowledge produced based on impact factors, argues that this impact factor, or citation rate, does not represent individual scientific production as it does not define its quality.
The author lists the problems at the basis of impact factors: 1) the impact factors of scientific journals are determined by technical procedures that are not related to the production of scientific knowledge per se; 2) the impact factors of scientific journals depends on the research area. Higher impact factors are usually associated with scientific journals that cover vast areas of exploratory research, fast-growing and short-lived literature that typically involves several references per article; 3) the fact that the citation rates of scientific journals determine the impact factor of the publication, and not the other way around; 4) citations of non-citable items are usually included in the same databases; 5) review or survey articles are strongly cited and inflate the impact factors of some scientific journals; 6) the larger articles result in higher citation levels, also inflating the impact factors of journals; 7) printed works (i.e. books) are not included in the databases as sources of citation; 8) the bibliographic databases are oriented to the English language and dominated by US publications; 9) the impact factors depend on the dynamics (expansion or contraction) of the different academic areas; 10) small areas of academic research tend to have fewer journals with higher impact factors; 11) the relationship between research fields also determines the impact factors of scientific journals (i.e. differentiated fields interconnected in the area of health compared to smaller fields of research); 12) the limitations of databases or the example given by the Science Citation Index, which covers a limited number of scientific journals around the world (Seglen, 1997). Ultimately, Seglen (1992) points out that it is the large variability in citation processes that renders the criterion of impact factor little accurate, which means that it should not be used to evaluate scientific production.
The article "Hate journal impact factors? New study gives you one more reason", 5 written by John Bohannon (2016) and published in Science Magazine, said that scientists have a love-hate relationship with the impact factor of scientific journals. This measure, used to classify scientific journal prestige, is seen by many as a destroyer of the scientific community.
Regarding the need to rethink all metrics and impact factors as legitimators of the knowledge produced, authors such as Adler, Ewing and Taylor (2009) suggest new multiple criteria to validate these metrics separately for each discipline, whilst being calibrated and adjusted according to the specificity of each discipline, as well as according to the attributes of each classification. In other words, metrics and impact factors should be as diverse and rich as possible.
Adler, Ewing and Taylor (2009), with a fundamentally optimistic view of the Open Science model, particularly due to its ability to provide greater variability of metrics that is contrary to structures of manipulation and misuse of themselves -possibly because, at the date of the respective article, they had not contemplated yet some bias of the model itself 6 -list some reasons, close to Seglen's model (1997), that explain their opposition to the panorama of metrics as a criterion of scientific validation: 1) the meaning of a citation may be even more subjective than the peer-review itself; 2) unique trust in citation metrics provides, at best, an incomplete and superficial understanding of research; 3) the fact that the validity of statistics for the impact factor and h-index are not well known or even well studied; 4) the fact that the citation metrics provide only a limited and incomplete view of the quality of research, and the statistics resulting from these metrics can be misunderstood and misused; 5) the possibility that exclusive trust in citation-based metrics may replace the criterion of subjective peer-review as the preponderant element to validate research; 6) the idea that impact factor cannot be used to compare scientific journals between disciplines; 7) the certainty that impact factor does not accurately reflect the range of citations in some disciplines, as not all scientific journals are indexed; and 8) the possibility that impact factor can be easily influenced by the (high) frequency with which some authors are wrongly identified.
JANUS.NET, e-journal of International Relations e-ISSN: 1647-7251 Vol. 9, Nº. 2 (November 2018-April 2019), pp. 32-44 The impact factor as a legitimator of the scientific knowledge produced: a review of the literature Tiago Lima Quintanilha, Gustavo Cardoso Recurring to Moed, Borges (2006) points out other limitations, such as: "when measuring the impact of citation in the second or third year after its publication, may be there is a biased towards magazines that have a faster IF maturation or decline".
One of the reasons given for the uselessness of the impact factors is that they have no validity as a predictive measure, since they are vaguely calculated. Paulus, Cruz and Krach (2018) attempted to illustrate the fallacies inherent in the use of metrics to evaluate journals and scientists' works. For the authors, the simple fact that scientific quality is judged by the impact factors of the journals shows that we are being driven by weak and invalid arguments. And the uncertainty regarding the quality of a work is exceeded by its impact factor rather than the quality of the work itself.
Kupiec-Weglinski (2015) argues, in turn, that impact factors do not reflect the quality or reliability of science nor the scientist's ability, valences and creativity. In addition, the author says that the most important thing is to publish quality and innovative knowledge whilst maintaining a continuous record of publications and good productivity. "The bottom line is, you need to publish research that is reproducible, impactful and moves your field forward" (Kupiec-Weglinski, 2015: 482).
For Satyanarayama (2008), impact factors can even skew the direction of scientific investigation, as the scientists themselves tend to direct their investigation to mainstream areas that are more easily financed. On the other hand, those researchers who are dedicated to less-mainstream areas, although important, have greater difficulty in getting funds for research and being recognised. "It is well known that funding follows what is considered 'significant' in science, which is usually science driven by citation hype and publications in high impact journals" (Satyanarayama, 2008: 4). Towpik (2015: 465), in contrast, points out a mania associated with the impact factor that persists and inflicts a pernicious effect on science and on some scientific conducts and practices.
Weglinski (2015) goes further and argues that high impact factors are often mere conveniences that money can afford. The higher the impact factor of a scientific journal, the higher the costs of publication addressed to the authors who decide to publish in that same scientific journal and/or to the institutions/researchers that intend to acquire these publications. As a consequence, the entire democratising root of the open science movement is also perverted, with its own impact factor contributing to this scenario. Moustafa (2015) considers that impact factor has become the worst enemy of scientific quality, placing great pressure on authors, editors, stakeholders and funders. Worst of all, in a significant number of countries, the allocation of budget and government funds is entirely channeled into the only journals with so-called high impact factors, leaving everyone else out.
Referring to the case of research in the area of health, more precisely in nursing, Johnstone (2007) argues that this obsession with impact factors endangers the sustainability and viability of scientific journals in nursing and its academic texts. In nursing, the author explains, researchers abandon their publishing agendas in order to publish only in elite scientific journals, some of them outside the area of nursing. The author argues that other ways of assessing the quality and impact of scientific journals JANUS.NET, e-journal of International Relations e-ISSN: 1647-7251 Vol. 9, Nº. 2 (November 2018-April 2019 The impact factor as a legitimator of the scientific knowledge produced: a review of the literature Tiago Lima Quintanilha, Gustavo Cardoso in nursing should be planned, and that books and book chapters should also begin to be included in the metrics. Ironside (2007) is peremptory in stating that impact factors are not useful and should be abolished, although they may provide some useful information for the review process if used in a judicious and conscientious manner.
Finally, it should be noted that the arguments that give a more stated defence of impact factor focus, above all, on 1) a kind of tacit acceptance of metrics by researchers and academics, and 2) the defence of initial reductionist thinking of Eugene Garfield who conceived the impact factor as a way to evaluate science and scientific journals per se. Some authors, such as Oppenheim (2008), recognising the importance of metrics, consider that impact factor is strongly correlated with other scientific quality criteria, such as peer-review. Oppenheim (2008) cannot foresee, for example, that impact factor can easily replace the criterion of the peer-review in the structures that validate the knowledge produced. Essentially, because the second is mostly a closed and silent criterion, it is restricted to the author-mediator-reviewer relationship and not available for consultation, which makes it vulnerable to the easy accessibility and enormous reach of the impact factor as a validation criterion of knowledge produced, as considered by most of the academic structure. For authors who celebrate the existence of metrics and impact factors as legitimators of knowledge produced, articles of proven quality are necessarily the most cited, since citations and other metrics correlate strongly with the most subjective side of the peer-review.
On the other hand, Kampourakis (2018) points out that the most beneficial side of the impact factor, albeit indirectly, is related to the very idea of dissemination and running of the knowledge produced because of two reasons. First, researchers and scientists have realised that the greater the dissemination of their work (i.e. ResearchGate), for example, amongst their networks, the greater the likelihood that they could be cited -besides generating greater scrutiny and verification of the knowledge produced. Second, the journals that reach higher impact factors end up having an acknowledgment that, in the short-term, allows them to benefit from a larger number of articles for publication, managing their needs from this bigger influx of articles.
Publications often tend to celebrate (with notifications to their subscribers) and to congratulate themselves 78 when entering major indexers and consequently accessing impact factors (i.e. Scopus -Scimago Journal & Country Rankings), since they perceive that being part of this system is, in the first instance, a crucial criterion for their validation and accreditation in the academic community in a much more decisive way than the degree of quality that may be associated with the material published. This is because, ultimately, impact factor is today understood as the first criterion to define the value of a scientific journal and published articles.

Final considerations
In general terms, the discussion in the scientific community about whether or not impact factors are reliable in measuring knowledge produced results in a theme that has been Although academic debate focuses less on the tacit aspect of the analysis and reading of the values of the impact factors and more on the contradictions and problems associated with the phenomenon -with repercussion on the skew of the processes to legitimate the knowledge produced -there is still room for the use and discussion of the main ideas listed in this article.
An example of the importance of a broad reflection on the limits of the impact factor concept can be defined by the attempt to make it increasingly efficient by introducing new sub-dimensions that give it greater scope.
The g-factor, y-factor as well as h-index are good examples that aim to make the definition criteria less dependent on the characteristics described by academia as being detrimental and little recognisable for researchers' work and the activity of scientific journals that are not part of the so-called A-list of scientific publication (Howard, 2009).
Eugene Garfield (2006) suggests that impact factor accuracy is questionable and that citation studies should be adjusted to include variables such as peculiarity of the knowledge produced, citation density -average number of references cited by origin article -as well as the criterion of half-life (also from Thomson Reuters) or the number of years required to find 50% of the references cited.
In the future, it will be interesting to note: if the calculation of impact factors evolves into a more inclusive and parity model that values issues such as the scientific productivity of researchers and scientific journals, based on criteria to define scientific quality; or if the legitimacy of the knowledge produced will continue to be an extension of the tyranny of the metrics and the assumptions that are intrinsic to it. What is important to discuss -and what constitutes the aim of this article -is whether or not scientific openness can thrive in a world of publishing relations in which impact factors tend to perpetuate monopolies of scientific dissemination, resulting from the infinite replication of the status inherited by the previous publication of others and perpetuated by the repetition of practices that legitimised a given position in the ranking of impact factors. Therefore, should science produce aristocratic scientists and knowledge, based on the inheritance of those who already had a status before by publishing in a given periodical; or should it be based on merit and virtue, regardless of where it is published, and on the peer-review of the scientific knowledge produced? Science develops itself in both democratic and autocratic societies, but it also has a contribution for the strengthening of democracy. It is up to us, scientists and researchers, to decide what contribution we are ready to give, beyond knowledge produced, to the society that gives us context and conditions to investigate and publish.