Sweet Spots for Manuscripts: Visualizing the Quality of Scientific Production

This study presents a visual approach to analyze scientific activity in indexed journals based on bibliometric indicators. The result of this analysis approach supports the process of choosing a journal to submit a manuscript, especially in knowledge domains where the author, whether novice or not, has failed to follow in detail the evolution of the domain. Moreover, the same approach can be used to an overall assessment of the production of researchers, research groups, institutions and countries. The conceptual foundations of the approach are volume and citation indicators of scientific journals articles. In particular, the proposed approach is based on SCImago Journal Rank index, the h-index and the annual number of articles published. The above indicators are transformed into percentiles and then displayed on a graph, integrating in this diagram the set of articles to be analyzed. The analysis approach was implemented in Sweet Spots for Manuscripts, online software based on open source components. As an example of its potential, an analysis of the 2014 research activity of the institutions belonging to the Council of Rectors of Chilean Universities is presented. Finally, the practical implications and opportunities for future research are discussed.


INTRODUCTION
Information is the key element of research activity. It is the researcher who decides how to obtain the information, what to do with it, how to treat it and finally, how and to whom it is delivered (Sarabia, 1999). And although today the huge volume and the great facility of access to digital media scientific information generates a promising turn scenario, this presents difficulties for its valuation and synthesis in an efficient manner (Ramirez-Correa and García-Cruz, 2004). In particular, the significant rate of growth of new scientific journals (Larsen and von Ins, 2010), imposes a cardinal challenge for the researcher: What is the appropriate journal to disseminate research results? Answering this question is a key milestone in the scientific process (Kelner, 2007;Resta et al., 2010). On the other hand, in the last decade, there have been efforts to help evaluate and synthesize various domains of scientific knowledge. Knowledge Domain Visualization (KDViz) deals with the analysis and modeling of the structure and dynamics of a scientific domain (Chen and Hicks, 2004). While in KDViz, various techniques are used, such as exploratory data analysis, information retrieval and text mining (Chen, 2006), the knowledge domain visualization is traditionally based on citations (Kraker et al., 2015).
In this context, the aim of this study is to present a visual approach for analyzing scientific activity in indexed journals based on bibliometric indicators. The result of this analysis can be useful to support the choice of a scientific journal to submit a manuscript with research findings, especially in knowledge domains where the author is a novice. Moreover, the same approach can be used to obtain an overall assessment of the production of researchers, research groups, institutions or countries. The following section explains the conceptual basis of the analysis approach. The subsequent section introduces the computational program Sweet Spots for Manuscripts (SSfM). Then, the result of applying SSfM to the 2014 research activity in institutions belonging to the Council of Rectors of Chilean Universities is presented. And finally, the last section provides a discussion of the possible practical implications and opportunities for future studies.

METHODOLOGY
Conceptual basis of the analysis approach: Until just a couple of decades ago, the review of the scientific literature had a strong qualitative mark. It was work of the researcher, after analyzing scientific articles, to evaluate the state of the art of the topic addressed in the form of an essay (Sánchez-Meca, 1999). Today, with the huge amount of articles to which researchers has access through the Internet and the many databases available, it is necessary to distinguish relevant articles and sources for the researcher, incorporating increasingly quantitative elements for this task. Following this idea, the meta-analysis to address the generalization of partial results collected by several researches was developed. However, besides its main objective, the meta-analysis is an important tool to guide research on any topic (Arenas and García, 2005). This method has answered questions such as: • What sample sizes have been used • Characteristics of the samples • Most important authors • Journals that have been published the most • Temporal analysis of the occurrence of articles, etc On the other hand, different databases have incorporated features that facilitate some of this information within their browsers. The analysis approach that supports SSfM goes a step further in this line. Giving answers to researchers when analyzing the literature and designing their own publishing strategies.
Specifically, the proposed analysis approach is based on volume and citation indicators associated with scientific journals articles. Citations are explicit relationships between articles that have common aspects (Chandy and Williams, 1994), these relationships are essential to build knowledge in a particular discipline and are one of the foundations on which the scientific method is built (Knight et al., 2000).
These citations can be seen as social exchanges between authors and readers (Zinkhan et al., 1992) and therefore the citation to a particular article can be considered as an quality indicator of such work (Knight et al., 2000). The "valuable" articles will be cited and drive both new research and future publishing efforts. A journal that is most often cited is more used by the scientific community and thus will be more attractive to submit new manuscripts. If we do not consider other elements in our analysis, more manuscripts increase competition for publication space in that journal and associated with this, the quality of the journal will increase. Then, the number of citations can be considered a scientific journals quality indicator. In particular, the approach is based on two indicators related to citations, SCImago Journal Rank (SJR) index (González-Pereira et al., 2010) and h-index (Hirsch, 2005) and one associated with the volume, annual number of published articles. The SJR index is a scientific journal prestige indicator and is calculated through a network of citations to the journal in which the nodes represent the active journal sources and the directed links between nodes represent citation relationships among those journals (González-Pereira et al., 2010). On the other hand, the h-index was proposed to measure the impact of an author with the total number of publications and citations generated from them, specifically, it is defined as the largest number h such that h publications have received at least h citations each (Hirsch, 2005). The same way as for authors, this index can be calculated for scientific journals.
Operationalization of the analysis approach: SSfM software was built to implement the analysis approach. SSfM operates online and is based on Java, PHP and Postgre SQL (http://www.sweetspotsweb.com). The inputs to SSfM are exported files as results of searches in Elsevier's Scopus database (http://www.scopus. com). Scopus is a bibliographic database covering about 20,000 journals in the scientific, technical, medical and social sciences. In addition, SSfM is used for processing the data taken from the SCImago Journal and Country Rank site (http://www.scimagojr.com). SCImago Journal and Country Rank site annually updates bibliometric indicators for journals contained in Scopus.
Given the differences between disciplines in relation to the volume of publications in each journal and the number of citations to each article, in SSfM the indicators used in the analysis must be transformed into percentiles. Therefore, each journal is valued at three ranges of percentiles: Based on the input provided by the user and using the indicators in its own database, SSfM generates an integrated graphic as a visualization element (Fig. 1). In this graph, each circle represents a scientific journal into the data contained in the files that were provided by the user to the application. The color depends on the percentile of the journal regarding its h-index, the circle is red if less or equal to the 33%, it is yellow if it is between the 33 and 66% and it is green if it is higher than 66%. The position on the X axis of the circle is associated with the percentile of the SJR index by the journal. On the other hand, the position on the Y axis of the circle is associated with the percentile of the annual number of articles published in the journal. The size of each circle indicates the number of journal articles into the data files that were provided by the user.
Finally, SSfM incorporates filters to the graph associated with the area, category and the country of the scientific journals.

THE RESEARCH ACTIVITY OF THE INSTITUTIONS BELONGING TO THE COUNCIL OF RECTORS OF CHILEAN UNIVERSITIES
By way of example, the analysis approach implemented in SSfM is applied to the results of the 2014 research activity of institutions belonging to the Council of Rectors of Chilean Universities (CRUCH, Consejo de Rectores de las Universidades Chilenas). CRUCH is an organization created for the national coordination of academic activity of the 25 Chilean universities that comprise it, ensuring the quality and excellence of them. The institutions belonging to CRUCH directly generate over 87% of the research produced in Chile with international visibility (Ramírez and Alfaro, 2013). Below the results of this analysis are shown.
For purposes of facilitating the comparison of institutions with similar volumes of scientific production, the following procedure was performed. Based on production information of articles contained in SCImago Institutions Ranking (SIR), a global report that ranks more than 2000 of the best research institutions and following the idea of classification of Chilean Universities of Ramírez and Alfaro (2012), a hierarchical cluster analysis was performed. This analysis identified four groups; the first three groups are composed of universities within the SIR report (which are grouped by volume) and the fourth group consists of those institutions that are not in the report. For reasons of space in this study, the results for all institutions can be obtained on the site http://www. sweetspotsweb.com.
The first group consists of University of Chile, Pontifical Catholic University of Chile and University of Concepción. Figure 2 shows the result for the first university.
The second group consists of University of Santiago de Chile, Federico Santa María Technical University, Austral University of Chile, Pontifical Catholic University of Valparaíso, University of La Frontera, Catholic University of the North, University of Valparaiso and University of Talca. Figure 3 shows the result for the first university.
The third group consists of University of Tarapacá, University of the Bío Bío, University of Antofagasta and University of La Serena. Figure 4 shows the result for the first university in this group.
The fourth group consists of University of Playa Ancha, University of Atacama, University of Los Lagos, University of Magallanes, Catholic University of the Maule, Catholic University of the Most Holy Conception, Catholic University of Temuco, Metropolitan University of Educational Sciences, Metropolitan University of Technology and Arturo Prat University. Figure 5 shows the result for the first university.
If we review the results, globally we can say that the saturation in the graph decreases rapidly from Group 1 to the following groups, denoting the clear difference between this group and other institutions belonging to CRUCH. About Group 1, we note that there are many prestigious journals where these institutions publish their results, however, it is surprising that given the scientific orientation of the institutions in this group, there are numerous publications in journals of low prestige.
About Group 2, we can say that compared to Group 1, a clear decrease in publications in prestigious journals is seen. Furthermore, there is a greater dispersion of the prestige of the journals where research results are published. This structure may be associated with policies to encourage research that reward volume over quality. In relation to this, a difference between institutions is viewed within the group, for example; the Catholic University of the North has less volume than the Universidad de la Frontera, but clearly higher journals prestige. The same happens between Federico Santa María Technical University and the University of Santiago de Chile.
About Group 3, it can be noted that in addition to having fewer publications than Group 2, the prestige of the journals where research results are published decreases, in fact, except for the University of La Serena, the volume is associated with journals with an average or low prestige.
And finally, on Group 4 we can indicate that because of the low volume of publications, there is no clear pattern and while there are publications in highimpact journals, this fact is an odd element among a few publications in journals of lower prestige.

DISCUSSION
The main practical implications of this approach can be summarized in four points.
First, concentrating on a single image the result of the research activity is useful and an efficient way for the assessment and synthesis of a knowledge domain, especially when the image is produced by a computer system as SSfM, as it allows to operate with large volumes of data quickly. Consequently, the use of SSfM should be considered by any researcher in the process to decide where to send your manuscript. SSfM supports not only knowledge domains where the author is a novice, but is also relevant for already consolidated authors. This latter is because both the thematic and indicators of journals, as well the topics on publishing are changing and the authors do not always follow these developments in detail. Furthermore, new journals constantly appear while others disappear. Therefore, having an updated vision is invariably useful for the consolidated author.
Second, it is advisable to have chosen the journal you will send a manuscript to before completing it, i.e., during the writing process and is always appropriate to see if the topic of work is aligned to the editorial policy of the journal, if the journal appears in our cited references, the length of articles published, number of articles published per year, if they are methodological or more theoretical journals, etc. (Carbonell and Calvó, 2009). Therefore, the use of SSfM be done in the early stages of writing the manuscript.
Third, the approach can be used for a rapid and comprehensive research results evaluation of authors or institutions. This implies that organizations that need to assess their researchers may consider using SSfM as a management control tool.
And fourth, with this approach it is possible to observe patterns of behavior that contrast volume and quality of research results, thereby allowing to establish research strategies at individual, team and/or institutional levels.
As opportunities for future studies, we suggest three ideas. The first, in relation to the approach, it is possible to incorporate other bibliometric indicators such as citations per document and the rejection rate of each journal. The second and related to SSfM, it would be useful to incorporate a new feature that shows how a graph decomposes and evolves by year, within the time frame of the analysis. You might also consider searching in Scopus within SSfM and analyze not only journals but sub knowledge areas. And a third possible area of future studies is associated with the CRUCH example, it would be advisable to conduct a study that shows how institutions have progressed over the last three decades, a period where the volume of publications in Chile has tripled.