The first three years of the Journal of Global Health: Assessing the impact

The Journal of Global Health (JoGH) is three years old. To assess its impact, we analysed online access to JoGH's articles using PubMed Central and Google Analytics tools. Moreover, we tracked citations that JoGH received in 2013 using ISI Web of KnowledgeSM and Google Scholar® tools. The 66 items (articles, viewpoints and editorials) published between June 2011 and December 2013 were accessed more than 50 000 times during 2013, from more than 160 countries of the world. Seven among the 13 most accessed papers were focused on global, regional and national epidemiological estimates of important infectious diseases. JoGH articles published in 2011 and 2012 received 77 citations in Journal Citation Reports® (JCR)–indexed journals in 2013 to 24 original research articles, setting our first, unofficial impact factor at 3.208. In addition, JoGH received 11 citations during 2013 to its 12 original research papers published during 2013, resulting in an immediacy index of 0.917. The number of external, non–commissioned submissions that we consider to be of high quality is continuously increasing, leading to current JoGH's rejection rate of about 80%. The current citation analysis raises favourable expectations for the JoGH's overall impact on the global health community in future years.

viding funds needed to ensure journal's sustainability, which is why most journals arose from professional societies. The search for citations was done by painstaking browsing through heavy, voluminous books that resembled phone directories of large cities -month by month, author by author, article by article. That was the reality of scientific publishing and impact assessment that had remained fairly unchanged throughout most of the 20 th century.
The three JoGH' s Editors-in-Chief still remember those times very well, both as authors of manuscripts and as journal editors. However, we are also aware that researchers who started their careers in the 21 st century probably cannot even begin to comprehend those times. Article mailing charges could have cost researchers from low-income countries their week's salary, which they were willing to pay despite an uncertain outcome. This obstacle is now replaced in most journals with a faceless electronic submission and electronically generated e-mail replies. The model of financing through subscriptions has been largely replaced, too. Many journals charge access to research articles that they publish. The traditional, reader-pays publishing model is today challenged by the open-access movement. New open-access journals are being launched on an almost daily basis, offering free access to their content, but charging authors for the article processing and publishing costs. Cyberspace has entirely replaced print, and it is difficult for us to remember holding a printed issue of any journal in our hands and reading it, article-by-article, as it was done only a decade ago. Nowadays, PDF versions of individual articles are downloaded, searched using browsers, and stored somewhere in the computer for further reading.
In this sea of change that has completely transformed scientific publishing, there is still one surviving feature that seems fitter than ever: the almighty "impact factor". Although much criticized, terribly flawed in so many ways, calculated in a non-transparent way, generated by a single, now private, enterprise, with numerator and denominator often not containing comparable items, and having a long history of being manipulated to a greater or lesser extent -it still remains the single most effective advertisement for any scientific journal, dwarfing all others by a large margin [1][2][3]. Therefore, anyone serious about their scientific publishing effort -and we certainly aim to be -simply cannot afford to ignore it, no matter what we may think of it personally.
The impact factor (IF) was instantly and firmly accepted by the scientific community because is successfully reduced all information about a journal' s content to a single number. For the vast majority of journals, their IF ranges between 0 and 10: among 8471 journals included in the Journal Citation Reports ® Science Edition in 2012, which are themselves considered to represent a selection of the world's journals of the highest quality, 8312 (98.1%) had IF smaller than 10 [4]. This means that only a small minority of the most competitive journals have the IF greater than 10. The IF tells any interested researcher the average number of citations that the articles published by the journal over the previous two calendar years received in the current calendar year. Subsequently, this implies that any articles that attracted 10 or more citations in any calendar year generated a substantial interest in the research community.
There have been developments in recent years that promise to at least provide some validation for the calculated impact factors, if not actually offering a viable alternative. First, the ISI Web of Knowledge SM Journal Citation Reports ® (JCR) by Thomson Reuters publishing corporation provides impact factors based both on 2-year and 5-year content follow-up, which prevents and exposes manipulation of the original 2-year metric [4]. Moreover, they also provide the Eigenfactor ® score and Article Influence ® score. The Eigenfactor ® score calculation is based on the number of times that the articles from the journal, that were published in the previous five years, have been cited in the current year, but it also takes into account the quality of the journals that have contributed these citations (ie, citation in a journal with a higher impact factor will influence this score more than one in a journal with a lower impact factor) and it removes a journal's self-citations [4]. The Article Influence ® score determines the average influence of a journal's articles over the first five years after publication. It is calculated by dividing a journal' s Eigenfactor ® score by the number of articles in the journal, normalized as a fraction of all articles in all publications. The mean Article Influence ® score is 1.00, and a score greater than 1.00 indicates that each article in the journal has above-average influence, and vice versa [4]. These additional metrics contribute additional validation to the original impact factor alone.
However, this still doesn't address the concern that all measurement of quality of scientific journals seems to be in hands of a single, private enterprise, and is dependent on their choice of the journals that are being followed and that contribute citations. They also make decisions on how to classify journal's published items, which determines the denominator of the IF equation. However, things have changed in this area, too, and competition has emerged. The ISI Web of Knowledge SM searching tool -Web of Science TM [5] -is no longer the only prominent web-based provider of citations to the published articles. Another publishing giant, the Reed Elsevier corporation, have developed their own citation database -Scopus ® [6]. It is a very similar search tool, although possibly more comprehensive in some areas of science -it covers more than 20 000 titles from over 5000 publishers, offering about 20% more coverage than Web of Science TM [7]. Both of these search engines require relatively expensive subscription for access. However, Google, Inc. corporation, which states that their mission is "...to organise the world's information and make it universally accessible and useful", have launched their own, free search engine that also tracks citations -Google Scholar ® [8]. The coverage of journals and academic sources in Google Scholar is not only completely free to the general public, but also much more comprehensive than either Web of Science ® or Scopus ® . This is because it takes into account citations found in virtually any document that has ever been exposed to the internet, in any shape or form. Therefore, citations to published articles can nowadays be tracked using at least three tools -Web of Science ® , Scopus ® and Google Scholar ® . They will quote different number of citations, with the first two being less inclusive, and the third one being more inclusive.
In addition to improved tracking of article citations and journal' s impact, another metrics of scientific impact has emerged -Hirsch index (or h-index) [9]. Designed initially to capture the productivity of any individual scientists in a single number, this metric can also be applied to sci-entific journals. h-index measures the number of articles associated with a scientist, or a journal -h -that have been cited h times or more. That means that a scientist, or a journal, with h-index of 50 would have published 50 articles that have each been cited 50 times or more. All other articles associated with this scientist would have been cited less than 50 times, and therefore they would not contribute to the score. The beautiful simplicity of this metric and its ability to capture both the quality and the quantity of research output in a single number has made it extremely popular in recent years [10]. Web of Science ® , Scopus ® and Google Scholar ® also provide it in association to research output of individual scientists. Interestingly, Google Scholar recognised its value in evaluating scientific journals, too, and it provides ranking of world's leading 100 journals in different languages by their 5-year h-index, ie, the h-index based on citations to all papers published in the previous 5 years [9]. While impact factor favours journals that publish small, selected number of papers which attract high number of citations each year (such as journals that specialize in publishing review articles), 5-year h-index does more justice to journals that publish large number of quality papers. With h-index, the quantity of published papers becomes a potential strength, as more papers could contribute to h-index; while with impact factor publishing many papers could be seen as a burden to achieving high metric, because they increase the denominator.
Finally, the widespread use of the internet and social media, and the possibility to document and store the information about its usage, has lead to an entirely new way of evaluating the impact of scientific publishing. Given that the papers are now accessed on the Internet rather than by reading printed journals, this allowed evaluation of the impact not only through article, but also through their access and usage. Thanks to tools such as PubMed Central ® [11] and Google Analytics ® [12], it is possible to follow access and downloads of individual articles in their electronic or PDF forms, tracing them to geographic location and other characteristics of the user. This has shown that many articles, especially those related to policy, are used, read and commented lot more than they are cited. Social media such as Facebook [13] and Twitter [14] allow following of how much immediate impact do research articles generate, and how quickly do their ideas spread through social media -ie, how often are they "shared", "tweeted" and "liked". This allowed a broad, multi-dimensional evaluation of research impact that could not even have been imagined only a decade ago.
So, finally to the point -how did the Journal of Global Health do over the past three years, taking into account all those different measures of scientific impact that exist today? We decided to focus on two measures of usage and two citationtracking tools. The measures of usage are based on online access to the articles through PubMed Central [11] and through our own journal's website, monitored using Google Analytics tools [12]. The two citation-tracking tools were ISI Web of Knowledge SM [5] and Google Scholar ® [8]. Table 1 shows the ranking (according to total access, ie, usage) of the 20 most accessed papers published in the first 3 volumes -between June 2011 and December 2013. The 66 items (articles, viewpoints and editorials) published during this period were accessed more than 50 000 times and from more than 160 countries of the world. Table 1 shows that among all recorded episodes of access, full-text access was typically 2-4 times more common than PDF download. Also, the access occurred about 8 times more frequently through PubMed Central than through our own website, as a result of PubMed searches that pointed to our content.
Seven among the 13 most accessed papers were focused on global, regional and national epidemiological estimates of important infectious diseases: childhood pneumonia (ranked 1 st ), typhoid and paratyphoid fever (2 nd ), sepsis (4 th ), neonatal sepsis (8 th ), maternal parasitic infections (9 th ), childhood diarrhoea (12 th ) and maternal bacterial and viral infections (13 th ). The high position of those articles could have been somewhat expected, but the most pleasant surprises on the list were the articles on the floods in Southeast Asia as a health priority (3 rd ), biomarkers for neonatal sepsis (5 th ), a historical perspective on communicable disease control in China (6 th ) and malnutrition as a contributor to "double burden of disease" in poor countries (7 th ). Two further related clusters worth mentioning were three papers on the topic of non-communicable diseases in low-and middle-income countries (ranked 15 th , 16 th and 20 th ), and the two research priority-setting exercises that used the CHNRI method (ranked 10 th and 18 th ). Table 1 also shows that access indicators correspond reasonably well to the number of citations received, especially when analyzed across all 66 published items and normalized for the number of months since the time of the publication. The number of citations recorded by Google Scholar® was typically between 25% and 75% larger than the number recorded by ISI Web of Knowledge SM . The two most accessed papers also stand out in terms of the number of citations received (especially when this is adjusted for the duration of citing period), but several other papers are already showing that they will likely accumulate at least 10 citations in ISI Web of Knowledge SM during 2014. to 24 original research articles in 2011 and 2012, setting our first, unofficial impact factor at 3.208. In addition, we received 11 citations during 2013 to our 12 original research papers published during 2013, leading to our first immediacy index of 0.917. Our journal's self-citations did not contribute to this calculation because JoGH is still not indexed among the journals that the ISI Web of Knowledge SM and the Journal Citation Reports® use for computing impact factors [15].
Can we be satisfied with our impact to date? The numbers above are certainly encouraging. When we launched the journal in 2011, we defined our mission as serving "the community of researchers, funding agencies, international organizations, policy-makers and other stakeholders in the field of international health by providing an independent assessment of the key issues that dominated the previous semester in the field of global health and development; publishing high-quality peer-reviewed original research; providing objective reviews of global health and development issues; and allowing independent authors and stakeholders to voice their personal opinions on issues in global health". We seem to have addressed all those goals to a substantial extent -through publishing a selec- Given that we first appeared on PubMed Central in January 2013, and that the large majority of access to our content is achieved through PubMed searches that return our papers as a result, the total access to our content (more than 50 000 requests) has essentially been achieved over a period of a single year. Moreover, this means that our average published paper has been seen nearly 1000 times over the past year -ie, between 2-3 times each day. In addition, we were pleasantly surprised by the fact that we recorded access to our content from nearly every country in the world over the period of just over a year. This is certainly not a negligible impact. Moreover, the current citation analysis raises favourable expectations for the future years, especially taking into account the average time between access to articles and their citations -suggesting that the potential for translation of a considerable recorded access to our published content into citations is yet to be revealed.
The encouraging signs that our journal is taken increasingly seriously among the major players in global health are reflected in the fact that the number of external, non-com-missioned submissions that we consider to be of high quality is continuously increasing: while we only received one external, non-commissioned submission in our first year (June-December 2011), we received 6 external submissions in 2012, then 42 in 2013, while the projection for the 2014 based on the first three months is already above 70. We publish a total of 20 items each year, which means that our rejection rate (when all the commissioned papers are added) is already approaching 80%. Another sign of our increasingly notable presence in the global health research community is reflected in the fact that we already have theme issues produced in collaboration with leading global health organizations -Program for Appropriate Technology in Health (PATH) and The Gates Foundation (2013 June theme issue on childhood diarrhoea), Imperial College London's Global eHealth Unit (2013 December theme issue on mHealth), and United Nation's Children Fund (UNICEF -2014 December theme issue on integrated community case management), at the same time keeping a rigorous peer review process. We are looking forward to further similar collaborations, as a growing evidence of our journal' s increasing impact in global health research community.