Radiation Therapy Research: A Global Analysis 2001-2015.

Radiation therapy is a core modality of cancer treatment; however, concerns have been expressed regarding its underutilization and its lack of prioritization as a research domain relative to other cancer treatment modalities, despite its rapid technical evolution. It is therefore important to understand, from a public policy perspective, the evolution of global radiation therapy research, to identify strengths, weaknesses, and opportunities. This study used a bibliometric approach to undertake a quantitative analysis of global radiation therapy research published between 2001 and 2015 and available in the Web of Science (Wos) database, with particular focus on the 25 leading research-active countries. A total of 62,550 radiation therapy research articles from 127 countries, published in 2531 international journals, were analyzed. The United States was responsible for 32.3% of these outputs, followed by Japan (8.0%) and Germany (7.7%). Nearly half of all publications related to preparation and delivery of radiation therapy, combined-modality regimens, and dose fractionation studies. Health services research, palliative care, and quality of life studies represented only 2%, 5%, and 4% of all research outputs, respectively. Countries varied significantly in their commitment to different research domains, and trial-related publications represented only 5.1% of total output. Research impact was analyzed according to 3 different citation scores, with research outputs from Denmark, The Netherlands, and the United States consistently the highest ranked. Globally, radiation therapy publication outputs continue to increase but lag behind other spheres of cancer management. The types of radiation therapy research undertaken appear to be regionally patterned, and there is a clear disconcordance between the volume of research output from individual countries and its citation impact. Greater support for radiation therapy research in low- and middle-income countries is required, including international collaboration. The study findings are expected to provide the requisite knowledge to guide future radiation therapy research programs.


Introduction
Cancer research is one of the most globally active domains of science, with more than $14 billion per annum in public and private expenditure (1). Research is integral to improving outcomes from cancer, be it through an improved understanding of the etiology of disease, identifying new treatment targets or modalities, or by the provision of information on how best to coordinate cancer services to deliver affordable and equitable cancer care.
Radiation therapy is one of the main modalities of control, cure, and palliation for cancer, with approximately 50% of cancer patients requiring radiation therapy during their disease course (2)(3)(4). Given the outcomes it can deliver with respect to survival and quality of life, its underutilization as a clinical modality (3) and the lack of prioritization it has been given as a research domain within the cancer spectrum remain major concerns. A recent study within lung cancer has found that only 8% of lung cancer research is devoted to radiation therapy research, compared with 20% for genetics, 17% for systemic therapies, and 16% for prognostic biomarkers (5). This undoubtedly will influence patterns of care for particular disease entities and the potential for new developments that will ultimately improve patient outcomes.
It is therefore important to understand from a public policy perspective how, why, and which particular research domains evolve and have an impact on outcomes. For example, how do different countries influence the radiation therapy research agenda, either through the volume of research they publish, the citation impact of their articles, or their commitment to particular research domains (eg, basic science, health services research)?
An empirical analysis of research outputs would also highlight gaps and provide direction as to which research areas should be prioritized to meet current and future challenges. For example, there is currently an ongoing debate (6) as to whether the increasing focus on technological innovation within radiation therapy to improve the therapeutic ratio has been at the cost of not developing a greater understanding of the potential role for radiation therapy in exploiting cancer weaknesses based on the biological hallmarks of cancer. It is thought that further research into these areas could unlock new and more costeffective opportunities for radiation therapy to improve outcomes from cancer. In addition, it would help to ensure that radiation therapy continues to be relevant and that research in this area is prioritized in an era of precision medicine, which is increasingly drug-focused (7,8).
Using a bibliometric approach, we present an analysis of global research on cancer radiation therapy between 2001 and 2015. This type of analysis is now used routinely in public policy analysis to study research domains (9,10). We examine the growth in research output from 25 leading countries, the volume of research produced relative to their wealth, the main radiation therapy research domains these countries prioritize, and the citation impact of radiation therapy research stratified according to country and research domain.

Methods and Materials
We performed a bibliometric analysis of radiation therapy research outputs during 2001-2015, based on articles and reviews in the Web of Science (WoS) database. This contains full bibliographic information about the articles, including all addresses as well as the numbers of citations received by each article in each year. Of note, the use of additional biomedical databases does not significantly increase the yield of relevant journals.
We used filters (macro algorithms based on logic functions) developed by an expert bibliometrician (G.L.) in conjunction with the team's expertise in radiation therapy and radiation research to identify relevant articles in the WoS. The search for radiation therapy articles included articles in ANY journal, including general medical and basic science journals, provided that they had a title term indicative of cancer (composed of 323 words and short phrases (11)) AND had the word "radiation or radiotherapy" including wildcard "rad*," OR contained 1 of the 12 radiation therapyespecific title words, such as "brachytherapy" (Supplementary Appendix 1; available online at www. redjournal.org). It also included articles in 7 specialist cancer radiation therapy journals identified by the study authors (Supplementary Appendix 1; available online at www. redjournal.org) and in 185 specialist cancer journals, provided that articles in the latter contained 1 of the radiation therapy filter terms. In addition, there were 20 more journals identified by the authors that also covered both cancer and noncancer radiation topics (Supplementary Appendix 1; available online at www.redjournal.org). Articles in these journals were retained ONLY if they had one of the 323 cancer filter terms.
All these filters were developed through iterative rounds, which involved creating datasets and having these manually coded by clinical experts as to their relevance to the research fields being sought (using methods previously described (12)). This process resulted in both a precision or specificity and a recall or sensitivity for identifying radiation therapy research articles of 0.95, which is considered very high (11).
There were 127 countries that contributed to these radiation therapy articles. However, the results presented in this study will primarily focus on the 25 leading researchactive countries that are responsible for 97% of the total.
The counts of the numbers of publications per year were obtained as both integer and fractional counts using the article's addresses. For example, if an article has 2 addresses in Germany and 1 in France it would be counted as 1 for each on an integer count basis, but as 0.67 and 0.33, respectively, on a fractional count basis. Fractional counts sum to less than the total partly because of the outputs of the other countries and partly because some articles had no addresses. Integer counts summed to more than the total because of international co-authorship. Unless otherwise mentioned, our analysis is based on fractional counts, which give a much better impression than integer counts of the relative research effort by each country (13,14).

Volume of research
For each of the 25 leading countries, we analyzed the numbers of published radiation therapy research articles for each year from 2001 to 2015 and calculated the annual average percentage growth rate (AAPG) and the ratio of outputs between 2011-2015 and 2001-2005. We also calculated the commitment of the 25 countries to radiation therapy research relative to their output of cancer research overall. For example, between 2001 and 2015, Canada published 3956 articles out of a world total of 62,550 (6.3%). In the same period, Canada contributed to 4.2% of all oncology research outputs worldwide. Canada's relative commitment to radiation therapy research was therefore 6.3/ 4.2 Z 1.50. For selected high-and middle-income countries we also analyzed the association between radiation therapy research output and each country's gross domestic product (GDP). The GDP is a measure of a country's economy and is the total market value of all consumer goods and services produced by all the people and companies in the country in a period of time (quarterly or yearly).

Radiation therapy research domains
Radiation therapy research publications were categorized into 10 research domains. These were defined using subfilters that contained a set of title words and strings to categorize relevant articles into particular domains. These subfilters were all developed as part of an iterative process by A.A. and Y.L. in collaboration with G.L., with additional terms being added to each of the sufilters to capture many of the articles not yet classified. We also used a complex logic process as the individual subfilters were applied to the spreadsheet, so that some articles were classified if they were identified by particular subfilters but NOT by others.
The research domains and individual codes used for analysis were as follows: BIOL ((radio)biology); PHYS (physics); ASSU (quality assurance); FRAC (dose fractionation and sequencing studies); COMB (multimodality studies involving radiation therapy); PRED (preparation and delivery of radiation therapy); PALL (palliative care); QUAL (quality of life); HESR (health services research); and REVS (review articles). Of note, the PRED domain also included studies evaluating particle therapy in the clinical setting. The domains were not mutually exclusive, and therefore articles reviewing quality of life in relation to clinical dose fractionation studies would be included in both categories (ie, QUAL and FRAC). In addition, articles classified solely within either BIOL, PHYS, or ASSU were treated as nonclinical basic science studies. The PRED, FRAC, and COMB domains included studies that measured clinical endpoints such as morbidity. Eventually, all but 6465 articles were classified into 1 or more of these categories (10.3%).
Fractional counts were calculated for each country and research type, and the results compared with the world totals so as to show whether a country was over-or underrepresented relative to the world average for each research domain. We also analyzed the number of articles describing radiation therapy trials (phases 1-4) and the fractional counts of these articles for each country.

Citation impact by country and research domain
The citation counts for each article from 2001-2011, year by year, were downloaded from the WoS. The 5-year citation counts (actual citation impact, ACI) beginning in the publication year were calculated. A 5-year window was used as a compromise between the need for immediacy (ie, citations to recent articles) and stability (ie, inclusion of the peak year for citations, usually the second or third year after publication). It is best determined for a country on the basis of fractional counts, because many of the most cited articles are multinational. Altogether, citation counts were determined for 32,162 articles.
For each country we calculated an arithmetic mean citation score as a measure of the impact of their radiation therapyerelated research. This was based on the ACI of each of their articles during the study period. To calculate this, the citation score for each article was multiplied by the country's fractional contribution to that article and the products summed, and the total divided by the sum of the country's fractional counts for the relevant years. For example, for Germany, the top-cited article, with 1360 citations, had a German presence among the addresses of 0.133. Germany was therefore credited with 1360 Â 0.1333 Z 181.3 citations. All the products for individual articles were summed, to give the fractional German citation total of 44,341.4 citations, which when divided by the German fractional count citable total of 3306.9 articles gives an arithmetic mean citation score of 13.41 citations per article. A geometric mean was also calculated on the basis of the logarithms of the actual citation counts. This value is considered to be a better indicator because it is less influenced by a few very high citation counts (15).
Another measure of citation impact is the number of a country's articles that receive enough citations to put them in the top 1%, 2%, or 5% of all cited radiation therapy articles. During the period of analysis (2001-2011), an article would have had to receive 93, 68, and 44 citations, respectively, to be in these centiles. A WorldScale (WS) value at a particular centile was calculated according to the ratio of the proportion of articles from a selected country in the top x% of cited articles compared with the proportion of all articles in that particular centile. So for Germany, with 3306.9 citable articles in the 11 years, because it published 136.5 articles with 44 or more citations, its WS value at 5% was (136.5/3306.9)/0.05 Â 100 Z 83. This value is lower than 100 and indicates it has fewer than the expected number of articles in this centile. The 3 WS values at 1%, 2%, and 5% for each country were averaged to give a composite value for a country's highly cited articlesdthe WS mean.
The 25 leading countries were ranked separately on these 3 citation indicators (arithmetic, geometric, and WS means), and the rankings were then averaged. Using similar methods, we also determined the citation impacts for articles in the 10 research domains. Finally

Results
In total, 62,550 articles were identified from 2531 international journals and from 127 countries. Nearly all of the articles were in English (60,494 articles, 96.7%), but others were in 20 different languages, led by French (1193, 1.9%), German (598, 0.96%), and Spanish (76, 0.12%). A few were in Chinese, Japanese, or Korean.
The 25 leading research-active countries, which accounted for 96.9% (or 60,673 articles) of the total in the file, included (in alphabetical order): Australia, Austria, Belgium, Brazil, Canada, China, Denmark, France, Germany, Greece, India, Iran, Italy, Japan, Netherlands, Norway, Poland, Republic of Korea, Spain, Sweden, Switzerland, Taiwan, Turkey, the United Kingdom, and the United States.

Volume of research
Between 2001 and 2015 the volume of radiation therapy research increased year on year, with a doubling in world outputs (Fig. 1). The United States was responsible for 32.3% of these outputs (Table 1) Most countries had a smaller percentage presence in radiation therapy research than they did in oncology overall ( Table 1). The Netherlands, Canada, and Belgium are notable exceptions. China, despite having a sustained and large increase in its volume of radiation therapy research over the 15-year period, still had a smaller percentage presence in radiation therapy research than in all cancer. A similar pattern was observed for Brazil. Figure 2 shows that the volume of radiation therapy research output is positively correlated with each country's GDP, with the spots for most of the selected high-and upper-middleincome countries close to the least-squares regression line. Notable exceptions are The Netherlands, which published more than 3 times as much as the regression line predicts, and Taiwan, Canada, and the United States, whose output was double or almost double the amount predicted. On the other hand, uppermiddle-income countries, such as China and Brazil, published less than half the predicted amounts. Russia published less than one-tenth of the amount predicted from the regression line. All these differences are highly statistically significant (P < .001% on the Poisson distribution with 1 degree of freedom). Basic science represented a further third of research outputs, with radiobiological (BIOL) investigation accounting for 19% and physics (PHYS) for 13% of total outputs. Health services research (HESR) was very limited, accounting for only 2% of the total. Palliative care (PALL) and quality of life studies (QUAL) in radiation therapy also made only small contributions to the literature, of 5% and 4%, respectively. Figure 3 shows that between 2001-2005 and 2011-2015, radiobiology research has had the most modest increase in volume of research output (AAPG Z 4.8) compared with clinical domains such as preparation and delivery (AAPG Z 9.0) and dose fractionation studies (AAPG Z 9.3). Health services research has had sustained increases over the time period, with an AAPG of 10.8. Table 2 shows the relative commitment of different countries to the 9 research domains. Whereas the United States, France, and Austria are well represented in all the domains, this is not so for other countries. For example, China's radiation therapy research is focused on basic science, particularly on radiobiology. Canada has a focus on physics, palliative care, and health services research. The Netherlands, Belgium, Denmark, and Iran have a strong commitment to physics research, with The Netherlands also noticeable for its commitment to quality of lifeerelated research and Iran to quality assurance. Australia, like Canada, and to a lesser extent the United Kingdom, is notable for its strong commitment to health services research; Norway excels in quality of life and palliative care research.

Clinical trial outputs
Of the 62,550 radiation therapy articles, only 3926 (6.3%) involved clinical trials. Of these, 34 described preparatory work, and 687 were secondary sources (meta-analyses, systematic reviews, and other trials without specification of stage). The remaining 3205 articles (5.1%) described trials in the 4 established phases: 769 phase 1, 1065 phase 2, 1367 phase 3, and 4 described phase 4 trials. Table 3 demonstrates the variation in clinical trial outputs across the top 20 leading countries. The United States published the greatest number of outputs across phase 1 to phase 3 studies. Japan is notable for the high proportion of publications describing phase 1 studies relative to all other countries. With respect to phase 2 trial outputs, Japan again features strongly, as does Italy and China. Phase 3 trial outputs are dominated by the United Kingdom, Germany, and The Netherlands after the United States. An analysis of the ratio of phase 3 to phase 2 trial outputs from each of  the 25 countries showed significant international variation. The United States, for instance, had a relatively low ratio of phase 3 relative to phase 2 outputs. This was also notable for countries such as Japan (0.50), Italy (0.58), Republic of Korea (0.60), and Spain (0.46). Conversely, countries such as the United Kingdom (3.45), Netherlands (4.72), Sweden (4.14), and Poland (5.02) published more than double the number of trial outputs of phase 3 studies compared with phase 2 studies in the study period. India, a middle-income country, also had a strong commitment to phase 3 studies (2.83). Table 4 shows the 25 leading countries with their mean ACI values (arithmetic and geometric) and their mean WS values. The latter is based on the numbers of their articles with enough citations to put them in the top 1% (!93 citations), top 2% (!68 citations), and top 5% (!44 citations), as described in the Methods and Materials section. The countries are ranked according to the mean ranking across the 3 scales.

Citation counts and research impact
We find that Denmark, The Netherlands, the United States, Switzerland, and Belgium are the only countries with consistently superior performance at all 3 WS percentiles (ie, values > 100). High research output countries such as China, Japan, and Germany rank in the lower half of the table. Conversely, Denmark, Switzerland, and Belgium seem to produce research with greater impact, despite the low volume of their research relative to that of the other countries. Table 5 shows the citation impact of different radiation therapy research domains according to the mean WS values. It shows that whereas review articles, and research focusing on combined modality treatment (eg, radiation and drug therapies), are highly cited and potentially will have a greater impact on practices of care, research into quality assurance and health services research may have less impact, with a small fraction of articles in the top centiles of cited radiation therapy articles.

Discussion
This analysis of the global research landscape is the first to characterize the output of radiation therapy research globally and to identify trends in research priorities and contributions by individual countries. We found a doubling in overall research output over a 15-year period, consistent with trends observed in other disciplines (16). The globalization of research has been a major contributor to this increase in output. Although the United States was responsible for more than one-third of the radiation therapy research output between 2001 and 2015, there was only a 55% increase in research output  in the United States during that period, compared with more than 2000% in China. Significant increases in research output were also seen in several other middle-income countries, including India, Brazil, Turkey, and Iran, although their overall contribution to total worldwide radiation therapy research remained small. In this regard, it is important to acknowledge the issues related to access to radiation therapy in the majority of low-and middle-income countries (LMICs) (17), which undoubtedly will have important implications on their ability to influence the research portfolio.
Furthermore, when these countries' research output is compared with their GDP, they still lag some way behind the major high-income countries. This is likely to be influenced by differences in cancer research funding between LMICs and high-income countries. We know that only 2.7% of total global cancer research investment is directly relevant to LMICs (18). Of that investment, the majority of cancer research funding is directed to studies that focus on cancer biology and drug development, rather than radiation therapyerelated research (5,19).
It is important that LMICs are supported and encouraged to participate in research because of the need to continue to develop cost-effective treatment pathways, which can also meet key goals such as equity and efficiency within the constraints of their health system (20). The wider research community will also benefit from a more globally inclusive research base, given inherent differences in cancer epidemiology and biology related to risk factor exposure. In the European Union for instance, limitations in the size (by population) and resources of individual countries is compensated by strong international collaboration, with the European Organization for Research and Treatment of Cancer (EORTC) acting to coordinate radiation therapy research in the region.
It may be argued that with improved survival outcomes and more people living with cancer (21), there should be greater attention in research to the domains of quality of life, health services, and palliative care. However, these areas represent only 4%, 2%, and 5% of total radiation therapy research outputs, respectively, over the study period. This is in stark contrast to other clinical research domains, such as preparation and delivery (24%) and dose fractionation studies (15%). Quality of life research seems particularly low given the dominance of clinical studies involving patients and the importance of this as a clinical endpoint.
This discrepancy between the health needs of the population and current research priorities was also found in other cancer studies. In a bibliometric analysis, Sanson-Fisher et al (22) found that there were 4 times as many publications on chemotherapy in 2005 as on quality of life research, despite steady increases in the latter over the preceding 20 years.
Health services research encompasses a broad multidisciplinary area addressing issues related to access, equity, and the value of health care. This branch of research helps to define new research priorities. It also aids effective implementation and sustainability of new innovative processes of care given the health system constraintsdfinancial, political, or geographicdof a particular country. However, despite its importance, this area of research still lags far behind more established clinical and basic science research domains. It may not be considered to have the same value or relevance as other domains, as shown by its relatively low citation impact. Furthermore, this is an interdisciplinary subject area and requires collaboration between radiation oncologists and social scientists, such as health economists and epidemiologists.
However, this view may finally be changing because of growing fiscal constraints that affect both high-income countries and LMICs and an increasing focus on valuebased frameworks within medicine (23,24). One example is the Health Economics in Radiation Oncology project under the auspices of the European Society of Radiation Oncology, which seeks to address the shortfall in applied research in this area (25).
The low commitment of several countries to palliative care research shown in Table 3 may be the result of the relative lack of senior academic appointments in this domain. This may limit opportunities for cross-sectoral research collaboration and for the attraction of research funding and the creation of research infrastructure (26). Norway demonstrated the strongest commitment to palliative care research, relative to other priorities. This is likely to reflect the country's strong support for palliative care at all levels of the public health care system (27).
The predominance of clinical research outputs in our study is also a likely reflection of the particular time period in which these analyses were undertaken, in which significant technical improvements have been made (intensity modulated radiation therapy, particle therapy, motion management) that have sought to reduce the morbidity Number of articles expressed as fractional country counts. * Percentage contribution of each individual country to total world phase 1, phase 2, and phase 3 research outputs. y Ratio of articles describing phase 3 studies and phase 2 studies for each individual country. associated with treatment. In addition, large clinical trials in this era, such as those assessing multimodality therapy (28) and different dose fractionation schedules (29), have been a natural consequence of the preclinical research performed in the 1980s and 1990s, which had a much greater biological focus, with studies addressing dose per fraction, hypoxia, and drugeradiation interactions. Review articles were the most highly cited research domain, which suggests that evidence syntheses in the form of systematic reviews and meta-analyses have the potential to greatly influence practice, which is desirable. Research into multimodality therapy involving radiation therapy is also highly cited, most likely because of the large investment made in such studies (especially those involving pharmaceuticals) and the importance that these outputs have across the entire cancer spectrum from basic science to medical, surgical, and radiation oncology. Abbreviation: WS Z WorldScale. Citable Z numbers of articles in these years; WS Z WS mean value at top 1%, 2%, and 5% of citations; Arithmetic Z arithmetic mean of actual citation impact values; Geometric Z geometric mean of actual citation impact values. Countries are ranked by mean ranking on these 3 indicators.  REVS  2733  256  104  59  189  191  216  199  COMB  6435  452  227  135  142  177  210  176  FRAC  5650  344  154  88  123  137  156  138  PRED  8944  601  207  94  135  116  105  119  PALL  1758  96  36  17  110  103  97  103  BIOL  7975  470  159  71  119  100  89  103  PHYS  4708  252  88  32  108  94  68  90  QUAL  1611  84  29  11  105  90  68  88  HESR  727  26  10  3  72  69  41  61  ASSU  2660  71  18  8  54  34  30  39 Abbreviations as in Table 2. Clinical trial publications accounted for 5% of total research output during the study period. The low overall proportion of publications related to clinical trials also points to a more worrying trend within radiation therapy research. This concerns the level of evidence required to integrate new processes of care and technologies into treatment, which remains reliant on small-scale observational studies and, more recently, modeling studies (30,31). Although there are constraints to conducting phase 3 trials in radiation therapy, increasing concerns about the value of new innovations means that investment in trials is required (public or private) to ensure new modalities are evaluated with rigorous methods so as to enable cost-effectiveness analyses to support their widespread implementation (32). Of even greater concern is that once available in the market following US Food and Drug Administration and European Union approval, few phase 4 studies are subsequently undertaken, even within centers that are early adopters of a new technology (33). Pragmatic research designs, such as multicenter observational cohort studies or nationally coordinated coverage with evidence development schemes, are alternative approaches that have been considered for evaluating the effectiveness of treatment in the real-world setting (31,34).
A number of interesting observations are offered in Table 3, which looks more closely at phase 1 to 3 trial outputs from individual countries. European countries such as the United Kingdom, The Netherlands, Sweden, and Poland produce a much higher ratio of phase 3erelated trial outputs relative to phase 2. India also demonstrated a greater commitment to phase 3 relative to phase 2 trials, compared with several high-income countries, where the reverse trend was observed. For example, the United States, Japan, and Italy had a significantly higher proportion of phase 2 study outputs relative to phase 3. It is not clear why such differences are apparent, but they may relate to economic and cultural factors. For example, whether the necessary infrastructure or funding is available to conduct radiation therapy research trials may be one factor influencing these figures. Countries may vary in the level of evidence required by health care reimbursement organizations before they will routinely fund new technologies or practices of care. In addition, organizations such as the US Food and Drug Administration require demonstration of safety, rather than efficacy, within phase 3 studies, before approving new technologies, which has a downstream effect on the types of research evidence likely to be generated before clinical adoption.
Although the United States was the largest contributor to randomized controlled trial publications, the proportion of its total radiation therapy research output devoted to randomized controlled trials was significantly less than that in other countries, such as India. This may reflect the recent trend of pharmaceutical and medical technology companies to conduct trials in countries where the personnel costs are lower and where the large pool of potential research participants can accelerate recruitment (35). There may be many benefits that accrue to these countries from clinical trials research, such as the opportunity for international collaboration, investments in healthcare infrastructure, and the redirection of research priorities toward locally relevant and feasible interventions. However, concerns have also been raised about whether there is adequate transparency and oversight of human subjects in these countries, which may have weak regulatory systems and limited experience in research (35).
In this study we used 3 different citation scores based on actual citation counts to rank countries on the quality and importance of their published outputs. Although China, Japan, France, and Germany were high-output countries in terms of the number of research publications, their citation performance was significantly lower than that of several other countries that had lower research outputs (eg, Denmark). This may be partly because their articles tended to be published in low-impact journals or related to the language in which they were published. In that regard, a study of the impact of publication language on citation frequency in the scientific dental literature showed that articles published in English had a 6-7 times higher chance of being cited than articles published in German or French (36). Furthermore, research articles from middle-income countries are poorly cited. It is unclear whether this relates to the perceived quality or level of interest in research from these countries or the low impact factor of the journals these studies are published in. This needs to improve to encourage middle-income countries to become more involved in radiation therapy research and influence practices of care. If not, practices of care may risk becoming regionally entrenched or influenced by a few select countries in North America and Europe, which could result in distinct knowledge gaps in the empirical literature.
International collaboration is one mechanism by which this situation can be improved and prevent unnecessary duplication of research, to ensure the best available evidence is used to drive radiation therapy practice. One could consider a regional approach to identifying gaps in the evidence base and undertake relevant clinical and nonclinical studies, for example, through pre-existing regional alliances such as the Regional Office for the East Mediterranean and the Pan American Health Organization. In addition, the International Atomic Energy Association continues to support and coordinate multinational clinical radiation therapy trials (www-naweb.iaea. org/nahu/ARBR/crp.html) that have impacted on clinical practice (37).
Our analysis of open access articles demonstrates that they are steadily increasing over time, representing nearly 40% of articles currently published in the WoS either as gold or green open access. Although the overall citation impact of more recent open access articles (2010-2012) is higher than that of closed articles, this varies depending on country of origin and the type of radiation therapy research undertaken.
The present study must be considered in the context of its strengths and limitations. The analysis has been undertaken on an individual country basis, and findings with respect to country outputs are potentially skewed depending on the size (by population) and resources of individual countries. However, this reflects the reality of their research strengths and weaknesses. Outputs relative to each individual country's GDP and research impact have also been presented. In addition, European countries that conduct a number of collaborative multinational studies may seem to produce comparatively less trial outputs when using fractional counts compared with the United States, for instance. A regionally based analysis may be one mechanism for addressing this in the future. We have used citation frequency as a proxy indicator for quality of research and dissemination of scientific findings. However, a true evaluation of the scientific quality of publications cannot be achieved without an independent and dedicated assessment of their merit. Furthermore, citation frequency cannot determine whether a publication changes practice and improves population health (16).
We have not provided a detailed analysis of the factors that have led to the observed trends and can only hypothesize potential reasons at this stage. The quantity of research outputs may be affected by publication bias, with failure of up to 20% to 30% of trials to report their results (38). Equally excellent research may not be published or published elsewhere. This will have an impact on country-level integer and fractional counts, as well as on potential underrepresentation of clinical research outputs.
We have selected publications available in the WoS for analysis, and it is therefore likely that some research output in national language journals has not been included, which could affect our results for country-level outputs. In addition, as with any bibliometric evaluation, it is not possible to guarantee inclusion of all relevant articles. However, attempts to minimize this have been sought by undertaking several iterations to develop the precision of our search filters to ensure inclusion of articles that have a relevant title word or Medical Subject Headings terms specific for clinical and basic science articles in radiation therapy. Although the WoS has selection criteria for the inclusion of journals based on repute and citation, it is unknown what proportion of low-quality "predatory" open access articles is included and the impact this has had on the estimations of total radiation therapy research output (39). Finally, although our coding scheme for research types was made as explicit as possible, it is possible that some publications were miscategorized or that not all publications could be categorized according to the selected domains.

Conclusions
To conclude, our findings provide a detailed analysis of trends in radiation therapy research since 2001. Although there has been a doubling of radiation therapy research outputs over the study period, significant variation exists in the research output of individual countries, with evidence that radiation therapy research output is falling behind that of other cancer-related research domains. Although LMICs such as India, Iran, China, and Brazil continue to increase their radiation therapy research output, this still lags behind what is expected given their economic strength. Greater support is required to develop the necessary infrastructure to support high-quality research in LMICs that will contribute to the development of the specialty overall but allow the essential upscaling of radiation therapy resources in these countries.
When considering the radiation therapy research types, there is evidence of individual countries' being committed to particular domains that reflect national cultures and economies. A major concern remains the very low proportion of trial-related publications within radiation therapy. This is an area that requires greater investment if we are to try and establish the relevant evidence base to promote the implementation of the most cost-effective, highvalue care. In its absence, there will remain a lack of transparency as to the comparative benefits of innovations relative to existing treatments, especially given that the bulk of research focuses on clinical domains. To this end, greater focus on quality of life studies is required: despite the clinical predominance of most research output, few articles considered these endpoints. In addition, it is unknown what impact the slowing down of basic science research outputs over the study period means in the long term, with respect to identifying new pathways for improving patient outcomes. Greater emphasis on health services research would provide robust evidence on translating clinical evidence into practice. Finally, although citation impacts do not necessarily reflect influences on practices of care, they provide some understanding of the degree of interest or quality of the article. Given differences across countries, this may suggest that research outputs from particular countries are considered to be of higher quality and potentially have a greater impact on influencing practices of care.