A decade of Rural Clinical School research : a PubMed review

Introduction: One parameter of the operational framework of the Australian Rural Clinical Training & Support Program (RCTS) is rural health research, yet there are no published reports of the research outcomes generated by these hallmarks of Australian rural medical education. To assess the contribution of RCTS to rural health research, their MEDLINE-indexed research publications over the last decade was analysed, using a bibliometric method. Methods: MEDLINE-indexed RCTS publications from 2004 to 2013 were retrieved using validated PubMed queries. Two authors independently checked all retrieved RCTS publications for validity. Australian rural health (ARH) publications from RCTS were selectively enumerated and their proportion among all Australian rural health publications in each year was determined. ARH publications were defined as Australian publications that explore issues relevant to the health of the regional, rural or remote Australian population. RCTS publications related to medical education, Indigenous health, rural service areas, National Health Priority Areas (NHPA), and National Rural Health Alliance Priority Areas (NRHAPA) were analysed. Frequency of publication in different journals was also compared. Results: A total of 280 RCTS publications were retrieved, increasing from 10 in 2004 to 49 in 2013. ARH topics dominated (177 articles; 67%). RCTS rural health publications increased as a proportion of all ARH publications from 3.4% in 2004 to 7.7% in 2013. Other RCTS publications increased from 2 (20% of total) in 2004 to 19 (39% of total) in 2013, and covered topics such as mental health, cancer, diabetes, obesity and asthma. RCTS medical education publications increased from 3 in 2004 to 14 in 2013. In total, 81 articles were retrieved comprising 28.9% of all RCTS publications. Indigenous health (18; 6%), rural populations (37; 13%) and rural health services (83; 29%) were the other important categories relevant to the RCTS funding parameters. RCTS


Introduction
The Rural Clinical Schools (RCS) program was implemented by the Australian Government in the year 2000 as a workforce strategy to address the shortage of medical practitioners within rural and remote Australia 1 .The RCS program was merged with the Rural Undergraduate Support & Coordination Program (RUSC) in 2011 and is currently known as the Rural Clinical Training & Support (RCTS) program 1 .The RCTS program is implemented across Australia in 18 RCS and regional medical schools (RMS).
In the early years of the RCS, the primary focus was recruiting medical students and addressing health workforce shortages, as well as encouraging health professionals to become academics 2 .In 2004, there were only 10 RCSs in operation, with a further seven being established throughout the decade.Research, although clearly important, was not a part of their operational parameters in this era.However, RCSs and RMSs are well placed to perform targeted rural health research to serve the unique needs of rural residents and better inform rural health policy 3,4 .In recent years, with the primary workforce and educational objectives having been satisfactorily addressed, rural health research has been included as one of the nine funding parameters of the RCTS program 5 .
Currently, there is a large number of publications about the RCTS program and medical student cohorts that have undertaken short-and long-term rotations [6][7][8] .However, very little is known about the academic and professional staff involved in the program, and their contribution to rural health research.The first 2014 RCTS Snapshot survey of professional and academic staff employed by the Australian RCTS program was recently completed with the support of the Federation of Rural Australian Medical Educators (FRAME) 9 .This article reports the results of a PubMed review of RCTS research for the decade 2004-2013.

Methods
A bibliometric method using PubMed was carried out to retrieve RCTS publications.When searching PubMed, advanced queries were used that included keywords (Medical Subject Headings -MeSH) and 'Tags' 10 .Tags are used to identify specific parts of a PubMed article.The following tags were used to retrieve relevant RCTS publications: Affiliation (AD) -identifies an organisation or postal/email address of the first author, and occasionally co-authors; Medical Subject Heading (MeSH) -papers linked to a specific topic/subject; and Title/Abstract (TIAB) -papers with a word or phrase in the title or abstract.The LIMIT option is used to define specific time periods.
A condensed version of a validated PubMed query to retrieve Australian publications 3 was combined with RCTS-related search terms targeting the affiliation and title/abstract tags (Table 1, query 1) to accurately retrieve MEDLINE-indexed RCTS publications from 2004 to 2013.Two authors (JB & KM) independently checked all retrieved RCTS publications for validity.Australian rural health (ARH) publications were defined as Australian publications that explore issues relevant to the health of the regional, rural or remote Australian population 3 .An ARH query (Table 1, query 2) was used to determine total ARH publications in the decade 2004-2013 so that the proportion of RCTS publications in this total could be determined.
To selectively enumerate RCTS publications related to rural health (Table 1, query 4), a rural health query (Table 1, query 3) was combined with the refined RCTS query (Table 1, query 1).Other publications from RCTS researchers were calculated by subtracting the rural health publication numbers from total RCTS publication numbers for each year examined.Confirmation of correct assignment into both categories (rural health/other) was determined during the validation process described above.
Medical education publications in the rural health category and other category were enumerated by checking all retrieved RCTS publications from 2004 to 2013.

Results
A total of 280 RCTS publications were retrieved for the decade analysed, increasing from 10 in 2004 to 49 in 2013 (Fig1).ARH publications from RCTS researchers increased from 8 (80% of total) in 2004 to 30 articles (61% of total) in 2013 (Fig1).RCTS rural health publications increased as a proportion of all ARH publications from 3.4% in 2004 to 7.7% in 2013.
Other RCTS publications increased from 2 (20% of total) in 2004 to 19 (39% of total) in 2013 (Fig1).Examination of the abstracts of all other RCTS publications showed that 23 of the 104 articles (22%) related to medical education.Other topics covered in a general, or non-rural, context included mental health, cancer, diabetes, obesity and bariatric surgery, health informatics, complementary and alternative medicines, point of care testing, and asthma.RCTS medical education publications increased from 3 in 2004 to 14 in 2013 (Fig2), giving a total of 81 articles (28.9% of all RCTS publications).In 2004, 30.0% of all RCTS publications related to medical education in either a rural or a general context.Although there were some fluctuations in the intervening years, in 2013 medical education publications still comprised 28.6% of all RCTS publications.Rural healthrelated medical education publications outnumbered general medical education publications by at least 2 to 1 in all years, with the exception of 2011 and 2012 where their numbers were relatively equal.
By using MeSH and relevant keywords, other important categories relevant to the RCTS funding parameters were analysed, namely Indigenous health, rural health services and rural population (Fig3).Indigenous health publications increased from 0 in 2004 to 4 in 2014.Four 'rural health services' articles were published in 2004, increasing to 11 in 2014, while 'rural populations'-related articles increased from 0 to 8 over the decade.
RCTS publications addressing NHPA (Fig4) show that mental health (n=15), diabetes (n=11) and cardiovascular disease (n=11) research articles were the top three areas covered.In contrast, no RCTS research publications related to dementia were found in the decade examined.
RCTS publications addressing the NRHAPA (Fig5) show that aged care (n=38) and mental health (n=15) articles were the most frequently published areas.In contrast, only one oral health research article was found and this was published in 2010.
The top five journals publishing RCTS research articles in the decade 2004-2013 were Rural and Remote Health (16%; n=45), Australian Journal of Rural Health (13%; n=35), Australian Family Physician (9%; n = 25), the Medical Journal of Australia (7%; n=19), and Australian Health Review (6%; n=16) (Table 2).Only two of these have a current journal impact factor greater than 3. Of the 103 journals in total that have published RCTS research, only 19 have a current journal impact factor of 3 or more.100) †Current journal impact factor greater than 3. ¶17 of these journals have an impact factor greater than 3.

Discussion
To the best of the authors' knowledge, this is the first detailed analysis of a decade of publications emanating from the RCTS program, and highlights that RCTS researchers are fulfilling their current operational parameter of progressing the rural health agenda through targeted research.This was seen by an overall increase in RCTS rural health publications plus an increase in the proportion of RCTS rural health publications making up all ARH publications in the decade 2004 to 2013.It is not surprising that total publication rates were low during the first few years from 2004 to 2006, as the focus of RCSs would have been primarily on establishing a quality medical curriculum for students 2 .It is encouraging, however, to see the upward trend in publications that has occurred throughout the decade, as the schools and academics became established and their focus was directed towards research as the funding parameter of rural research came into place.
ARH topics were the major focus of RCTS publications throughout the decade of our analysis.Around one-third of all RCTS publications related to medical education, and those with a rural context were more frequent than general medical education publications.The research focus on medical education aligns well with the initial objectives of the rural clinical schools of delivering one clinical year of their medical curriculum in rural/regional settings.A large proportion of RCTS publications (37%) are aligning with the current NHPA 11 and NRHAPA 12 .In particular, RCTS research is addressing areas such as mental health, diabetes, cardiovascular disease and aged care.However, some other rural health priority areas such as obesity and oral health might need a greater focus 13,14 .RCTS publications were also detected that did not fit this review's criteria for rural health research 3 and were categorised as 'other' research.In spite of this, many of these publications also addressed NHPA and NRHAPA.Although not in a strict rural context, these research publications are of relevance to the rural health agenda, since this general research will most likely be used to inform more specific rural research of this nature.Further reasons for publication of 'other' research from RCTS may be: (a) collaborations with metro centres 15 ; (b) researchers with urban backgrounds who continue working on their research interest that started while in urban settings; and (c) RCTS staff continuing to work in their primary disciplines such as public health, general practice and informatics.
The authors note that among the top 10 journals that published RCTS papers, only two have a current impact factor of more than 3.The impact factor provides one means of assessing the quality of a journal and is sometimes used to infer the quality of the articles within.As such, it could be seen as one way of assessing the research coming out of the RCTS program.However, the journal impact factor alone is not the best indicator of the quality a journal article 16,17 although it is one of the indicators used to inform the Excellence in Research for Australia (ERA) 18 .A more extensive examination of citation rates per article, study design types and altmetrics 19 , such as number of downloads and sharing through social media, would provide a more holistic measure of research quality 16 .
The data shown here for the previous decade of RCTS research provide a useful benchmark to which future data can be compared.Continued assessment of RCTS research outputs against these parameters using the reproducible/transparent methods employed here would provide a useful point of reference for monitoring the performance of incumbent rural research academics, as well as providing a guide for new academics in targeting their research to the rural health agenda.The advantage of this approach is that it is focused on rural health research priorities, as opposed to research outputs in a more general context, and does not attempt to compare rural academics with those academics in metropolitan areas.In a recent study of the current RCTS workforce 9 , only 26% of participating academics indicated that they worked full-time, as compared to the 71% reported by the Federal Government for academics Australia-wide 20 .Furthermore, just over 58% of RCTS academics reported working at 0.5 full-time equivalent or less 9 .Given this disparity in working hours, it would be unhelpful to assess rural researchers against their urban counterparts.An approach that assesses their research against health priority areas seems more appropriate for this context.
The current analysis was limited to MEDLINE-indexed journals in PubMed.This approach could have missed some RCTS publications not indexed in MEDLINE.However, Australia's health and medical research has been established to have high international visibility 21 , and so the numbers missed should be relatively small.Sensitivity testing was carried out using publication listings for several RCSs and RMSs that are reported on the FRAME website 22 .All listed publications emanating from these institutions were retrieved using the authors' queries.However, it is uncertain how accurate this listing is because all RCS/RMS have not listed their publications on this website and therefore it cannot be ruled out that some RCTS publications may have been missed by these methods.
The classification of rural health and related publications was limited by the MEDLINE MeSH tree classification and indexing by librarians.In addition, publications are tagged by multiple MeSH terms, which can be multidisciplinary and not include 'rural health'.In these instances, recognised primary disciplines such as public health will be used and the context of rural or regional health may not be in its MeSH term.Visibility of RCTS research publications is critical to achieving their research objective of addressing the distinctive needs of rural populations to better inform rural health policy 3,4 .Ensuring author affiliations are accurate and include the institution title, along with the secondary department or unit, such as rural clinical school or regional medical school, will maximise visibility, particularly when evaluating RCTS outcomes using bibliometric methods such as those used here.Visibility of medical education publications could be further enhanced by including this term in the keywords and title/abstract and not relying on a MeSH keyword assignment.For example, it was found that searching for 'medical education'[MeSH] only detected around 30% of RCTS medical education publications.Therefore the method of detecting these articles had to be altered by manually searching all retrieved RCTS publications.

Conclusions
This first study on the research outputs of RCTS researchers has shown that RCTS researchers are making a valuable contribution not only to medical education research but to rural health research more broadly.These data represent a benchmark of current research strengths, and highlights research areas that need greater focus in a rural context.The research parameters analysed here also provide a means of monitoring and assessing the outputs of future research academics that take up positions at RCSs.As many newer RCTS continue to build their research capacity in the coming years, tracking the contribution of RCTS research to rural health research in this manner can provide a useful outcome measure.

Figure 1 :
Figure 1: Total PubMed publications by Rural Clinical Training and Support researchers from 2004 to 2013.

Figure 2 :
Figure 2: Medical education publications in PubMed by Rural Clinical Training and Support researchers from 2004 to 2013 (RCTS, Rural Clinical Training and Support).

Figure 3 :
Figure 3: Rural Clinical Training and Support publications from 2004 to 2013 in rural health services, rural population and Indigenous health.

Figure 4 :
Figure 4: Rural Clinical Training and Support publications addressing National Health Priority Areas.

Figure 5 :
Figure 5: Rural Clinical Training and Support publications addressing National Rural Health Alliance Priority Areas.
Medical subject heading (MeSH) terms 'rural health services' and 'rural population' were selectively enumerated by searching within the RCTS publication results to tabulate the respective subject areas (Table 1, queries 5 and 6).RCTS publications specific to Indigenous health (Table 1, query 7), National Health Priority Areas (NHPA; Table 1, query 8), and National Rural Health Alliance Priority Areas (NRHAPA; Table 1, query 9) were also enumerated (queries first published in Mendis et al 3 ).