The Value of Formal Research Training in Orthopaedic Surgery Residency

Introduction The Accreditation Council for Graduate Medical Education (ACGME) requires that orthopaedic surgery residencies "maintain...an active research component." Nevertheless, there are barriers to resident participation in research. The purpose of this study was to identify the number of orthopaedic residency programs that have research curricula and to determine the effect of such programs on resident research. Methods The study group consisted of programs whose website included information about research curricula or training in research methods. An unpaired t-test was used to compare the mean number of publications between PGY-4 residents in the study and control groups and the mean number of publications between programs in the study group that offered protected research time versus those that did not; the t-test was two tailed with an alpha of 0.05. Results Twenty-seven programs, with a total of 128 PGY-4 residents, met our inclusion criteria. A random sample of 27 orthopaedic surgery residency programs with a total of 127 PGY-4 residents formed the control group. The mean number of publications was 1.6 for the study group and 1.30 for the control group. The difference in mean number of publications between the two groups was not statistically significant (p=0.22). Layon D, Wang C, Su M, Mulcahey M MedEdPublish https://doi.org/10.15694/mep.2017.000146 Page | 2 Discussion Although studies at orthopaedic residency programs in the United States demonstrate that research curricula correlate with increased resident publications, our data did not support these findings. Further inquiry is needed to identify the how to improve orthopaedic resident research productivity.


Introduction
The Accreditation Council for Graduate Medical Education (ACGME) requires that orthopaedic surgery residencies "maintain…an active research component." Nevertheless, there are barriers to resident participation in research. The purpose of this study was to identify the number of orthopaedic residency programs that have research curricula and to determine the effect of such programs on resident research.

Methods
The study group consisted of programs whose website included information about research curricula or training in research methods. An unpaired t-test was used to compare the mean number of publications between PGY-4 residents in the study and control groups and the mean number of publications between programs in the study group that offered protected research time versus those that did not; the t-test was two tailed with an alpha of 0.05.

Introduction
The Accreditation Council for Graduate Medical Education (ACGME) requires that orthopaedic surgery residency programs "establish and maintain an environment of inquiry and scholarship with an active research component." Despite this mandate, there are significant barriers to conducting research during residency, including a lack of training in research methodology (Merwin, Fornari, & Lane, 2014). In addition, there is such a dearth of dedicated orthopaedic physician scientists that they have been described as an "endangered species" (Hurwitz & Buckwalter, 1999).
Several orthopaedic surgery residency programs in the United States have attempted to improve resident research productivity by developing "research teams", clinical research programs which target both faculty and residents, or by instituting formal research curricula (Konstantakos, Laughlin, Markert, & Crosby, 2010;Merwin et al., 2014;Robbins, Bostrom, Marx, Roberts, & Sculco, 2013;Torres, Gugala, & Lindsey, 2015). These studies as well as similar studies in non-orthopaedic surgery residencies have identified several key components of effective research programs, including the presence of a formal research curriculum, training in research methods, and protected research time, among others (Table 1) (Hebert, Levine, Smith, & Wright, 2003;Merwin et al., 2014).
To our knowledge, there are no studies in the literature that have evaluated the effects of research curricula on orthopaedic surgery resident academic productivity. The purpose of this study was to determine the number of allopathic orthopaedic residency programs that have a formal research curriculum and to assess whether residents who attend these programs publish peer-reviewed articles, on average, more frequently than residents who attend programs without formal research curricula.

Methods
A list of all ACGME accredited orthopaedic surgery residency programs in the United States was obtained from the Fellowship and Residency Electronic Interactive Database (FREIDA). The program websites were accessed between April 20, 2016 and June 1, 2016 and were queried using the following search terms, "training in research methods" or "research curricula." Residency programs were included in the study group (programs with research curricula) if their website specifically mentioned research curricula or training in research methods. Other data collected from the program website included dedicated research time (either optional 1 year research tracks or protected time), mandatory research projects, designated funding, formal research director, formal mentorship, support from program director, and opportunities to present research. If the aforementioned data were not found on the program's website, they were marked "not reported".
Postgraduate year 4 (PGY-4) residents' names from programs with formal research curricula were searched in PubMed to identify the number of publications during residency (determined by author affiliation, Layon D, Wang C, Su M, Mulcahey M MedEdPublish https://doi.org /10.15694/mep.2017.000146 Page | 3 submission/publication dates, and author credentials). PGY-4 residents were selected for two reasons 1) they are senior enough to have completed and published research projects and 2) the majority of dedicated research time for programs occurs before PGY-4 year. Inclusion criteria were: ACGME accredited orthopaedic surgery residency programs with publicly accessible, functioning websites.
Exclusion criteria were: Non-ACGME accredited orthopaedic surgery residencies, residency programs which did not have publicly available resident names or class years, and programs without functioning websites. A random sample of orthopaedic surgery residency programs without research curricula were used as a control group. Data collected from the control group consisted of only the number of PGY-4 residents and the number of publications by each PGY-4 resident, using the same method described above. This sample was obtained by assigning a random number to all orthopaedic surgery residency programs that were not included in the study group. The programs were then sorted by number, creating a random sample from which the first 29 programs were selected.
An unpaired t-test was used to compare the mean number of publications between PGY-4 residents in the study and control groups and the mean number of publications between programs in the study group that offered protected research time versus those that did not; the test was two-tailed with an alpha of 0.05. All statistical analysis was performed using Microsoft Excel 2013 (15.0.4841.1000) Analysis ToolPak.

Results
Of 158 ACGME accredited orthopaedic surgery residency programs in the United States, 27 (17%) programs with formal research curricula met our inclusion criteria ( Table 2). These 27 programs had a total of 128 PGY-4 residents. The control group consisted of 27 orthopaedic surgery residency programs with a total of 127 PGY-4 residents.
The total number of publications in the study group was 202 while the total number of publications in the control group was 165. The mean number of publications was 1.6 (SD: 3.0) for the study group and 1.30 (SD: 2.70) for the control group. There was not a statistically significant difference between the mean number of publications between the two groups (p=0.22). Among the 27 programs with formal research curricula, 20 (74%) had either optional or mandatory protected research time. Of the 20 programs that had protected research time, 5 (25%) offered optional or mandatory one-year long research programs.
All 22 programs with protected research time offered the time before or during the PGY-4 year. There was a statistically significant difference in the number of publications by PGY-4 orthopaedic surgery residents at programs within the study group that offered protected research time compared to PGY-4 residents at programs which did not offer protected research time (p = 0.005). Among the 128 residents in the study group, 69 (54%) had no publications at the time of data collection; while 77 residents in the control group (61%) had no publications.

Discussion
Despite the ACGME requirement to "establish and maintain an environment of inquiry and scholarship with an active research component," few orthopaedic surgery residency programs have formal research curricula. Several single institution studies have demonstrated that dedicated training in research methodology can improve resident publication rates, grant funding, and number of academic presentations (Konstantakos et  orthopaedic surgery residency are more likely to continue publishing after residency and to pursue careers at academic centers (Macknin, Brown, & Marcus, 2014;Namdari, Jani, Baldwin, & Mehta, 2013). Although multiple studies performed at orthopaedic surgery residency programs have shown that formal research curricula increase resident publication rates, our data did not support a statistically significant difference in number of publications between PGY-4 residents in orthopaedic surgery residencies with research curricula as compared to PGY-4 residents in orthopaedic surgery programs without research curricula. These results suggest that training residents in research methods via formal research curricula alone is insufficient to increase the scholarly output of orthopaedic surgery residents. There was, however, a statistically significant difference in the mean number of publications by PGY-4 residents between programs with dedicated research time versus those without dedicated research time, These results are supported by a survey-based cross-sectional study of Canadian orthopaedic surgery residents, which identified a statistically significant association between number of dedicated research months and academic productivity, as measured by grants obtained (p=0.048) as well as number of publications (p=0.012) (Chan, Lockyer, & Hutchison, 2009). There are studies in the literature, however, that have reported the opposite finding. A study by Krueger et al. (2016), which was unable to find a statistically significant association between protected research time and number of peer-reviewed publications despite examining both residents as well as faculty at three orthopaedic surgery residency programs (Krueger et al., 2016).
Our study has several limitations. The first, and most important, is that the data set relied on publicly available information. It is possible that programs have research curricula, but do not include information about it on their websites. For example, researchers at the Hospital for Special Surgery (HSS) published an article detailing the creation of a formal research curriculum at their institution in 2013 (Robbins et al., 2013). Nevertheless, during our data collection period there was no indication of such a program on the HSS's website. Furthermore, for those programs with formal research curricula, it was not necessarily clear how long the program had been in place or for that matter what the degree of institutional "buy in" was. The "seniority" of a program and the level of support for the program by faculty and residents may have influenced the total number of publications by PGY-4s at that institution. Programs with more established research curricula, may in theory have better infrastructure and hence have higher number of publications by their senior residents. Another weakness was our reliance on PubMed for identifying publications. Our data do not, therefore, take into account articles that are "in press" (i.e. accepted for publication, but not indexed by PubMed) or those published in non-PubMed indexed journals.
To our knowledge, no previous studies have examined the association between resident research curricula and academic productivity in orthopaedic surgery residency programs at a macro scale. The results of this study suggest that the presence of a research curriculum alone is insufficient to stimulate higher academic productivity in the form of publications. Simply put, "research output" may be an inadequate measure of the success of a research curriculum. It is possible that although the number of publications may not necessarily increase, residents who are trained in research methods may publish higher quality studies. A study by Torres et al. found a statistically significant increase in the impact factor of published articles after implementing a formal research curriculum for their orthopaedic surgery residents (Torres et al., 2015). The majority of existing literature supports an association between increased resident research output in orthopaedic surgery residency programs that offer protected research time or training in research methods via formal research curricula (Chan et al., 2009;Konstantakos et al., 2010;Robbins et al., 2013;Vinci et al., 2009). Promising future directions include the development of innovative programs such as using a "point based system" to reward resident research productivity (Chang & Mills, 2013). Orthopaedic surgery residency programs may benefit from offering dedicated research time to stimulate research productivity and support the development of more clinician scientists.