Relationship of residency program characteristics with pass rate of the American Board of Internal Medicine certifying exam

Objectives To evaluate the relationship between the pass rate of the American Board of Internal Medicine (ABIM) certifying exam and the characteristics of residency programs. Methods The study used a retrospective, cross-sectional design with publicly available data from the ABIM and the Fellowship and Residency Electronic Interactive Database. All categorical residency programs with reported pass rates were included. Using univariate and multivariate, linear regression analyses, I analyzed how 69 factors (e.g., location, general information, number of faculty and trainees, work schedule, educational environment) are related to the pass rate. Results Of 371 programs, only one region had a significantly different pass rate from the other regions; however, as no other characteristics were reported in this region, I excluded program location from further analysis. In the multivariate analysis, pass rate was significantly associated with four program characteristics: ratio of full-time equivalent paid faculty to positions, percentage of osteopathic doctors, formal mentoring program, and on-site child care (OCC). Numerous factors were not associated at all, including minimum exam scores, salary, vacation days, and average hours per week. Conclusions As shown through the ratio of full-time equivalent paid faculty to positions and whether there was a formal mentoring program, a highly supervised training experience was strongly associated with the pass rate. In contrast, percentage of osteopathic doctors was inversely related to the pass rate. Programs with OCC significantly outperformed programs without OCC. This study suggested that enhancing supervision of training programs and offering parental support may help attract and produce competitive residents.

I n 2015, 26,252 residents were admitted to the first year of a residency program. The largest type of residency program Á categorical internal medicine Á has 6,698 positions (1). Medical students and physicians are required to pass multiple examinations (e.g., United States Medical Licensing Examination [USMLE], Comprehensive Osteopathic Medical Licensing Examination of the United States [COMLEX-USA]) to get into a residency program. Moreover, physicians must pass a board exam to be eligible to practice in their medical specialty. Likewise, to practice as an internist, a physician must pass the American Board of Internal Medicine (ABIM) certifying exam, which had an average pass rate for first takers of 86% from 2012 to 2014 (2). Failing this exam doubtlessly has an impact on a physician's career plan, as the exam is only given once a year.
Passing the ABIM exam is important for physicians regardless of their gender. In the past 30 years, the gender distribution of the global physician community has changed considerably; specifically, in both the United States and global contexts, the percentage of female physicians has been continuously increasing (3,4). Residency programs must therefore promptly adapt to this changed scenario. On the residents' side, numerous factors can relate to the selection process of residency programs, such as the location of the program, program type, educational tracks, or compensation. Some medical graduates might have specific preferences in this regard, such as a women's health track, on-site child care (OCC), or subsidized child care. However, as no studies have directly assessed how these program characteristics differently associate with the pass rate, it would be important to clarify this point.

Medical Education Online ae
Medical Education Online 2015. # 2015 Amporn Atsawarungruangkit. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license. Previous studies on what variables relate to pass rate have looked primarily at the relationship between pass rate and exam scores (i.e., for in-training examinations or USMLE) (5Á11) or the effect of duty hour reform (12Á14). Unfortunately, neither factor is directly related to the characteristics of a residency program, because the former is an important predictor of individual performance whereas the latter is a regulatory factor. Some evidence suggests that location and program size are associated with the pass rate of the American Board of Family Medicine (ABFM) (15) and American Board of Pediatrics (ABP) certifying exams (16). Moreover, recent studies have shown that the pass rate of the ABFM certifying exam was also associated with accreditation cycle length, opportunities for international experiences, and training in alternative medicine (17).
Besides location and program size, there is limited evidence on the relationships between pass rate and the characteristics of internal medicine residency programs. Thus, the objective of the present study is to evaluate the relationships between the pass rate of the ABIM certifying exam and the characteristics of three-year categorical internal medicine residency programs. Because the educational environment plays a crucial role in the success of medical education (18) and creating competitive residents (19), understanding these relationships will help to improve the quality of residency education and should be beneficial for various stakeholders, including program directors, residents, residency candidates, and patients.

Methods
This study used a retrospective, cross-sectional design to evaluate the relationships between pass rate of the ABIM certifying exam and most of the program characteristics available in the Fellowship and Residency Electronic Interactive Database (FREIDA † ), a freely available online database containing self-reported program characteristics. The scope of this study covered all three-year categorical internal medicine residency programs in the United States and Puerto Rico, a US territory. A list of three-year categorical internal medicine programs and their characteristics were extracted from FREIDA † using computerized automation on April 24, 2015. The 2012Á 2014 pass rates of the ABIM certifying exam, the most recent statistics at the time of the study, were obtained from the ABIM website. Residency programs that did not report their pass rates were excluded from the study.
The internal medicine residency programs were classified into 10 regions as listed in the FREIDA † (Table 1). Program size was defined as the average number of residency positions from postgraduate year 1 to 3. I used only the USMLE score requirements for interviews in this study as most residency candidates had taken this exam rather than the COMLEX-USA. Salary and vacation days were taken from first-year data only because this year had the greatest number of data observations. Hard-toquantify data (e.g., sick days, call schedules, and average USMLE Step 1 score) were not taken into account in this analysis. The number of program faculty members was excluded from the analysis because the ratio of full-time equivalent paid faculty to positions (FTP ratio) provides more meaningful information in this regard. Finally, the visa qualifications of international medical graduates and major medical benefits were excluded, as they did not seem much relevant to the pass rate. In total, 69 program characteristics in a variety of categories were considered in this study, such as location, general information, number of faculty and trainees, work schedule, educational environment, education benefits, education features, program evaluation, resident evaluation, employment policies and benefits, and compensation and leave. First, the descriptive statistics of the program characteristics were calculated. Then, to identify the relationships between pass rate and the program characteristics, univariate, linear regression analyses were performed under the assumption that all variables were normally distributed. Program characteristics with a p-value of less than 0.10 in the univariate analysis were included in the multivariate, linear regression analyses. Then, stepwise, multivariate, linear regression analysis was performed to identify the significant independent predictors of pass rate. The significance level (a) was 0.05. All statistical analyses were conducted using STATA version 13.0 (StataCorp).

Results
There were a total of 405 three-year categorical internal residency programs in FREIDA † at the time of study; however, only 371 programs (n0371) were included in the analysis because the other programs did not report a pass rate. As shown in Table 2, the pass rate of one region, a US territory containing only the state of Puerto Rico, was significantly lower than those of other regions (p B0.001). However, because the programs in this region did not report any other program characteristics in FREIDA † , I could not analyze any relationships between other characteristics and the pass rate. As such, the factor of location was dropped from further univariate and multivariate analyses.

Discussions
The study findings are distinct from those of similar studies focusing mostly on location and program size. Although the locations of residency programs have been reported to be significantly related with the pass rate of board certifying exams (ABFM and ABP) (15,16), the only region in this study found to have a significant relation with the pass rate was Puerto Rico. However, none of the residency programs in this region reported any other program characteristics. As a result, the location factor was excluded from further analysis. Therefore, to my limited knowledge, this is the first time that location has not been reported as a significant predictor of the ABIM pass rate.
Many investigators have previously demonstrated that program size is a significant predictor of the pass rate on board certifying exams of many specialties, including internal medicine (15, 16, 20Á22). In the univariate analysis, I confirmed that program size was also a significant predictor (pB0.001). However, in the multivariate analysis, FTP ratio was considered a better predictor of pass rate, which conforms to the results of one previous study (23). Compared to program size alone, the FTP ratio contains information from a number of faculty positions relative to program size, which is very similar to studentÁ faculty ratio. It is important to understand that studentÁ faculty ratio is a popular measure of educational quality in higher education (24) and is used by global ranking agencies such as QS Quacquarelli Symonds (25) and US News Ranking (26). In other words, a small residency program might not be a disadvantage as long as the program has a sufficient FTP ratio, which is a better indicator of educational quality.
Of the 295 programs that reported information on FMP, 277 programs (93.9%) offered it. Thus, even if a mentoring program in internal medicine is unstructured, under-monitored, or under-evaluated (27), it appears to have a significant positive relation with pass rate. Some previous researchers looked at the effectiveness of Table 2. ABIM pass rate and regional location of residency program  mentoring programs, commonly concluding that both residents and program directors had positive attitudes toward mentoring programs (27Á31). However, there are no previous investigations on the relationship between FMP and pass rate of board certifying exams. As such, this seems an interesting area of future exploration. Interestingly, the % DO was found to be the only significant negative predictor of the pass rate in the multivariate, linear regression. My findings are perhaps explainable by the fact that higher % DO in a given program is an indicator of lower competition in that program. As shown in the case of general surgery residency programs, more competitive programs are significantly more likely to select applicants with higher USMLE Step 1 scores (32), which are also a significant predictor of passing the ABIM certifying exam (8). Thus, the % DO can be seen as an inverse indicator of competitiveness. In this study, I also noted that the percentage of US medical graduates was positively associated with pass rate; however, it was not statistically significant. To my limited knowledge, there have been no empirical comparisons in academic performance between doctors of osteopathic medicine and doctors of medicine in the past.
Only 34.24% of internal medicine residency programs offered an OCC benefit, despite the fact that parenthood during residency is common together with the rising number of residents having babies during residency training (33Á35). Having an OCC or another parental support policy could be a critical factor for attracting competitive residents to a program. Indeed, one study on general surgery programs across Canada suggested that a lack of program-specific maternity/parenting policies could lead  to lower interest among candidates (36). Moreover, I found that programs with OCC had slightly lower % DO, which would further benefit the pass rate. Such evidence indicates that programs with OCC are more competitive because they can attract more US medical graduates. Several limitations of this study need to be addressed. First, the data from both the FREIDA † and the ABIM website could be subject to human error by the data reporters or data gatherers. Second, approximately onefourth of programs opted out of reporting program characteristics other than location to FREIDA. Although such reporting behavior could lead to further bias in the data, the difference in pass rate between fully opted-in programs and fully opted-out programs in this study was found to be non-significant. Third, the study used data from the FREIDA † at a single point of time, and thus the program characteristics and their relations with pass rate might differ by time period. Fourth, the adjusted R 2 of 9.61% means that 90.39% of the variance in pass rates is attributable to factors other than FTP ratio, % DO, FMP, and OCC. Further analysis with a larger dataset would be useful to validate the study results, especially regarding the influence of location using multivariate analysis. Despite the limitations of the data and low adjusted R 2 , the findings of this study can still be applied to most programs in the United States (except those located in Puerto Rico). Therefore, the results of this study have several implications for improving internal residency programs. First, programs should focus on improving the supervision of training experiences such as enhancing the quality of the mentoring program or balancing the faculty to position ratio. Second, programs should pay

Conclusions
According to the results, success on board certifying exams is associated with two main factors: the competitiveness of individual residents and the training environment. Doubtlessly these factors are related. Specifically, the significant findings about FTP ratio and FMP supported the benefits of a well-supervised training environment, whereas higher % DO had a negative effect on pass rate. Finally, the OCC was directly related to training environment in terms of quality of life. The result of this study suggested that internal medicine residency programs could better attract competitive residents into programs as well as produce competitive residents by enhancing the supervision of training environments and offering parental support.