Improving Transparency in the Residency Application Process: Survey Study

Background Increasing numbers of residency applications create challenges for applicants and residency programs to assess if they are a good fit during the residency application and match process. Applicants face limited or conflicting information as they assess programs, leading to overapplying. A holistic review of residency applications is considered a gold standard for programs, but the current volumes and associated time constraints leave programs relying on numerical filters, which do not predict success in residency. Applicants could benefit from increased transparency in the residency application process. Objective This study aims to determine the information applicants find most beneficial from residency programs when deciding where to apply, by type of medical school education background. Methods Match 2023 applicants voluntarily completed an anonymous survey through the Twitter and Instagram social media platforms. We asked the respondents to select 3 top factors from a multiple-choice list of what information they would like from residency programs to help determine if the characteristics of their application align with program values. We examined differences in helpful factors selected by medical school backgrounds using ANOVA. Results There were 4649 survey respondents. When responses were analyzed by United States-allopathic (US-MD), doctor of osteopathic medicine (DO), and international medical graduate (IMG) educational backgrounds, respondents chose different factors as most helpful: minimum United States Medical Licensing Examination (USMLE) or Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Step 2 scores (565/3042, 18.57% US-MD; 485/3042, 15.9% DO; and 1992/3042, 65.48% IMG; P<.001), resident hometown region (281/1132, 24.82% US-MD; 189/1132, 16.7% DO; and 662/1132, 58.48% IMG; P=.02), resident medical school region (476/2179, 22% US-MD; 250/2179, 11.5% DO; and 1453/2179, 66.7% IMG; P=.002), and percent of residents or attendings underrepresented in medicine (417/1815, 22.98% US-MD; 158/1815, 8.71% DO; and 1240/1815, 68.32% IMG; P<.001). Conclusions When applying to residency programs, this study found that the factors that respondents consider most helpful from programs in deciding where to apply differ by educational background. Across all educational groups, respondents want transparency around standardized exam scores, geography, and the racial or ethnic backgrounds of residents and attendings.


Introduction
An increasing number of residency applications has made it harder for programs and applicants to assess if they are a good fit during the residency application and match process [1].Residency application volumes continue to increase [1][2][3] without a change in match outcomes [4].This increase in applications burdens applicants and residency programs in time spent [5], costs [5][6][7], and stress [8].Applicants face limited or conflicting information as they assess programs, leading to overapplying [1,5].A holistic review of residency applications is considered a gold standard for programs [9,10], but the current volumes and associated time constraints leave programs relying on numerical filters [1,5,9], which do not predict success in residency [11][12][13].There are solutions proposed to improve the residency application process, including a cap on the number of applications submitted or interviews accepted [1], preference signaling [14], and a multiphase match process [14][15][16][17].A common theme among these solutions is increasing transparency around program selection criteria [1,2,9].Current resources for applicants are fraught with biases or missing data and discrepancies [18][19][20][21].This incorrect information is further compounded by mixed messaging from medical school advisors [2,8].Increased workplace transparency leads to greater job satisfaction [22,23], trust [20], psychological safety [24], decreased intent to leave [23,24], and better commitment and performance [23].As applying for residency is a process similar to a job application, it should be as transparent and equitable as possible to help both applicants and programs.
Still, increasing transparency is only helpful if we know what information to provide.To our knowledge, no study has identified what information residency applicants would find most helpful in determining where to apply.We asked respondents to select the top 3 factors from a multiple-choice list of what information they would like from residency programs to help determine if the characteristics of their application align with program values.Important factors were used in the survey based on program directors' reports of factors applicants are evaluated on [25].An open-ended free response option was included among the listed important factors (Multimedia Appendix 1).A copy of the survey is available in Multimedia Appendix 1; it contains a total of 14 items.Respondents were able to review and change their answers through a Back button.Results from the other questions asked in the survey were analyzed in separate forthcoming work.

Statistical Analysis
ANOVA was used to assess differences in helpful factors selected by educational background.Free-response factor answers were included in the analysis as the "other" category.SPSS Statistics (version 25; IBM Corp) was used to assess statistical significance (P<.05).

Ethical Considerations
The study was reviewed and deemed exempt by the Washington University in St Louis Institutional Review Board.Privacy and confidentiality were protected through anonymous study data.Participants voluntarily consented to the study, and compensation was not offered.Contact information for the research team and human research protection office were provided.

RenderX
When looking at the top 3 factors that would be most helpful to decide where to apply, there were differences in the percentages of respondents  2 shows differences in all factors.A higher percentage of URiM respondents across all educational backgrounds consistently selected the "percentage of URiM residents and attendings" as factors that would be most helpful to decide where to apply, compared to non-URiM respondents (903/1283, 70.38% URiM vs 655/2698, 24.28% non-URiM; P<.001) ranked it as an important factor.

Principal Results
The factors that respondents find most helpful in deciding which residency programs to apply to differ by educational background (US-MD, DO, and IMG), but the most highly sought are around USMLE or COMLEX Step 2 scores, hometown regions of residents, and the racial or ethnic background of residents and attendings.Some of this information is available on databases like the Fellowship and Residency Electronic Interactive Database (FREIDA) [27], the Orthopaedic Residency Information Network (ORIN) [28], and program websites, but some of it is outdated [18][19][20][21].Given that some programs screen applications by test scores [1,5,9], it would be helpful for applicants to have updated information about a program's review criteria.Geographic preferences may also play a large role in the match process [29].For the 2021 Match, there was a significant increase in matched applicants to their home orthopedic surgery and integrated plastic surgery programs [30,31].Additionally, programs likely consider that more than half of residents practice where they train [32].Consequently, increased transparency about geographic interests as an optional addition to the application could benefit programs to improve resident retention over time and for applicants to show interest in specific areas of the country.In the Match 2023, a total of 16 specialties asked about geographic preferences in the supplemental application, and program directors expressed positive feedback on its inclusion [33,34].As of the Match 2024, the optional geographic interests section has been expanded to all specialties, representing an initial step in allowing applicants to express interest in those programs who consider geography important in their decision-making process.
Structural racism persists in medical training and match processes [35].Residents and attendings who identify as URiM routinely experience bias at work and are relied on to fulfill a lack of standardized systems to promote diversity [36].Although a higher percentage of URiM respondents selected the percentage of URiM faculty and residents as important in assessing fit, many non-URiM applicants selected it, too.Both groups likely value a safe and inclusive work environment for themselves or their colleagues.It is important to recognize various intersectional aspects of diversity, equity, and inclusion in the match to mitigate perceived (and real) risk for applicants; however, there is little best practice data for programs to counteract inequities of the application process.Some strides toward greater equity and inclusivity for the match include the addition of pronouns to the Electronic Residency Application Service.In addition, some programs may choose to screen out identifiers such as date of birth, headshots, and hometown when reviewing applications [37].While additional data can benefit applicants, it can also help advisors provide tailored information.Metrics such as test scores, clinical grades, and experiences from previous applicants can be used to guide applicants on programs to apply to.This method of advising can help applicants strategically apply where they are likely to match, which may decrease anxiety, expense, and overapplying [38].Growing internal and external databases like FREIDA [27] or the Texas Seeking Transparency in Application to Residency (STAR) [39], which contain aggregated data from recently matched medical students, can provide additional guidance.Expanding access to the Texas STAR by removing the requirement that medical schools must share their data to gain access and increasing IMG-affiliated medical schools' participation, may increase transparency and improve match outcomes.Expanding specialty-specific databases to all specialties would be helpful, like ORIN [28] for orthopedic surgery and the Alignment Check Index [40] for obstetrics and gynecology.Using information from this study, updating these databases with the information applicants are most interested in may provide a better assessment of applicant fit and decrease overapplying.In addition to the percentage of URiM residents and attendings, which applicants selected as a helpful factor, XSL • FO RenderX programs can update their website and centralized databases with greater diversity, equity, and inclusion content [41], including information about URiM away rotation scholarship opportunities [42].

Limitations
Limitations of this study include self-selection bias through the use of social media to publicize the survey.However, the study has a large applicant pool; we detected differences among respondents based on educational backgrounds.

Conclusions
There have been numerous calls to improve the residency application and match process as application volumes increase.
A common theme in the solutions proposed is improving transparency and equity in the process, but only some have highlighted how.Although we found differences across educational backgrounds in the top factors that respondents would find most helpful from programs in deciding where to apply, many want information on the hometown region of residents, test scores, and the diversity of programs, which programs can consider adding to centralized residency databases.Increasing transparency may help applicants determine the programs that align best with their application and interests and improve the guidance from medical school advisors.

Figure 1 .
Figure 1.Factors applicants selected as most helpful in assessing program fit and where to apply.* Denotes a difference in the percentage of applicants selecting each factor across the 3 educational backgrounds with a P<.05.COMLEX: Comprehensive Osteopathic Medical Licensing Examination; DO: doctor of osteopathic medicine; IMG: international medical graduate; URiM: underrepresented in medicine; US-MD: United States-allopathic; USMLE: United States Medical Licensing Examination.

Table 1 .
Characteristics of applicants, including doctor of osteopathic medicine (DO) students, United States-allopathic (MD) students, and international medical graduates (IMGs).

Table 2 .
Factors applicants (N=4649) selected as most helpful in assessing program fit and where to apply.
c DO: doctor of osteopathic medicine.d URiM: underrepresented in medicine.e USMLE: United States Medical Licensing Examination.f COMLEX: Comprehensive Osteopathic Medical Licensing Examination.