Factors associated with attaining a pharmacy residency interview: A scoping review

Postgraduate pharmacy residencies are essential in preparing graduates for pharmacy careers. Residencies are highly competitive and despite efforts to increase the number of residency programs in the United States, the demand still outweighs the supply. Attaining a residency interview is a critical step in securing a residency position. The objective of this scoping review is to describe the published literature on factors associated with attaining a pharmacy residency interview. The updated Arksey and O'Malley scoping review framework was followed for this scoping review. The review is reported according to The Preferred Reporting Items for Systematic Reviews and Meta‐Analyses Extension for Protocols and Scoping Reviews (PRISMA‐ScR). The search strategy contained a combination of medical subject headings (MeSH) and keywords from a review of article title and abstracts. The inclusion and exclusion criteria were created a priori. English language studies describing pharmacy residency interview, selection, rubrics, and descriptive, quantitative, or qualitative data were included. Title and abstracts were independently reviewed, followed by independent full‐text reviews. Covidence©, an online systematic review platform, was used in the screening, review, and selection process. Thirteen studies assessing factors associated with a residency interview met inclusion criteria. The majority of these studies (n = 7) analyzed data from residency program director (RPD) surveys. Most of these studies found that RPDs highly value letters of recommendation when evaluating a candidate's application. Three studies analyzed retrospective data. Several studies focused on letters of recommendation. The studies described letters of recommendation, letters of intent, grade point average, and leadership experience as factors associated with attaining a residency interview. Most studies had limitations in generalizability, study design, and heterogeneous data.


| INTRODUCTION
2][3] Residencies are highly competitive with overall demand by applicants outpacing supply or number of available positions. 4During the 2023 American Society of Health-System Pharmacists (ASHP) Resident Matching Program, over 1000 applicants were left without a residency position after two phases of the residency match, and an additional 876 applicants registered for the match but withdrew from the process due, at least in part, to a lack of interview offers. 4ASHP reports an overall growth of 12% for open residency positions over the past 4 years.This highlights the continued and steady growth in the total number of available pharmacy residency positions. 5e competitive nature of the residency match process often causes stress and uncertainty for students in their final year of pharmacy school hoping to pursue postgraduate residency training.Early preparation, utilization of mentors, and individual applicant factors can play a role in candidates ultimately securing a residency position. 6fore candidates navigate the residency interview and match process, the first pivotal step is to secure an interview with a residency program.All residency program directors (RPDs) evaluating candidates have access to the same set of data through the use of a centralized residency application portal for their applicant pool, and they must ultimately decide which candidates to extend interview invitations with their respective programs.However, RPDs often evaluate and prioritize different components of the candidate applications specific to their program.In 2023, the ASHP Standard was updated to require programs to have pre-determined, objective criteria for determining which applicants would be invited to interview. 7However, the ASHP Standard does not define which criteria should be evaluated.While data exist on factors leading to overall success of attaining a residency position, there is a paucity of data surrounding factors that are associated with attaining the interview itself.The objective of this scoping review is to describe the literature published on factors associated with attaining a pharmacy residency interview.

| METHODS
][10] The Preferred Reporting Items for Systematic review and Meta-Analysis extension for Scoping Reviews was followed to ensure adherence to scoping review reporting guidelines. 11A protocol was not registered or uploaded before beginning the scoping review.

| Search strategy
The search strategy was developed with a collaborating librarian specialist (A.P.).A combination of medical subject headings (MeSH) and keywords were identified from a review of titles and abstracts of seed articles adapted from a previous review (Tables S1-S3). 6The search was conducted in PubMed, Web of Science's Science Citation Index Expanded (SCIE) and the Emerging Sources Citation Indices (ESCI), and International Pharmaceutical Abstracts on April 7, 2023.A filter for manuscripts published between 2012 and 2023 was applied to identify relevant articles of interest.This time frame was selected to best reflect the current residency landscape relative to the increase in the number of residency programs and the revision of two Residency Accreditation Standards.

| Study selection
Covidence© (Melbourne VIC, Australia), an online systematic review platform, was used in the screening, review, and selection process. 12 Phase I, two authors independently reviewed the titles and abstracts of identified studies for relevancy using pre-specified inclusion criteria, while a third author resolved discrepancies in study inclusion or exclusion (C.K., B.R.J., and S.W.).Studies meeting inclusion criteria were English language, primary literature in peerreviewed journals describing interview selection, pharmacy student interviews for residency, rubrics or tools for interview selection, and descriptive, quantitative, or qualitative data (Figure 1).In Phase II, two authors (R.S. and S.P.O.) reviewed the full text of studies included in Phase I using pre-specified inclusion and exclusion criteria.Excluded studies were those with a lack of a comparator group, insufficient data to determine outcomes, describing outcomes unrelated to attaining a residency interview, conducted in non-pharmacy professions, or non-US-based pharmacy residency programs (Figure 1).

| Data extraction
Following Phase II, two authors collectively extracted all data from included studies and inputted points of interest into an Excel spreadsheet (D.L. and S.W.).This spreadsheet was used to inform the development of the manuscript and tables.Data points extracted included authors, year, study objective, population, methods, and outcomes.

| RESULTS
The search identified 860 results for review, and 27 studies underwent full-text review.A total of 14 studies were excluded for study outcomes not related to attaining a residency interview (n = 10), and the presence of insufficient data to determine an outcome (n = 4).
For the studies deemed to have insufficient data available to determine outcomes, the authors of the studies were not contacted in an attempt to obtain data.18

| Retrospective data
Three single-center studies retrospectively evaluated residency applications for a specific residency program to determine the importance of certain applicant characteristics in attaining an interview.Although all of the studies evaluated different program types, some factors were associated with a higher likelihood of an interview offer across all programs, including grade point average (GPA), and leadership/ professional organization involvement. 15,16,24Work experience, rotation experiences, and poster presentations at a national meeting were also commonly identified.
Ensor and colleagues reviewed 277 applicants during two residency recruitment cycles for a Postgraduate Year (PGY) 1 Pharmacy residency to determine characteristics associated with a higher likelihood of an interview offer. 24Applicant characteristics reviewed included rotation experiences, letters of recommendation, professional organization involvement, GPA, community service, professional awards/scholarships, research experience, presentations, skills/certifications, pharmacy work experience, general leadership experience, and publications. 24Each of these characteristics were rated on a scale of 0-5 by two individuals and the scores were added together for a total possible score in each category of 10 and 130 overall.Those offered interviews scored significantly higher in all categories when compared with those who did not ( p < 0.001) when utilizing a univariate model. 24In the Final Logistic Regression Model, it was determined that pharmacy work experience, professional organization involvement, rotation experiences, publications, presentations, skills/certifications, and GPA were all more strongly associated with those who received interviews than those who did not (p < 0.05). 24Laughlin and colleagues examined 110 applicants over two recruitment cycles to a PGY1 pharmacy residency program conducted in an ambulatory care setting by a college of pharmacy. 15The authors found a higher GPA correlated with being offered an interview (3.59 vs. 3.4; p < 0.01). 15Also, excellence in academics (defined as higher grades in courses related to clinical science), holding leadership positions, strength of rotations, interest in both teaching and ambulatory care, and professionalism were all rated higher in those who received interviews than those who did not ( p < 0.01). 15imilar to both Ensor and colleagues 24 and McLaughlin and colleagues, 15 Morbitzer and Eckel found that a higher GPA was associated with a higher likelihood of interview attainment (p < 0.001). 16In addition, hospital intern experience, presenting a poster at a national meeting, holding a leadership position in national pharmacy organizations, and number of scholarships received in pharmacy school were all found to be significant predictors of interview attainment (p < 0.05). 16wever, the factors most highly associated with interview attainment were pharmacy school GPA, hospital intern experience, and total scholarships received in pharmacy school. 16

| Residency program director data
1][22][23] Surveys gathered responses from RPDs of PGY1 pharmacy residencies, 19 PGY1 community pharmacy residencies, 17 combined PGY1/PGY2 residency programs, 19 and at a consortium of academic medical centers. 25[20][21]25 There were mixed opinions regarding GPA, ranging from perceptions of being an important characteristic by more than 90% of RPDs 20 to perceptions of somewhat or neutral importance in two other studies.21,25 Several studies found leadership experience ranked in the top third of important factors.[19][20][21]25 Claiborne and colleagues found 72.8% of RPDs saw student membership in professional organizations as somewhat important while 56.3% of RPDs found student leadership to be very important. 22 The mjority of these RPDs also agreed they would rank high-level leadership over mid-level leadership as well as preferred depth (high-level leadership in one organization) over breadth (mid-level leadership in multiple organizations).22 Previous work experience had variable levels of importance across the studies.[19][20][21]25 In studies for which work experience was differentiated, hospital work experience was deemed as more important to hospital-based programs than work experience in the community or other areas.21,25 Multiple studies ranked rotations as an important factor when selecting candidates for interviews, [19][20][21]25 with three of the studies further differentiating rotations into clinical Advanced Pharmacy Practice Experiences (APPEs), type of clinical rotation, or quality of rotation.20,21,25 Clarke and colleagues found when selecting candidates for residency interviews, the most important factor related to APPEs was the reference from the APPE preceptor.This was followed by the location, structure, elective type, and timing of the APPE.23 Regarding the APPE location site for residency applicants, a University-based hospital or health system was the most desired, closely followed by the location type of the participating RPD's program.23 RPDs equally preferred balanced inpatient/outpatient APPEs, more inpatient than outpatient APPEs, letter grades for APPEs, and longitudinal APPEs at the same site where the residency is based.23 RPDs found transitions of care, infectious disease, and critical care APPEs as more desirable, and nuclear pharmacy, pharmaceutical industry, and veterinary pharmacy as less desirable. 23 RPs equally preferred preceptor references from someone they knew or from alumni of the program.Residency program directors preferred references from two APPE preceptors over only one APPE preceptor. 23 The last important factors chosen by RPDs varied widely among the studies.Hillebrand and colleagues found that honors, scholarships, volunteer experience, college of pharmacy, additional degrees, and work experience in the outpatient/community setting were neutral to somewhat less important to hospital-based programs when ranking a candidate for an interview.25 For PGY1 programs, Blake and colleagues found less than 50% of RPDs utilize advanced degrees, research experience, and prior experience with the program as selection criteria, 20 while Cho and colleagues found that ASHP residency showcase attendance, non-pharmacy-related work experience, and pharmacy placement services participation as the least important criteria.21 The study by Scalise and colleagues differed in characteristics most important when selecting an applicant to interview, likely due to it being the only study based on community pharmacy residencies.17 Scalise and colleagues found the majority of Community PGY1 RPDs selected pharmacy work experience and letters of recommendation as the most important characteristics.17 Community PGY1 RPDs deemed APPE rotation experience, skills and certifications, presentations, research experience, pharmacy programs attended, and publications as the least important characteristics with less than 15% of Community PGY1 RPDs selecting these as important.17

| Candidate references
4][15] Atiya and colleagues analyzed the impact of references on the interview status for PGY1 program applicants to academic medical center-based programs. 13For residency application cycles 2015-2018, a total of 5923 references were listed on a total of 1867 applications.The references were reviewed across 13 prespecified areas reflecting on the candidates' attributes. 13Nearly 75% of all characteristics were scored as exceeding expectations, and very few characteristics were rated as failing to meet the level expected to enter a residency program (0.08%). 13Of all letters of recommendation reviewed, 24.8% rated the applicant as exceeding expectations in all 13 areas and over 90% highly recommended the applicant.Average applicant characteristic ratings were found to correlate poorly with an interview offer (interview offered: R 2 = 0.07, p < 0.0001; no interview offered: R 2 = 0.11, p < 0.0001).Additionally, overall reference ratings correlated poorly with interview status (interview offered: R 2 = 0.04, p < 0.0001; no interview offered: R 2 = 0.06, p < 0.0001). 13Laughlin and colleagues conducted a similar survey focused on specific characteristics of letters of recommendation comparing those who received interview invitations with those who did not for a PGY1 Pharmacy residency program in an ambulatory care setting. 14 were more likely to receive interviews. 14Those with higher overall recommendation letter scores (3 [3] vs. 3 [2.7-3])and leadership scores (3 [2.7-3] vs. 2.7 [2.3-3]) were more likely to receive an onsite interview. 14rupky and colleagues conducted a survey of 1222 PGY1 Pharmacy residency RPDs in 2018. 18Of the 291 respondents, 277 completed the portion of the survey regarding letters of recommendation.
Over 75% of respondents rated letters of recommendation as quite or extremely important.The nature of the author's relationship with the residency candidate was noted as important, as well as the nature of contact (88%), student responsibilities (78%), and duration of relationship (57%).Letters of recommendation that were accurate (98.2%) and detailed (90.2%) were rated as quite or very important, specifically those that discussed strengths (94.5%) and areas of improvement (96%). 18When completing the required components in the letter of recommendation, 95% of respondents noted the writer's evaluation of those components impacted the likelihood of an interview offer. 18tters of recommendation where "exceeds" was selected had a favorable impact on the offer of an interview, whereas selecting "failing to meet the level expected to enter a residency program" negatively impacted an interview offer.The overall evaluation was reported to impact the offer of an interview by 87% of respondents. 18

| DISCUSSION
Many studies evaluating factors associated with pharmacy residency interview attainment have been published in the literature which provide data from different perspectives.This landscape of available literature can be grouped into three main focus areas: a single-center retrospective evaluation of candidate applications over two to three recruitment cycles, a survey of RPD perceptions, and an evaluation of the impact or perception of references.Although much information can be extracted from these studies, factors impacting the generalizability of the studies include relatively small sample size, sampling bias, and insensitivity to detect differences in candidate characteristics through references.Further, survey data based on perceptions may not be representative of actual pharmacy residency interview attainment.
Several studies have evaluated factors associated with attaining a residency interview using various methodologies with inconsistent results.21][22][23]25  While retrospective studies 15,16,24 provide valuable insight into characteristics of those selected for on-site interviews, the difference in the program types, PGY1 residency in a hospital or ambulatory care setting or a PGY1/PGY2 health system administration residency, and relatively short study period make it difficult to extrapolate the results.While there was overlap in some characteristics identified as important (e.g., GPA, leadership), most of the characteristics identified were specific to or deemed important by that program type, and generalizability is limited.In addition, all of the studies were conducted at a single center and may not be a representative sample of residency programs across the country.Future studies should aim to collect data from many different programs and program types over several recruitment cycles to allow for wide application of results to better prepare pharmacy students to be competitive residency candidates.
21]25 While RPDs deemed references as a critical factor in selecting applicants for interviews, the components that were valued varied in each study.Skrupky and colleagues provide suggestions to improve references for both the writer and the centralized application service. 18Suggestions to writers include only writing letters of recommendation for candidates that you can write a favorable letter for, provide context of the relationship with the applicant, and being honest with examples of strengths and areas for improvement. 18Suggestions for the centralized application service include providing more guidance to writers, allowing programs to customize their letter of recommendation, and modifying the letter of recommendation form to require a higher number of comments. 18nally, it should be noted that all of these studies were completed before the recent focus on diversity, equity, and inclusion within the profession that started in 2020. 26Additionally, residency programs are now required to update their recruiting strategies to promote diversity. 27It is unknown whether this emphasis has changed factors that determine interview attainment.Future research is needed to determine whether factors such as GPA, work experience, leadership roles, and others are still highly valued in the interview selection process.
There are several strengths and limitations associated with this scoping review.By closely following the Arksey and O'Malley scoping review framework, this paper provides a systematic, rigorous, and transparent process for reporting published literature on this topic. 8,28e inclusion criteria were broad and did not narrow the information to a particular type of pharmacy residency.Beyond limitations in the application of the study results discussed previously, another area Retrospective studies evaluating pharmacy residency applications.Pharmacy residency program director surveys.Impact of references on pharmacy residency interview.
F I G U R E 1 PRISMA-ScR diagram.PRISMA-ScR, Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Protocols and Scoping Reviews.T A B L E1• Pharmacy school GPA was significantly higher in those offered onsite interviews than those not offered onsite interviews (3.59 ± 0.045 vs. 3.40 ± 0.034; p < 0.01) • Those receiving interviews scored higher in academics (strong grades in courses related to clinical sciences), leadership, strength of rotations, interest in teaching, interest in ambulatory care, and professionalism than those who did not receive interviews ( p < 0.01 for each) • Pharmacy school GPA was significantly higher in those offered onsite interviews than those not offered onsite interviews (3.63 [IQR: 3.46-3.79]vs. 3.35 [IQR: 3.2-3.49],p < 0.001) • Hospital experience was also more prevalent in those who received interviews (49 [70%] vs. 17 [38%], p = 0.001) Abbreviations: CI, confidence interval; GPA, grade point average; HSPA, health system pharmacy administration; IQR, interquartile range; NS, not statistically significant; OR, odds ratio; PGY, postgraduate year; UNC, University of North Carolina.T A B L E 2 Abbreviations: ACCP, American College of Clinical Pharmacy; APhA, American Pharmacist Association; APPE, advanced pharmacy practice experience; ASHP, American Society of Health-System Pharmacists; COP, college of pharmacy; CRPD, community residency program director; GPA, grade point average; LOI, letter of intent; LOR, letter of recommendation; NAPLEX, North American Pharmacy Licensure Examination; PGY, postgraduate year; Pharm.D., Doctor of Pharmacy; RPD, residency program director; UHC, University Health Consortium.T A B L E 3 Abbreviations: IQR, interquartile range; LOR, letter of recommendation; PGY, postgraduate year; RPD, residency program director.
the inconsistencies in the data.It is difficult to know if all of the surveys would have yielded the same results from RPDs if they had the same characteristics listed in the surveys, so general trends are used to identify the most common themes within the critical factors.The least important factors chosen by RPDs varied widely among the studies, likely due to the variability in the surveys as well.Another limitation of the RPD data is that while RPD perceptions of desirable or important characteristics are valuable, it is unclear how much impact this has on which applicants are invited to interview.
While these studies have identified common themes of important characteristics including letters of recommendation, letters of intent, GPA, and leadership experience, the variability in survey selection options is the driving force for