Multiple mini-interview scores of medical school applicants with and without rural attributes

Introduction: Students from rural areas are under-represented in medical schools. Concerns have been raised about rural applicants' qualifications relative to those of their urban counterparts, and the impact such potential differences in competitiveness may have on their under-representation. Although studies have reported no differences in Grade Point Average (GPA) and Medical College Admission Test (MCAT) scores between applicants with and without rural attributes, to date no study has assessed if performance on the multiple mini-interview (MMI) varies between the two groups. Methods: The MMI scores of 1257 interviewees for admission to the MD program at the Faculty of Medicine, University of Manitoba, in years 2008 to 2011, were studied for an association with graduation from a rural high school and attributes in the following three domains: rural connections, employment in rural areas, and rural community service. Results: There were 205 (16.3%) rural high school graduates among interviewed applicants. Rural high school graduates scored significantly lower (mean of 4.4 on a scale of 1 to 7; p<0.05) than urban high school graduates (4.6). Among rural-attribute domains, those with rural community service alone had the highest MMI scores (4.9) while those with rural connections alone had the lowest scores (4.3; p=0.016). After adjusting for demographics, GPA, and MCAT scores in a multiple linear regression model, rural-attribute domains were not significant predictors of an applicant's MMI score. However, graduation from a rural high school was significantly associated with decreased MMI scores (a 0.122 decrease in predicted MMI scores on a scale of 1 to 7). Conclusion: Despite graduates from rural and urban high schools having comparable GPA, there exists a rural–urban divide in MMI scores that could exacerbate the under-representation of rural students in medical schools. Aboriginal applicants can also potentially be disproportionately affected, as they were more often from rural high schools than from urban high schools. Future studies need to determine systematic and institutional reasons, if any, for the differential in MMI scoring that can affect admission


Introduction
Physician shortage and maldistribution in rural areas is a critical issue worldwide 1 .An important predictor of a physician's choice to practice in a rural location is the rural origin of the physician, such as rural roots and upbringing 2 .In countries such as Canada, Australia and the United States, rural practitioners and medical students considering a future in rural medicine reported having a rural upbringing, family in rural areas, and early experience living in smaller communities and attending a rural high school [2][3][4][5][6][7] However, even in Canada, students from rural areas are underrepresented in the nation's medical schools 8 .While 22% of Canada's population lives in rural areas, only 11% of first-year medical students were from rural areas 8 .Due to concerns about the qualifications of rural applicants, relative to those of their urban counterparts, and the impact such potential differences in competitiveness may have on their under-representation, the academic credentials and progress of rural applicants through the stages of admission have been studied [9][10][11][12][13] .It appears that although there is a smaller-than-representative proportion of medical school applicants from rural areas, rural applicants were not unduly disadvantaged by medical school admission processes.Rural applicants were as competitive as urban applicants as measured by the grade point average (GPA), Medical College Admission Test (MCAT) scores and the proportion admitted [9][10][11][12][13] .However, to date no study has assessed if performance on the multiple miniinterview (MMI) varies between the two groups of applicants.
The MMI is a reliable and valid method of selecting candidates for admission to medical school 14,15 .It is increasingly employed internationally during medical school selection and admissions.
The purpose of this study was to determine if an association is present between MMI scores and rural-origin applicants to medical school.Graduation from a rural high school is used widely as a criterion to designate applicants as rural applicants.Therefore, the primary objective of this study was to seek an association between MMI scores and applicants' place of high school graduation.A secondary objective was to find associations between MMI scores and applicant attributes in the following three domains: rural connections, employment in rural areas, and rural community service such as volunteer and leadership activities.As Aboriginal communities in rural Manitoba are rapidly growing 16 , yet another objective of the study was to understand MMI performance of applicants from Aboriginal communities.

Methods
The MMI scores of 1257 applicants interviewed for admission to the MD program at the Faculty of Medicine, University of Manitoba, in years 2008 to 2011 (approximately 315 interviewed applicants a year), were studied for an association with applicants' rural attributes.The conduct and 11-station design of the MMI in Manitoba was similar to those of MMIs in other Faculties of Medicine in Canada 17 .Each applicant was allowed 2 minutes to read a question or scenario and 8 min of discussion time with a single interviewer at each station.The one exception is that one of the 11 stations was a ten-minute writing sample station.The McMaster University question bank with questions that measure personal attributes such as communication skills and maturity within specific station scenarios was used for question selection.Interviewers were clinicians, faculty members or medical students.Applicants were rated on a labeled interval scale ranging from 1 to 7. Therefore, it is possible for applicants to receive a maximum of 77 points; but as the faculty's admissions office uses an operational score that is averaged across the 11 stations, MMI scores are reported here on a scale of 1 to 7.
Within Manitoba, as per our admissions policy, 'rural' was defined as a place with a population of less than 40 000, and this category included all of Manitoba except Winnipeg and its surrounding areas.In the name of equity, the same population-based definition was applied to Canadian high schools outside Manitoba.Also, applicants were required to check a box to indicate if the high school they attended was located in a rural or urban location.This information was routinely verified and followed up by the admission staff and incorporated in applicant decisions and admission processes.From 2009, admissions office staff assigned applicants' rural-attribute domains using detailed, verifiable information collected on students' supplementary application form and by applying comprehensive faculty-specific guidelines and definitions 10 (Appendix I).Details of this methodology and criteria used to categorize applicants as having different rural attribute domains have been published previously 10 .This meant that information on rural-attribute domains was not available for the 2008 applicant cohort.

Statistical analyses
Demographic characteristics of applicants with and without rural attributes were compared using chi-square analysis.Mean MMI scores for applicants with and without rural attributes were studied using Student's t-test.The correlation of GPA and MCAT with MMI was determined through Pearson's correlation coefficient for applicants with and without rural attributes.As per our faculty's admission policies and procedures, the MCAT Writing Sample letter-based scores were converted to numerical scores where M = 7, N = 8, O = 9… T = 14.The MMI scores were regressed on applicants' high school location and applicants' attributes in rural domains in a linear regression model 18,19 .Parameter estimates were adjusted for demographics, admission assessment components such as GPA and MCAT scores, and year and date (which represents the unique combination of selected stations) of MMI attendance.Significance level a was set at 0.05.Model fit and assumptions were assessed by regression diagnostics.

Ethics approval
The study received ethics approval from the University of Manitoba Health Research Ethics Board (#HS11221).

Distribution of demographic characteristics and admission assessment components
Of the applicants interviewed, 205 (16.3%) and 1052 (83.7%) were graduates of rural and urban high schools respectively.A higher proportion of applicants from rural high schools were of Aboriginal ancestry (self-declared), were Manitoba residents, and had attributes in all three rural domains compared with applicants from urban high schools (Table 1).For example, 30% of applicants with Aboriginal ancestry were rural high school graduates compared with 16% of applicants with no Aboriginal ancestry.Gender and age were not differentially distributed for applicants from rural and urban high schools.
Applicants from rural high schools obtained significantly lower scores on the MMI compared with applicants from urban high schools (4.4 vs 4.6).Similarly, those with rural connections obtained significantly lower scores than those with no rural connections (4.4 vs 4.6) (Table 2).When categorized into mutually-exclusive applicant groups based on rural-attribute domains, those with rural community service alone had the highest MMI scores (mean of 4.9), while those with rural connections alone had the lowest scores (4.2).Also, compared with applicants from urban high schools, applicants from rural high schools scored significantly lower on the MCAT, specifically on MCAT Writing Sample, MCAT Biological Sciences and MCAT Physical Sciences (Table 3).Scores on the MCAT Verbal Reasoning and MCAT Writing Sample were weakly correlated with MMI scores for applicants from urban high schools, but for applicants from rural high schools only MCAT Writing Sample scores were correlated with MMI scores.The GPA (mean of 4.0) of rural high school graduates was similar to that of graduates of urban high schools (4.1).

Multiple linear regression analysis
In a linear regression model built using data for years 2009-2011, graduation from a rural high school was significantly associated with a 0.17 decrease in MMI scores after adjustment for demographics, GPA, MCAT scores and year and date of MMI administration (Model 1, Table 4).Applicants' rural-attribute domains were not significant predictors of MMI scores.So rural-attribute domains were omitted from the model.When data from admission year 2008 were included, we observed that graduation from a rural high school was associated with a 0.12 decrease in predicted MMI scores after adjustment for all other variables in the model (Model 2, Table 4).Adjusted R 2 for this model was 0.124.

Discussion
Despite comparable GPA, the distribution of MMI scores among our applicants suggests that there may exist a ruralurban divide in MMI scores.Although only 12.4% of the variability in MMI scores was explained by the chosen final model, our study shows that, all other investigated factors being equal, an applicant from a rural high school is predicted to obtain a MMI score that is 0.122 (on a scale of 1 to 7) lower than an applicant from an urban high school.As our applicants obtain MMI scores that differ by as little as 0.03, this associated decrease is a big enough difference that can affect ranking for a few applicants who fall around the cut-off values for admission.This is similar to an applicant from a rural high school obtaining a MMI score that is 1.75% lower than that of an applicant from an urban high school, if MMI scores were reported on a percentage scale.Yet whether this predicted difference in MMI scores is pertinent for longer-term outcomes, such as future physician performance, is unknown and remains to be determined.It is highly likely that this difference is irrelevant for practice competence.Nevertheless, the significant decrease in average MMI scores among graduates of rural high schools may further affect their under-representation in medical schools, and for that reason is quite concerning.This impediment faced by rural applicants is noteworthy in an atmosphere where targeted admissions policies are recommended to plan for a sustainable rural physician workforce 1,20,21 .
As in a previous study 22 , Aboriginal ancestry was not found to be predictive of MMI scores.Yet applicants who self-declared as having Aboriginal ancestry were more likely to have graduated from rural high schools than applicants who did not self-declare their Aboriginal ancestry (30% vs 16%).Due to the small number of interviewed applicants with Aboriginal ancestry, the power of our study was too low (19%) to consider as statistically significant the observed difference in MMI scores between applicants with Aboriginal ancestry from rural high schools (mean = 4.5 on a scale of 1 to 7) and those with non-Aboriginal ancestry from urban high schools (4.6).In 2006, based on data from five self-declared Aboriginal applicants and seven general-pool applicants, Moreau et al concluded that the MMI protects visuallyunidentifiable Aboriginal applicants against any scoring bias 22 .But the potential interaction of Aboriginal applicant status with rural background may need to be studied further, as increasing Aboriginal representation in medical schools can be an effective way of providing access to physician care in rural areas, specifically in rural areas with Aboriginal populations 22,23 .
The present study is not without limitations.It is not known to what extent these findings can be generalized beyond Manitoba.Contrary to reports published elsewhere 12,13 , our applicants from rural high schools have lower MCAT scores.
The definition of rural used in this study differs from definitions used in studies from Ontario, Alberta and United States 9,[11][12][13] .As a result, direct comparison across studies is somewhat difficult.Typically, rural classification in other Canadian studies was based on population counts of less than 10 000 9,11,12 .Therefore, the observed magnitude of association between MMI scores and graduation from rural high school may have been underestimated in our study because of our more inclusive definition of 'rural.'It is likely that potential dramatic differences in MMI scores between applicants of rural and urban origins have been diluted.After all, although it is known that rural and small town Manitobans are not equivalent to their urban counterparts with respect to economic prosperity, social wellbeing, educational attainment and access to health care, progressively greater differences reportedly arise with increasing rurality as measured by the Metropolitan Influenced Zones system 16 .Moreover, the share of the population that is Aboriginal increases as the level of urban integration declines 16 .Therefore, similar studies on MMI scores from other regions and countries may help us further understand the rural-urban divide in applicant populations.
It would be imperative to determine reasons for the significantly lower MMI scores of graduates of rural high schools.Future studies at different geographic locations could explore potential interactions between applicant demographics, context-specificity of MMI performance of rural high school graduates, and differential scoring by interviewers of rural-and urban origins.As the MMI is increasingly employed in applicant selection, and location of high school is widely used to categorize applicants as rural, our finding may be of interest and concern to medical school admission officers, educational administrators and rural physician workforce planners.

4.
Rabinowitz HK, Diamond JJ, Markham FW, Paynter NP.Critical factors for designing programs to increase the supply and retention of rural primary care physicians.Journal of the American Medical Association 2001; 286(9): 1041-1048.5. Rourke JT, Incitti F, Rourke LL, Kennard M. Relationship between practice location of Ontario family physicians and their rural background or amount of rural medical education experience.Canadian Journal of Rural Medicine 2005; 10(4): 231-240.6. Laven GA, Beilby JJ, Wilkinson D, McElroy HJ.Factors associated with rural practice among Australian-trained general practitioners.Medical Journal of Australia 2003; 179(2): 75-79.

Table 1 :
Demographic characteristics of medical school applicants interviewed for admission at the University of Manitoba in the years 2008 to 2011, by place of high school graduation † Unavailable for the cohort of 318 applicants interviewed for admission in the year 2008.

Table 2 :
Multiple mini-interview scores of medical school applicants with and without rural attributes interviewed for admission at the University of Manitoba in the years 2008 to 2011 M, Mean; MMI, multiple mini-interview; SD, standard deviation.† ¶ Means of groups identified by same symbol were significantly different after controlling for multiple comparisons.

Table 3 :
Distribution and intercorrelation of admission assessment components of medical school applicants interviewed for admission at the University of Manitoba in the years 2008 to 2011, by place of high school graduation

Table 4 :
Results from multiple linear regression analysis of multiple mini-interview scores of medical school applicants interviewed for admission at the University of Manitoba in the years 2008 to 2011 GPA, Grade point average; MCAT, Medical College Admission Test scores; MMI, multiple mini interview scores.† Model 1 was based on scores of 939 applicants to the University of Manitoba Faculty of Medicine in the years 2009 to 2011.¶ Model 2 was based on scores of 1257 applicants to the University of Manitoba Faculty of Medicine in the years 2008 to 2011.§ Adjusted for all other independent variables in the model including year and date of MMI attendance.‡ Reference category † † Continuous variable.
10pendix 1: University of Manitoba Faculty of Medicine definitions and guidelines for assigning rural-attribute domains10