Deep learning questions can help selection of high ability candidates for universities

Abstract

Selection of students for places at universities mainly depends on GCSE grades and predictions of A-level grades, both of which tend to favour applicants from independent schools. We have therefore developed a new type of test that would measure candidates’ ‘deep learning’ approach since this assesses the motivation and creative thinking that we look for in university students. We recruited 526 applicants to Oxford University and gave them a short commentary test and a learning style questionnaire. Specific deep learning approach questions correlated with results in the new test, and both predicted whether the candidate subsequently obtained a place at Oxford. Furthermore high scores on one open-ended commentary question, demanding arguments in favour of a case, produced a greater than 70% chance of obtaining a first class degree at the end of their course irrespective of the candidates’ type of school attended or GCSE scores. Candidates from State schools scored as well as those from Independent schools in both tests. Thus our test seemed to index candidates’ potential to succeed at a highly selective university, and might usefully be added to current selection procedures for such universities.

Appendix

Canonical correlation analysis (Manly 1986) was carried out in the following way. One set consisted of all the individual deep learning questionnaire variables and the other consisted of the scores on commentary questions 2 to 5; correlations between the two sets were calculated (Table 4). There were 39 missing values corresponding to 10 individuals in these two sets of variables. Rather than miss out all the information on these individuals (who only had missing values in some of the scores), we imputed these 39 values in the original scales of the questionnaires using the programme transcan from the R-library Hmisc (Alzola and Harrell 2004). This imputation method does not make any distributional assumptions on the data.

Table 4 The coefficients for the first two canonical components

We tested the assumption of normality using Kolmogorov–Smirnov test for the 27 variables considered and found that none of them satisfied it. As a result, we used Spearman’s correlation coefficient (ρ) and did not make formal inferences based on the results of canonical correlation analysis. In any case, our sample was not large enough to provide sufficient power to detect significant canonical correlations (Stevens 1986); our use of these is purely descriptive.