Determining student satisfaction: An economic analysis of the National Student Survey

The UK National Student Survey (NSS) represents a major resource, never previously used in the economics literature, for understanding how the market signal of quality in higher education works. In this study, we examine the determinants of the NSS overall student satisfaction score across eleven subject areas for 121 UK universities between 2007 and 2010. Using a unique panel data set and estimating random effects and fixed effects models, we find large differences in NSS scores across subjects and across different groups of universities, which implies that the raw scores should not be used as a method of ranking. Additionally, the student– staff ratio and student employability are strong influencers of student satisfaction; both of which suggest that a policy which places emphasis on student support, personal development and employability skills will yield an advantage in the higher education marketplace. © 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).


Introduction
The annual National Student Survey (NSS), introduced in the UK in 2005 and completed by the graduating student body of all publicly funded higher education institutions and by some private institutions, was intended to be a method by which universities could assess their own teaching quality and seek to improve student satisfaction with their courses. The survey is commissioned by the Higher Education Funding Council for England (HEFCE), administered by the social research company Ipsos Mori, and forms part of the quality assurance framework implemented by the Quality Assurance Agency (QAA) for higher education. The NSS has become one of the major instruments by which universities seek to compete in the market for student recruitment by sending a signal of teaching quality. Furthermore, these scores are not only * Tel.: +44 114 222 3418; fax: +44 114 222 3458.
E-mail address: p.lenton@sheffield.ac.uk used as a ranking device of student satisfaction across universities, they are incorporated into the major university league tables (HEFCE, 2008).
The administering of the survey itself, between January and April when students are in their final year, has been subject to many allegations of potential distortion including claims that students could be encouraged by their teachers to provide an excellent review (The Times Higher Education Supplement: May 2008). As noted by HEFCE, "Whether attempts to use the results to enhance quality have been successful is an unresolved issue" (HEFCE, 2010, p.11). However, since the announcement of the new higher student fee schedule that saw fees for many full-time courses increased to £9000 per annum from 2012, students possess more incentive than ever to search for the best value student experience they can find, hence the NSS is potentially a key weapon for universities to deploy in search of market share.
The economics literature, in measuring institutional performance, has followed one of two strategies: either it has focussed on the labour market outcomes of graduates or average graduate wage returns as indicators of institutional performance (Blundell, Dearden, Goodman, & Reed, 1997;Bratti, Naylor, & Smith, 2005;Chevalier & Conlon, 2003;Smith, McKnight, & Naylor, 2000); or alternatively it has viewed the higher education institution as a multiproduct firm, assessing the determinants of the 'firm's' productivity (Johnes, Johnes, Lenton, Thanassoulis, & Emrouznejad, 2005;Johnes & Taylor, 1990;Lenton 2008). However, the various university 'quality rankings' can also be seen as an important performance indicator, particularly in determining student demand, and this has become an important area of research since the changes in funding regimes have led universities to actively seek highly qualified students as a means of sending a 'quality signal'. International students, in particular, have been found to rely heavily on university rankings (Chevalier & Jia, 2012;Soo & Elliott, 2010). Horstchraer (2012) finds that university rankings of student satisfaction play a more important role than research rankings for students making their choice of medical school in Germany.
In the UK the literature on the NSS is extremely sparse, consisting mainly of reports by HEFCE of the descriptive statistics along with trends in the scores across years. The education literature contains discussions surrounding the usefulness of NSS scores and rankings, concluding that they are a stable measure of teaching quality (Cheng & Marsh, 2010), and indeed they remain the method by which university teaching quality is measured in many ranking systems. 1 Vaughan and Yorke (2009) noted that arts programmes produce low NSS scores in their qualitative study. Latreille (2010) examined the NSS overall score across economics departments in the UK and notes differences in scores across universities, and more recently McCormack, Propper, and Smith (2014) include NSS scores in their examination of university management and performance. However, to-date there has been no statistical analysis, we believe, that examines determinants of the scores or which assesses their use as a ranking tool.
The major contribution of this paper is for the first time, to conduct an econometric analysis of the NSS which assesses the possible determinants of student satisfaction, and considers whether this survey is an adequate tool for ranking student satisfaction across subjects and universities. We do this by examining possible influences on the overall NSS scores for 11 subject areas within 121 UK universities over a four year period from 2007 to 2010, using a unique constructed panel dataset. The following section provides a description of the data and estimation technique. In Section 3 we discuss our results and in Section 4 we draw our conclusions and implications for policy.

The National Student Survey data
The NSS questionnaire, administered to all graduating students, consists of 22 questions across six areas of university life: teaching; assessment and feedback; academic support; organisation; resources and personal development. A final question asks students for an overall rating of their satisfaction with the quality of their course which is answered on a five point Likert scale from 'definitely agree' to 'definitely disagree': "Overall, I am satisfied with the quality of the course" The NSS data is provided on the 'HEFCE' website 2 and available for subjects within the joint academic coding system (JACS) 4 digit code level, of which there are 142. The NSS data presents JACS subjects coded in three levels according to the level of aggregation, for example, Languages is given as level 1 which includes all programmes that are language related. A lesser level of aggregation is European languages at level 2, which as it name implies consists of all European languages. A further disaggregation is of single programmes i.e. French, German and Italian etc. which are classed at level 3. However, not all universities have recorded NSS results at level 3, either because the programme is not offered or because there are not enough observations to be able to report the NSS response in that year. 3 For this reason subjects were selected from levels of aggregation where we have sufficient NSS observations within one year. The subjects selected include Biological sciences at level 1 and Art and Design, Business, Computing, Economics, European languages, History, Mathematical sciences, Management, Psychology and Sociology at level 2. The choice of subjects in this paper was also made to reflect different faculties and different types of teaching, i.e. laboratory versus classroom taught subjects.
We match in data on finance resources, student numbers in higher education, student performance indicators and the destination of graduates for each of our subject areas within 121 UK universities; all this subject-related data is provided by arrangement by the UK Higher Education Statistics Agency (HESA). 4 The overall NSS score for each subject area is calculated as the average of the answer to the final score. Additionally, the percentage of students who agree or definitely agree with the statement above, that is they rate as a 4 or 5 on the Likert scale, is often cited in rankings of university performance. In the analysis presented here we use both these measures and focus upon eleven subjects within each university, namely; biological sciences, mathematical sciences, psychology, computer sciences, economics, sociology, business and administration, management, European languages, history and art and design. The choice of these eleven subjects was driven largely by the availability of matching data across our data sources and this provided a total sample of 3438 observations within 121 Universities across England, Scotland, Wales and Northern Ireland (see Supplementary Materials, Appendix Table A1 for a list of universities in this study along with their classification and Table A2 for an overview of degree classification and funding regimes within each country of the UK). The dataset is an unbalanced panel due to missing information at the subject level in particular years from some