Improving feedback through computer ‐ based language proficiency assessment

This paper reports on the proposed transfer of a paper-based English proficiency exam to an online platform. We discuss both the potential predetermined advantages, which were the impetus for the project, and also some emergent benefits, which prompted an in-depth analysis and reconceptualisation of the exam’s role, which in turn we hope will promote positive washback as well as washforward. This change will be afforded through more granular feedback on student performance, which will be facilitated by the online platform.


Introduction
The testing team at University of Central Lancashire (UCLan) produces a proficiency exam of English as a foreign language which is used in various situations, primarily to allow international students into programmes in their chosen discipline.It is known as the Test of English Language Level 3 (TELL) and is described as 'English for Academic Purposes (EAP)-light' in that it aims to cover the basic skill set students will draw on in their studies, albeit pitched at a relatively general level given that the exam can potentially be taken by a student of any discipline.Recently, a project was initiated to explore the viability of moving the exam to an online platform and we report on this in this article.This paper discusses the rationale for moving online and describes the test adaptation process and platform set up.We report here only on the receptive skills assessed on the UCLan-TELL, although the exam comprises all four skills.The piloting of the new computer-based exam is still ongoing and will be reported on at a later date.

Initial goals of moving online
There were a number of issues which acted as the impetus for investigating a new mode of delivery.The first was that, since the exam is used both in the UK and at partner institutions, reliability is compromised, somewhat inevitably, by a lack of parity of administration, which García Laborda (2007) suggests can be addressed through computerisation.It has long been acknowledged that there are various ways in which the consistency may be threatened (Lado, 1961), including variability in exam conditions (nature of invigilation, environmental factors, etc.).The UCLan-TELL is a relatively high-stakes exam in that it allows or denies access to further study, and thus as such, threats to reliability are not acceptable.
Another advantage of altering the mode of delivery was the possibility to improve security.There is always the possibility that exam integrity can be compromised, especially when operating at a distance from the UK campus, where it cannot be totally ensured who has access to the exam.As stated, the UCLan-TELL is a high-stakes exam for the candidates and consequently also for their tutors, as students' success tends to impact on them.Such tutors (and admin staff) are consequently in a difficult position due to the stakes involved.Anecdotally, we understand information has on occasion been passed on, even if with quite benign intentions, rather than constituting any nefarious activity.
This may even happen at a sub-conscious level, e.g.topics covered in revision lessons leading up to the exam.This compromises exam integrity and, thus, affects exam validity, and administration-related reliability.The online format avoids any possibility of prior access to test content and thus eliminates any possibility of passing on information about the papers.
A further driver to initiating this project was the future possibility, should the online version prove successful, of generating income by offering the exam to other institutions with similar needs, who may not have the resources to produce their own exam for the purpose of certifying international students' readiness for higher education.With this in mind, the salience of improvements in key areas such as reliability and a well-argued case for its validity would be paramount to make it an attractive product.
In addition, we envisage attractive features such as increased efficiency in scoring the objectively marked sections (reading and listening).On top of being inherently advantageous, this would reduce costs by obviating the need for human markers, or optical mark scanners and forms, as documented in the literature (see Chapelle & Voss, 2016).This might thus lead to a more marketable product, which would increase the chance of institutional project support.

Further benefits
In the process of researching the platforms and in having to provide a sound rationale for what could potentially be a costly undertaking, further advantages came to light.The first of these was improved authenticity.While this is widely-interpreted, often in terms of "how well [the test] replicates real life in the tasks" (Fulcher, 2010, p. 98), we tried to address authenticity in relation to how far the means by which candidates accessed and produced text were in alignment with students' practice in general.The majority of students in higher education in the UK nowadays live their study lives mostly online (PWC, 2015); much, if not all, of their coursework is researched and submitted online, and the vast majority of their reading is online (Seyenney & Ross, 2008), as UCLan's Digital Shift project has widened the range and improved accessibility of the digital reading material.Therefore, bringing our English proficiency exam in line with this was felt to be a major advantage as it required students to read online and produce text online, given that this exam is most commonly used as "gatekeeping" (Davies et al., 1999, p. 66) for entry to higher education study in the UK.
Another emergent benefit, while not an immediate priority, was to better manage individual needs of students with certain specific learning difficulties.The inclusivity agenda, i.e. to maximise opportunity for all learners (NCEO, 2011), has for too long been left on a wish-list of future improvements regarding our exam, but being able to operationalise this effectively and reliably contributed to the list of positives for an online approach.
Above all, what became apparent through our investigation and development of ideas for adapting the exam and exploring possibilities in its new format was the potential to provide far better information to students on their performance.The online platform facilitates furnishing candidates with individualised information at a far more granular level than previously practically possible.Rather than an overall grade, or a grade per skill, the chosen online format allowed a breakdown of which sub-skills candidates demonstrated strengths or weaknesses in.For example, the feedback will highlight whether candidates demonstrated the ability to read for gist and detail, but not to infer meaning.Such information, combined with individualised prompts about how to improve in the weaker subskills, offers a formative element to what is usually perceived as summative assessment.

A paradigm shift?
In our experience, proficiency exams and other forms of summative assessment are viewed as an end point.They are frequently seen as an activity which needs to be pursued simply to allow entry to another phase of one's life (such as study, or a new job).They tend to foster a retrospective view of learning up to that point when the exam is taken.All in all, this approach can be summed up as being assessment of learning.In contrast, we felt the new candidate feedback format may allow a shift towards it simultaneously becoming assessment for learning (Gardner, 2012), seeing no good reason why the exam cannot straddle both functions.
Furnished with an individual personalised score profile and associated advice for how to improve areas of weakness, a candidate could utilise this to inform future language learning.The information supplied could help not only the candidate, but also any tutors who may be involved in their learning by taking on a diagnostic role to guide future study (Shohamy, 1992).On entering the higher education institution, international students will need to continue developing their language skills (Evans, Anderson, & Eggington, 2015) and a diagnosis of their needs should guide them in effectively selecting from the means of support available to them.Thus, the exam need not be only summative, but also formative and inform the next stage of further language development.In other words, the aim is for the promotion of this information to support positive washback, to influence study prior to the exam (Alderson & Wall, 1993), as well as addressing the exam's validity for its assigned purpose.Yet it also aims to enhance washforward (Andrews, Majer, Sargeant, & West, 2000), the effect of an exam on future learning.The format and content of high-stakes proficiency exams have been shown to influence aspects of the learning and teaching which takes place prior to that exam.Therefore, the feedback from performance on such exams can influence future learning and should be encouraged, e.g.addressing weaknesses identified by the exam outcomes.

How the individualised feedback is achieved
While the chosen platform allows for the generation of reports that give feedback to candidates, the default setup focusses on individual tasks.We felt that it would be more useful for candidates to receive feedback on their performance in the different skills and subskills coupled with suggestions on how to improve.For example, with regards to the receptive skills, this was achieved by tagging each item with a description (Figure 1) depicting the underlying sub-skills (Figure 2).The system analyses candidates' performances on all items that have the same tag and generates an automated email message detailing the scores achieved as well as explaining how improvements can be made on specific sub-skills and how this will reflect on academic language use.
Figure 1.Item tagging Our rationale for not using the default setup was that we wanted to help candidates think about their language ability in terms of specific skills and sub-skills, thus ultimately improving their understanding of what the exam is targeting and hence their assessment literacy.One way this could be improved would be to provide a visualisation of the candidate's individual profile, in the form of a chart depicting their performance in the specific subskills (Figure 3).This would enable candidates to better understand the areas that they need to focus on to improve their language ability.

Conclusions
From an initial proposal to move our test to an online platform for administrative and business reasons arose a means to achieve potential pedagogically-focussed benefits.Although the project's future is not yet secured, meaning the exam may not move online, the process of investigation has been of great benefit, prompting us to instigate improvements in the current version.