The benefits and challenges of large-scale deployment of electronic voting systems: University student views from across different subject groups
Introduction
Since their introduction in the late sixties (Kay & LeSage, 2009:819) Electronic Voting Systems, also known inter alia as “clickers”, Audio Response Systems, or Personal Response Systems are being increasingly used in university education as a means to increase students' engagement, particularly in large classes. This is commonly achieved via in-class quizzes based on multiple-choice questions (MCQ), which can offer formative or summative outcomes. The technology behind EVS is very simple, consisting of a receiver, used for collecting the question responses and a number of handsets used for sending the responses by individual users. The examples of use cited in the literature are largely focused on small-scale implementation, within a single module and typically within subjects such as science, technology, engineering and mathematics (STEM). Using a mixed-methods approach, this paper seeks to address the gap in the literature and to explore the large-scale institutional implementation of EVS and differences in use and outcomes across academic disciplines, including those outside the STEM subjects.
The research presented in this paper is part of a wider evaluation project initiated in the 2011/12 academic year that examined benefits and issues of large-scale institutional deployment of EVS technology for assessment and feedback at a medium-size post-92 UK university (UH) with in excess of 27,000 students. The aim of the project was to extend the benefits of EVS experienced by the early adopters from the area of Health and Human Sciences, across the whole university, while increasing the efficiencies related to the management of the handsets and the staff workload. The latter was achieved through an increase in the use of EVS for summative assessment. The central implementation campaign included regular staff training, installation of receivers in all teaching rooms, purchase of additional handsets, and the standardisation of operational procedures including distribution, ownership, and cost of handsets. By September 2012, 7265 EVS handsets had been purchased, 35% of the classrooms had been equipped with the EVS technology and 114 additional staff members i.e. 9.6% of the total staff population had been trained on technical and pedagogical use of the EVS.
The aim of the evaluation project, which worked alongside the institutional deployment project, was to support a scholarly approach to the changes in practice, while the purpose of this paper is to unravel some of the tacit and less explicit findings supported by the further analysis of the existing data. It is important to emphasise that although the data used for this paper are a subset of data collected during the evaluation project, they were collected purposefully in order to answer specific research questions, in an attempt to address some of the gaps identified in the current literature.
Section snippets
Literature review
The literature on EVS in university education dates from the 1990s (Dufresne et al., 1996, Mazur and Hilborn, 1997) to the present day and includes seven literature reviews conducted between 2002 and 2014. Amongst these, the review paper by Kay and LeSage's (2009) remains the most systematic and comprehensive appraisal of the literature, which has provided a framework for research in this area.
Research questions
This research accordingly addresses some of the issues and gaps reported in the literature review above and considers the following research questions:
- 1.
What are the main components contributing to the students' views on EVS in a context of a large-scale institutional deployment?
- 2.
What are the new student-centered challenges that arise in a large-scale institutional deployment of EVS?
- 3.
What is the impact of increased summative use of EVS on students' views?
- 4.
What are the differences in usage and
Methodology
This research is based primarily on the gathering of data through a purpose-designed student survey (N = 590) with some additional comments provided by students studying at UH in 2011/12 via their online reflections. In the latter case, the students (N = 26) captured their experiences of using EVS in class by either a daily written blog or by regular audio or webcams recordings. For the purpose of this paper these comments are used to illustrate the findings from the survey in the related
Data analysis and results
The first part of this section includes the rationale for using the specific data analysis techniques in relation to the research questions.
To answer the first research question (“What are the main components contributing to the students' views on EVS in a context of a large-scale institutional deployment?”) the Principal Component Analysis (PCA) technique was used (Section 5.1).
PCA is a technique used for reducing a number of variables into a smaller set of “principal components” representing
Discussion
The goal of the paper is to offer a deeper insight into students' views on the benefits and challenges of EVS in the context of a large-scale institutional deployment and across different subject areas and different levels of summative and formative use. This section includes the discussion of the results and limitations of the study and it is structured according to the initial research questions.
Conclusions
The benefits of EVS for students' learning, engagement and motivation are well-known from the numerous studies on small-scale implementation. The aim of this research was to obtain a deeper insight into students' views on the benefits and challenges of EVS in the context of a large-scale institutional deployment and across different subject areas, the topic that is currently under-represented in the EVS literature.
The results of this research suggest that the scale of EVS deployment can create
Acknowledgements
The research was partially supported by the JISC Assessment and Feedback Programme Grant Funding 5/11. We thank Neil Spencer and the anonymous reviewers for their useful comments on the draft of this paper.
References (34)
The student experience of a collaborative e-learning university module
Computers & Education
(2011)Using a voting system in conjunction with interactive whiteboard technology to enhance learning in the English language classroom
Computers & Education
(2008)- et al.
The role of online reader experience in explaining students' performance in digital reading
Computers & Education
(2012) Academic dishonesty and video game play: is new media use changing conceptions of cheating?
Computers & Education
(2012)Unpacking and repacking the factors affecting students' perceptions of the use of classroom communication systems (CCS) technology
Computers & Education
(2014)- et al.
Understanding the effects of professors' pedagogical development with clicker assessment and feedback technologies and the impact on students' engagement and learning in higher education
Computers & Education
(2013) - et al.
Examining the benefits and challenges of using audience response systems: a review of the literature
Computers & Education
(2009) - et al.
Effectiveness of clickers: effect of feedback and the timing of questions on learning
Computers in Human Behavior
(2014) - et al.
What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction
Computers & Education
(2008) - et al.
The System Usability Scale (SUS): an empirical evaluation
International Journal of Human-Computer Interaction
(2008)
System Usability Scale (SUS): A quick-and-dirty method of system evaluation user information
Classroom response system (‘clickers’) bibliography
Clickers in the large classroom: current research and best-practice tips
Life Science Education
The use of clickers in the law school classroom
The Law Teacher
Student engagement and student learning: testing the linkages
Research in Higher Education
Research methods in education
Scale development: Theory and applications
Cited by (18)
Quality-based focused learning model for social and ubiquitous learning environments: A theoretical review
2023, Revista de Educación a DistanciaGame-based student response system: Revisiting its potentials and criticalities in large-size classes
2021, Active Learning in Higher EducationAssessment of Kahoot’s employment in university teaching based on student considerations
2020, RISTI - Revista Iberica de Sistemas e Tecnologias de InformacaoE-voting scheme using secret sharing and K-anonymity
2019, World Wide WebDigital technologies: Supporting and advancing assessment practices in the classroom
2019, Classroom Assessment and Educational MeasurementUniversity students' perception of the use of socrative in learning experiences with mobile technology
2019, Revista Electronica de Investigacion Educativa