The Impact of Live Audience Participation Teaching on Medical Education at The Surgical Scousers, an Undergraduate Surgical Society

Background: The use of Live Audience Participation (LAP) allows students to respond to questions anonymously and receive instant feedback. Electronic devices promote interaction and participation from learners which has the potential to further engage students and lead to an increased level of learning. It is important however to evaluate the aﬀordances and limitations of such devices. Objectives: The aim of our study was to assess the aptness, utility, practicalities and acceptability of LAP during lectures. Methods: The interactive ‘PollEverywhere™’ tool was used during 5 extra-curricular lectures, organized by ‘The Surgical Scousers’ society, allowing student participation to anonymously answer questions. Live results were displayed instantly and data was collected prospectively in 2013. Results: Out of 523 responses, 417 were correct. Students were also able to ask questions and give live feedback. Students gave positive feedback highlighting how LAP maintained their attention. The majority of students expressed that they would not have asked the same questions in a conventional set up. Conclusion: We have demonstrated that the use of LAP holds the attention of students, provides opportunity to ask questions, and facilitates ironing out any misconceptions within subject areas. However, interactive devices have certain inherent limitations, namely cost and potential to go wrong. The majority of literature focuses on students’ views towards interactive devices therefore. Further studies are required to assess the impact of interactive devices on learning outcomes in medical education.


Introduction
Lectures remain a mainstay of education at universities. Traditional lectures are used in order to present high volumes of information to students without gaining feedback as to whether they are following or understanding the contents. This allows students to remain as passive attendees throughout the learning experience. 1 Modern learning theories describe learning as an active process, this highlights the flaws in the traditional lecture process. [2][3][4][5][6][7][8] We believe there should be a drive to encourage an interaction between the speaker or presenters and the students during a traditional medical school lecture.
In the era of modern information technology, most students own portable electronic devices. There has therefore been a drive to utilise this technology to improve the quality of lecture based teaching. [8][9][10] Live Audience Participation (LAP) is a method by which students can actively participate and can answer the lecturer's questions individually using an electronic device, in an anonymous fashion.
The technology consists of transmitters and receivers linked to a computer which then displays the student responses in real time. The lecturer is provided with instantaneous feedback and is able to make an assessment of the room's knowledge, doubts, misconceptions and understanding. Additionally, this active learning approach allows students to test their knowledge and challenge any uncertainty straight away. 8,11 LAP has been assessed in a number of educational settings, but little literature has been published regarding its use in medical education. The aim of this study was to evaluate the suitability of LAP in undergraduate medical education lectures, provided by an undergraduate surgical society.

Methods
The interactive 'PollEverywhere™' tool was utilized at 5 extra-curricular surgical lectures provided by 'The Surgical Scousers' society in 2013. PollEverywhere™ allows lecturers to create interactive lecture slides that can be inserted into a PowerPoint™ presentation. Students can participate by logging on to www.polleverywhere.com and answering these questions using their smartphones, tablets or laptops. The tool allows students to answer questions anonymously and the responses appear in real time on the projected screen. Briefly, the system allows the lecturer to use multiple choice questions, true or false statements, solicit short free text answers or ask the students to identify structures on an image. When the lecturer arrives to the question slide the question appears on the devices used for logging on to the poll server and students mark their answers, in case of an image by pointing or clicking on the relevant structure. In our study students were allowed to answer each question only once, without the ability to change their responses afterwards. Answers are displayed in graphical form to both students and the lecturer to see. Data was collected for all correct and incorrect answers. Students were also able to ask anonymous free text questions throughout the lecture.
In addition to the online polling, students were also asked to complete an anonymous feedback form about their opinions on the inclusion of LAP in the teaching session. This feedback was collated using the same methods, where students provided their opinions of LAP using their portable electronic devices at the end of the individual sessions.

Results
A total of 26 questions were embedded into 5 lectures (median = 4, range= 4-9). The question formats used were: multiple choice (n = 15), true or false (n = 5), short answer responses (n = 4), or the students were asked to identify a structure on an image (n = 2). For an example of the latter see figure 1.
A total of 523 answers were submitted by students. From the answers given, 417 were correct and 106 were incorrect. The faculty was able to address the issues that were unclear to students and. thereby, clarified these doubts and misconceptions that surfaced as a result of live feedback. The distribution and frequency of correct and incorrect answers is shown in figure 2. As shown, only 2 out of 26 questions were answered correctly by all responders.
Students gave very positive feedback to their overall experience of LAP teaching ( figure 3D). They stated that it allowed them to ask questions which they otherwise wouldn't, and that it resulted in being able to better keep their attention (figure 3 A+B). They also agreed that the LAP teaching did not significantly inhibit the flow of the lecture ( figure 3C).

Discussion
Lectures, with inherent limitations, remain a necessity in modern day education, allowing one individual to educate a high volume of learners. However, they require the constant attention of students. It has been demonstrated that participants are more likely to maintain attention after the inclusion of questions or short tests. 2,12,13 We have demonstrated that LAP can be utilised by a lecturer in the setting of medical education, allowing the teacher to make an instant assessment of the level of understanding within the room. We have also demonstrated that students are more likely to ask questions when using LAP (figure 3A), and that their overall experience of LAP is positive ( figure  3D).

LAP makes the lecture more interesting and keeps the attention of the audience.
Live audience participation (LAP) systems are a method for which seminars can be made interactive in an attempt to actively engage the participant. 11,[14][15][16][17][18][19][20][21] Interactivity has been demonstrated to increase motivation and keep the attention of students for longer. 1,2,6,22-26 Wessels et al demonstrated that students who had interactive lectures felt more attentive, and they felt that their knowledge acquisition improved. 2 We have demonstrated that students have a positive experience whilst using LAP (figure 2D), which has been repeatedly demonstrated in the literature. 2,14,17,18,[27][28][29][30][31][32][33][34] LAP provides instant feedback to students in the form of graphs which are displayed both to the students and lecturer. We believe this keeps the audience's interest better than traditional lecturers, and this was reflected in the survey responses (figure 3B).
In previous publications, students have stated that LAP sessions were "interactive and enjoyable" and that it stimulated their interest. 2,35 Additionally, LAP improves participation, motivation and understanding, 21,36,37 and lecture topics that are discussed using interactive devices result in students being more engaged, focused and interested. 9,[38][39][40][41] It has been reported that the discussion sparked by LAP is one of its most important benefits, as students prefer to  42 This results in focused and effective peer-to-peer discussions when using interactive learning tools. 9,43,44 LAP allows the audience to ask and answer questions which they otherwise would not.
One of the major benefits of LAP is the anonymity it provides, giving students a 'safe' environment in which to engage in answering questions. 17 This is in keeping with the feedback we received via our questionnaire and was found to be a recurring theme throughout the literature. 21,35 In fact, one publication found that 42% of students stated the anonymity of LAP encouraged them to participate in lectures. 45 The reasons why this anonymity is so important have also been explored. It is suggested that students who may not ordinarily participate in open group discussion feel able to answer without fear of humiliation especially if they are unsure of the correct answer. 11,21,46,47 Students also enjoy being able to gauge their knowledge against their peers. 35 On the contrary, some students do not enjoy the experience of discussing answers that have been presented, especially if they feel they have chosen an incorrect answer when the majority of a class has achieved the correct one. 9,48,49 Limitations of LAP during the lectures.
Unfortunately, LAP is not without its problems. The process can be hindered by technology failure and one study found that when a group suffered these technical difficulties only 15% of respondents stated that they favoured the use of LAP and specifically suggested that the issues they experienced are a disadvantage of incorporating the technology. An example of one of these failures was due to the set-up of the room obstructing the technology, rendering some students unable to respond to the questions asked using LAP. 45,50 However, it has also been shown that most objections to the use of interactive learning devices occur during the initial period of implementation, perhaps before the initial flaws are rectified. 9 As figure 2 demonstrates there were fewer responses using LAP in the first lecture (questions 1-4) which may highlight the initial reluctance to engage in a new system.
The positives of LAP are well documented but it is important to note that while students may enjoy using the technology, it may not necessarily improve learning. 11,19,38 Although our study found that students do not feel LAP impedes the flow of the lecture (figure 3C), one major negative which has been suggested is that the time taken waiting for responses from students may affect the pacing of the lecture and thus reduce the amount of information covered by the lecturer. 35,45 In addition to this, the involvement required from students to use LAP has been shown to reduce the number of notes they take and there is dispute as to whether this is increasing attention on the lecture and is therefore positive, or is disadvantageous to some students who rely solely on note taking for their learning and revision. 20 However, we believe that LAP ensures that the teacher is not able to rush through the slides, creating of weakness. 2,9,11,14,17,18,38,52 The ability to alter lectures based on class response is something that educators have been particularly positive about. 11,39,46,53 On the other hand, if the audience already has a full grasp of the subject the lecturer can move forward at an increasing rate.
During our study there were 106 incorrect answers across the 5 lectures, which the lecturer was able to address immediately. There were also 2 questions which the whole room answered correctly, allowing the lecturer to move through these portions of the lecture at a faster pace.

Negatives of LAP for the lecturer
On the whole, the use of interactive devices is relatively simple. 15,54 It is the construction of applicable and effective questions that can be time consuming and often challenging. 9,55 It has been suggested that some of the most effective questions allow recently acquired knowledge to be used, focus on reasoning and aim to identify any student misconceptions. 9,56 As aforementioned, a benefit of LAP is that it allows the lecturer to adjust his or her teaching session based on learning needs identified by the students' responses. However, an important point made by one study suggests that this benefit can only be achieved if the lecturer is able to deviate from a previously prepared lesson plan. 17 This potential difficulty is partly due to the unpredictable nature of the students' responses, requiring the lecturer to have a broad knowledge of the subject, and allow for flexibility within the session to address any confusion that is highlighted. 35 Another common and primary concern of using these interactive tools on a regular basis is a reduction of content covered. Both teachers and students feel that due to the discussions that take place and the time taken by submitting responses less subject matter is covered. 9,38,57,58 In contrast, Tregonning et al reported that there was no difference in the amount of material covered when using LAP compared to a traditional lecture. 35

LAP improves exam results and levels of understanding
A number of studies have demonstrated that learning outcomes improve when using interactive devices compared to conventional teaching methods. 25,31,59,60 Students in a study using interactive devices stated that they felt they acquired a better understanding of the lecture topic than a traditional lecture would have allowed. Participants also provided positive feedback regarding the anonymity of their answers and the discussion sparked within the class, helping to understand the reasoning behind the answers to questions. 21 The ability for students to assess their understanding of the lecture content through discussion meant they felt their learning was more effective. 11 Improvement of understanding has been further demonstrated by a number of publications showing improved exam results in participants using interactive devices compared to a control group. 21,35 Other studies have also demonstrated an improvement in exam results following the implementation of interactive devices. 61 Tregonning et al found that when comparing interactive devices and traditional lectures with a quiz immediately following the teaching session, 5 th year medical students scored significantly higher mean scores following interactive sessions, than those without this system. It should be noted however, that although scores increased on quizzes immediately following the session in the LAP group, there was no difference in scores between the interactive and traditional lecture groups at 5 weeks post-session; this may show that whilst interactive devices help students to gain more from the lecture in the short term it does not improve long-term retention. 35  The use of paper feedback forms can be a struggle as students are anxious to leave the teaching session at the close. LAP goes some way to addressing this problem as it allows for feedback to be collated instantly. Our data supports this rational as we saw the majority of our responders rate the use of LAP as a 6 or above; with 10 being 'excellent' (figure 3D). A further affirmative characteristic of the use of LAP has been highlighted in the literature; the use of LAP in teaching sessions resulted in improved attendance. 9,11,39,40,46,50,53,62 It would be idealistic to imagine that this technology, whilst it has its benefits, does not come without some obstacles.

Effects of LAP on the institution
Arguably the most important of these is the issue of cost. We have not explored this within this study, but evidence supports the indication that these sessions come with an unavoidable cost to the faculty, both monetary and time required to prepare sessions and train staff in its use. 16,26,63 In addition to this, to participate students need a computer, other wireless device or mobile phone and access to a wireless internet connection is essential for the system to work. In a study in 2004 Guthrie & Carlin found that whilst most students respond well to the use of LAP, only 27% expressed that they considered the benefits of the technology to justify the costs. 45 However, with the almost universal ownership of smartphones this issue is becoming less relevant. The University of Liverpool started to provide wireless tablets for students who cannot afford a portable device and it is not uncommon higher education institutions to do so for all of their students.

Limitations
Our study assessed the use of LAP during 5 non-compulsory extracurricular lectures. It can be argued that students who attend such teaching sessions are already motivated to learn. As such they may be more open to the use of interactive technology which may mean our data is not representative of the entire student population.
We did not explore the impact on long term retention within this study, nor did we attempt to quantify the monetary cost for the university. A further limitation is that that the number of students attending these sessions was significantly smaller that the audience of a regular university lecture with mandatory attendance.

Conclusion
We have demonstrated that LAP is an enjoyable experience for medical students having "out of hours" teaching. It keeps participants' attention better than traditional lectures and provides an opportunity to ask anonymous questions, without interfering with the flow of the teaching sessions. Incorrect responses trigger the lecturer to correct misconceptions. LAP allows the audience to ask and answer questions which they otherwise would not.

Notes On Contributors
EN, PM, BR & LA collected the data.
EN and PM wrote the initial manuscript. AS and LP provided a supporting role.
All authors approved the manuscript before submission.