Using an online student response system, Socrative, to facilitate active learning of Physiology by first year graduate entry to medicine students: a feasibility study

Technologies such as audience response units (‘clickers’) have been used to facilitate greater student engagement within a variety of educational settings, but numerous technical issues have limited their more widespread use. More recently, flexible, cloud-based student response systems (SRSs), which are designed for use with student mobile devices and overcome most of the limitations of clicker systems, have become widely available. However, the suitability of use for such systems in accelerated degree programmes such as graduate entry to medicine (GEM) has yet to be assessed. Therefore, we utilised Socrative, a freely available SRS, in a physiology component of a first year GEM module to ascertain, a) its ease of deployment, b) its popularity with students and, c) if they felt it improved their learning. There were no technical problems using Socrative. Further, 93% of respondents to an attitudinal survey strongly agreed or agreed that they favoured using Socrative in the classroom, and that they felt that it had improved their learning (92%). Thus, our data strongly indicate that the use of SRSs like Socrative would be highly valued even by the time-pressured, relatively mature students enrolled on accelerated professional courses such as GEM.


Introduction
Prior to the advent of Web-based teaching tools around 2011, if educators wished to gather in class, real time, information on student understanding/opinion, they could either simply ask students to raise their hands in response to a question or utilise a computerised audience response ('clicker') system. The benefits to the student of the latter For these reasons, Socrative seemed to us to be an attractive replacement for the clicker audience response system we had used previously. However, before fully endorsing this new SRS and proposing its use by other lecturers, we wanted to run an initial feasibility study to determine the reaction of students using it. For example, although today's students have a preference for, and a desire to utilise, technology during the learning process (so-called 'Digital Natives'; Prensky 2010), we wanted to know if they might perceive Socrative as merely a gimmick without any sound pedagogical benefits. As Dervan (2014) observed, educators using new technology for teaching "can be perceived to be more focused on technology than with teaching" to the annoyance and distraction of students, particularly if technical issues overshadow the technology. We also wanted to ensure that Socrative could be employed in a relatively seamless manner within the classroom such that vital teaching/learning time was not wasted either setting up or troubleshooting, both of which can be troubling for students (Stuart et al. 2004). Finally, as Knight and Wood (2005) noted, many students are still incorrectly of the opinion that meaningful learning can only take place whilst the lecturer is talking, and that time spent on other classroom activities such as interactive quizzes Rae M, O'Malley D MedEdPublish https://doi.org/10.15694/mep.2017.000004 Page | 3 and discussions can be viewed as both distracting and a waste of time for them. This latter point is a particular concern for us teaching students enrolled upon the graduate entry to medicine (GEM) program here at University College Cork (UCC). In addition to requiring students to possess a minimum of a second class honours, grade one (2H1 or equivalent) result in their first honours undergraduate degree (from any discipline) to gain entry to the GEM program, its duration is only four, rather than the five, years of traditional 'direct-entry' undergraduate medical (DEM) degrees (Duggan et al. 2014). This is primarily accomplished by compressing the pre-clinical teaching on GEM programs into just over one, rather than two, years which means that the GEM students must learn the same amount of material as their DEM counterparts in effectively half the time, requiring that they manage their study time very efficiently. As such, they do not appreciate, and may not attend, teaching activities which they perceive to be of little value or relevance to them.
In the present study therefore, we sought to answer the following research questions: What were the attitudes of the first year GEM students towards Socrative as a teaching/learning tool? 1.
Did attitudes differ depending on educational background (i.e. whether or not the students' first degree was in 3. a so-called 'biomedical' versus a 'non-biomedical' discipline)?

Method
The study was undertaken at University College Cork (UCC), Ireland as part of the Physiology component of a fourteen week pre-clinical module, GM1002 (Fundamentals of Medicine II), which also comprises Anatomy, Microbiology, Pathology and Pharmacology components (N.B. to date, the author (M.G.R.) is the only lecturer to have used Socrative during lectures and/or practical sessions on the GEM program and therefore the students had not had any prior exposure to this SRS).
For this study, we prepared eight quizzes in total using the Socrative website. Questions were of a single best answer format where a question stem could be answered by selecting one of up to five possible answers, only one of which was the correct answer. These quizzes comprised of between four and seven questions with each focussing on different elements of respiratory physiology and acid-base balance that are covered in six lectures and four practical sessions within this module. The questions used for the study were a mixture of pure recall and application type questions, consistent with the format and difficulty of those used in their fomal examinations. As the point of the exercises was to encourage student participation and discussion without 'exam pressure' no summative marks were awarded for performance. Indeed, to further encourage participation, they were free to log-in to the Socrative site anonymously if they so wished. The quizzes were usually deployed towards the end of each lecture and/or practical class and related primarily to material that had been taught in that session. On two occasions however it was felt that there would be insufficient time to run the Socrative quizzes and so they were held at the start of the next available teaching session, be it either a lecture or a laboratory practical.
At the end of GM1002 in March 2015, upon completion of the six lectures and four practical sessions included in the study, students were asked to complete an anonymous online quantitative and qualitative survey containing multiple Likert-style (five-point balanced -with text strongly agree, agree, neither agree nor disagree, disagree or strongly disagree), as well as several free-response, questions relating to their Socrative use. The participants provided their consent to being surveyed when they accessed and completed the survey. In order to avoid unintentional bias inadvertently affecting analysis of respondents' data by the lecturer delivering the taught material and Socrative exercises in class (MGR), responses to the attitudinal survey were gathered, analysed and categorised (in the case of the qualitative responses) by one of the authors (DoM) who was not involved in the teaching, examining or organisation of the GEM programme. Further, DoM had no contact with any of the students enrolled on the programme. From the data captured by the attitudinal survey we obtained basic demographic information (e.g. age, sex, nationality, educational background) and self-reported attendance. Where relevant, we were able to pair this with the information provided in the four Likert-type questions related to their perceptions of Socrative and use of questions in lectures. Responses to the Likert-type questions were further sub-categorised into, a) male versus female and, b) biomedical educational background (all students, male and female) versus nonbiomedical educational background (all students, male and female). Thus, in the manuscript, data derived from the Likert-style questions are displayed both as percentage values in histograms as well as means where values have been attributed to the Likert responses (e.g. strongly agree = 5, strongly disagree = 1). In this way we were able to make statistical comparisons between values for the different sub-categories of students in the study using, unless otherwise stated, Student's unpaired t-test (assuming equal variance between the groups). P < 0.05 was considered significant.

Student demographics
For the current study 65 out of 70 (93%) eligible students in GEM year 1 completed the survey towards the end of  Table 1 presents demographic data for the respondents and illustrates a roughly even split between males and females (with no significant differences in their ages) and those coming from either a European Union (EU) state (mainly Ireland) or a non-EU state (mainly Canada). There are however significantly fewer entrants to the GEM program of students whose first degree was classified (by the students themselves) as 'non-biomedical' (e.g. law, music, English, etc.) relative to those coming from a 'biomedical' undergraduate degree or discipline (e.g. Biochemistry, Physiology, biological research, etc.). Of the 65 students who responded to the survey, a minimum of 64 students answered all of the questions referred to in this study. Attendance at the six lectures and four practical sessions which were the subject of the current study was excellent, with 85.9% (55 students) attending all six lectures and the remaining nine (one student did not answer this question) attending at least five lectures. Students were overwhelmingly positive about their experience using Socrative, as measured both quantitatively and qualitatively. For example, in response to the statement, 'I liked using Socrative to answer questions during lectures/practicals', 95.3% (61/64 students; 1 unanswered) either strongly agreed or agreed (mean = 4.55±0.07). When these responses took into account sex and/or educational background there were no statistically significant differences between groups, illustrating their almost universal popularity with the class (see figure 1A).

Student perceptions of Socrative
Although this result seemed to be a ringing endorsement by the students for Socrative, we wanted to ensure that they were also able to express their opinion on in-class questioning in general, just in case their positive view of Socrative had been swayed by the novelty factor of this new tool. However, we found that responses to the statement, 'I would prefer NOT to have to answer questions at all during lectures', supported the previous result in that 92.2 % (59/64 students, 1 unanswered) disagreed or strongly disagreed with the statement (mean = 1.57 ± 0.08), with none agreeing or strongly agreeing. It is interesting to note however that the percentage of students who disagreed or strongly disagreed with the statement was in excess of 90% for all subcategories of students described above except for female non-biomedical students, where it fell to 75% (mean = 2 ± 0.27, n=8). This was significantly different to the all-male (mean = 1.5 ± 0.1, n= 35, P< 0.05), female biomedical (mean = 1.5 ± 0.13, n= 33, P< 0.05) and male biomedical (mean = 1.5 ± 0.11, n=27, P< 0.05) subgroups. However, due to the relatively low numbers of female non-biomedical students (eight), in reality this simply reflects the fact that two of those students selected the 'neither agreed nor disagree' option for that particular question (see figure 1B). In an attempt, again, to ensure that any positive student sentiment towards Socrative was for pedagogical reasons rather than the novelty of new technology (Blood and Neel 2008), we sought to ascertain if they felt that using Socrative had actually helped them to learn, which ultimately was one of the main purposes of the exercise. Thus, in response to the statement, 'Using interactive software like Socrative improves the overall learning experience', 92.2 % (60/65 students) strongly agreed or agreed, with none disagreeing or strongly disagreeing (mean = 4.49 ±0.08). When the responses were broken down into their various subcategories, it was notable that all sixteen of the non-biomedical students strongly agreed or agreed with the statement (mean = 4.5 ± 0.12, n=16), whereas female biomedical students polled the lowest percentage strongly agreeing or agreeing with the statement (86.4%; 19/22 students; mean = 4.5 ± 0.16, n=22). Statistically however, there were no significant differences between these or any other subgroup (see figure 2A).
One final Likert-style question in the survey asked students to respond to the statement, 'I would have preferred Dr Rae just telling us the answers to questions rather than using Socrative' in order to ensure that we were obtaining student impressions of Socrative rather than simply their opinion of gaining access to exam-style questions for example. Overall, 60/65 students (92.2%) disagreed or strongly disagreed with the statement, with only one (female biomedical) student agreeing with it (mean = 1.77 ± 0.08). Similar percentage values were found across all subcategories, although the largest percentage of students who neither agreed nor disagreed with the statement was the male biomedical students (11.1%, 3 students; mean = 1.7 ± 0.13, n=27; see figure 2B).

Student Qualitative Responses Regarding Socrative
In the free essay component of the survey, students were invited to provide comments about their use, and impressions, of Socrative in lectures/practicals. Socrative is very useful as it makes you apply your knowledge on the spot. Discussion of the answers is also useful for learning the lecture material.
A further 19/58 (32.8%) comments related to how the anonymous nature of the quizzes encouraged them to answer questions in class.
Being anonymous allows students to answer without repercussion [sic], allowing some students who are unsure of their answers to take risks and learn from their mistakes The anonymity of answering with socrative definitely makes it much easier to answer questions.
I think the anonymous nature is perfect and does encourage you to answer all the questions.
Eighteen students (31%) also explicitly stated that they wished Socrative would be utilised by other disciplines/ lecturers on the GEM course.
Finally, but importantly, four students (6.9%) specifically referred to the fact that they enjoyed the collaborative nature of Socrative exercises.
Socrative was a very good way to help establish concepts we just went over in lecture, but also to discuss with neighboring [sic] classmates and guage [sic] my level of understanding relative to the class.

I took away a lot of info just from doing the socrative questions and being able to discuss why answers were correct and incorrect. I also liked working out the answers with peers in class.
Although no comments were designated as being explicitly negative about Socrative per se, two students (3.4%) felt that the time provided for the class to answer each question was too long (it was approximately 60 seconds per question but longer if the author felt the question was more challenging than normal, or if a calculation was required). Another three students (5.2%) stated that, depending on the room in which the quizzes took place, internet connectivity could hinder their ability to both log-in to Socrative as well as for their answers to be logged by the website.

Discussion
The growth of accelerated graduate entry programs within Ireland and the UK, and the attendant time pressures being placed on students enrolled upon such courses, is increasingly challenging traditional approaches to teaching and learning. This is particularly true with regard to the role of traditional, passive, didactic lectures, which, in spite of the large body of evidence indicating that they are a relatively ineffective way of maximising student learning and understanding (Bransford et al. 2000, Frederick 1986, Stuart and Rutherford 1978, are still the primary mode of 'information transfer' utilised by most places of higher education. Indeed, their function becomes even less clear when one considers that students can now easily access a wealth of useful, and often better, educational information on the internet and, increasingly, can also view recorded lectures (Cardall et al. 2008, Chandra 2011, Dupagne et al. 2009, McNulty et al. 2009, Schreiber et al. 2010, Soong et al. 2006, Traphagan et al. 2010, Vaccani et al. 2016, Young 2008. With these thoughts in mind, many educators are now utilising their relatively limited face-to-face time with students in a much more student-focussed manner in order to maximise their learning and understanding. This is primarily achieved by encouraging active student participation within lectures by, for example, having them apply new knowledge or participate in discussions, which has well described and proven pedagogical benefits to student learning and exam performance over and above passive listening to entirely didactic lectures (see inter alia Beichner and Saul 2003, Crouch and Mazur 2001, Deslauriers et al. 2011, Gannod et al. 2008, Hake 1998, Knight and Wood 2005, Smith et al. 2009).
However, a particular difficulty for any educator interested in delivering 'active' learning lectures lies both in identifying a learning activity or technique that both fits the particular demands of the course(s) upon which they teach, and which will also obtain sufficient student 'buy in' for it to work. For example, previous attempts by us to 'flip the classroom' (Bergmann and Sams 2012, Betihavas et al. 2016, Crouch and Mazur 2001, Gillispie 2016, Morgan et al. 2015, O'Flaherty and Phillips 2015, Pierce and Fox 2012, Sharma et al. 2015, Street et al. 2014, Tolks et al. 2016, whereby GEM students were asked to study material before attending class so that scheduled lecture time could be utilised more fruitfully to ask questions and stimulate discussion, were not particularly popular with these students. The two main reasons given for their dislike of this teaching method were that, a) they did not have enough time to prepare in advance for lectures and, b) they still wanted 'normal' lectures (in spite of their aforementioned pedagogical imperfections).
Therefore, given these particular constraints, we had to devise a strategy which would both allow us to still apply the principles of active learning within lectures to increase student participation but, at the same time, not have the students perceive either that they were receiving 'extra' work, or that there was a significant change in format of their lectures. To this end, we hypothesised that utilisation of the cloud-based SRS, Socrative, in lectures would provide us with just such a solution. Further, it had the added attraction to the students of being a novel, hi-tech tool which was accessible using their mobile devices, which we felt would strongly appeal to this generation's innate desire to utilise technology during the learning process (Prensky 2010).
The results displayed herein appear to bear out our initial hypotheses and concur with previous studies using this SRS (Awedh, Mueen, Zafar and Manzoor 2015, Coca and Ciesielkiewicz 2014, Dervan 2014, Kaya and Balta 2016, Liu and Taylor 2013, Wash 2014. For instance, the utilisation of Socrative did indeed prove to be a very simple way of increasing student participation within lectures which, importantly, the students themselves viewed very positively. Furthermore, from the educator's standpoint, it's deployment did not significantly impact upon the amount of course content which could be covered in each session. Furthermore, this approval was almost universal across gender and educational background subgroupings, albeit with a small, but significant, decrease in rating by the female non-biomedical subgroup. Equally importantly, the vast majority of students felt that it improved their learning, and therefore clearly did not view its use as a simple 'gimmick' employed by the lecturer for their amusement. Indeed, nearly a third of the class explicitly stated that they would like to see it employed by other disciplines teaching on the UCC GEM program. Although we were unable to test this perception of improved learning empirically, numerous studies support a connection between increased student interactivity and learning outcomes (Cain and Robinson 2008, Caldwell 2007, Keough 2012, Lantz 2010, Liu and Taylor 2013, Michael 2006, Vicens 2013.
One additional benefit of using Socrative, which had also been observed by Awedh et al. (2015), was its facilitation of in-class discussion. Significantly, in-class discussion, in and of itself, has been conclusively shown to improve student learning (Knight and Wood 2005, Liu and Taylor 2013, Smith, Wood, Adams, Wieman, Knight, Guild and Su 2009). This was an interesting observation for us as we had originally expected each student to log in and answer individually (although the lecturer (MGR) did not make this mandatory). However, it very soon became clear that not all students were logging in to Socrative, but instead that small ad hoc discussion groups formed organically amongst themselves with only one of the group actually logged in. On some occasions, there were as few as 22 respondents to a Socrative quiz even though it was clear that virtually the whole class was fully engaged with the exercise. By virtue of the fact that in-class discussion also frequently took place after the correct answer had been revealed, particularly if the students had collectively answered it poorly, a useful two-way learning experience was opened up (Bates et al. 2006, Burnstein and Lederman 2001, Liu and Taylor 2013. Thus, for the students, I could work through the problem with them so that they understood why a particular option was correct, but it also helped me to identify areas where I had not initially been clear, thus enabling me to improve my teaching in future.

Impediments to using Socrative
One of the main reservations about using SRS tools such as Socrative is that course content would have to be sacrificed as a logical consequence of devoting more time to questions and discussions. A counter argument to this is that students benefit more from an in-depth understanding of underlying concepts and learning to problem solve than rote learning facts 17,50 . Although in a broad educational sense this is undoubtedly correct, for certain professional courses such as pre-clinical medicine, elements of the prescribed curriculum cannot be omitted simply because there hasn't been enough time to teach them. The suggestion that the responsibility for learning extra material outside of class should lie with students (Bates, Howie andMurphy 2006, Knight andWood 2005) is not an unreasonable one but has to be gauged against the existing workload of such students for their other classes. For example, our own attempt at semi-flipped classroom teaching on an accelerated GEM course, whereby students were expected to watch specified material before each class and then be questioned on, or discuss, it in class, largely failed as many students, already struggling just to keep up to date with the content of their daily lectures, resented the imposition of additional 'homework'. The response of over a third of the class was to simply stop attending Physiology lectures as, without having done the preparatory work, they felt left out of the interactive sessions.
However, as described herein, using Socrative to interrogate students' knowledge of specific concepts or their ability to apply certain concepts using a small number of specifically targeted questions can be achieved with relatively minor tweaks to pre-existing lectures. For example, with regard to my own lectures I found myself eliminating superfluous information that had been retained simply because the lectures already fitted their allocated fifty minute slot.

Student Wear out
One other possibility that might affect student positivity towards systems such as Socrative is student 'wear out'. That is, student's might simply get bored of this type of SRS once the initial novelty wears off. However, Wang (2015), in the only study that we are aware of that has looked at this potential problem, albeit with the game-based SRS, Kahoot, found no fall off in 'student engagement, motivation or learning' even after using it continuously for five months. However, as the lack of student wear out with this system was attributed to the competitive nature of Kahoot, it remains to be seen if using Socrative will be so resistant to student habituation. Rae M, O'Malley D MedEdPublish https://doi.org/10.15694/mep.2017.000004 Page | 11

Limitations of the study
There are three principal limitations of our current study. Firstly, the data were obtained from one relatively small cohort of students, within a single taught module, attending a particular Medical School. Thus, in the absence of further studies, it is difficult to extrapolate our findings to other GEM programmes. Secondly, as this study was carried out principally to determine the feasibility of using cloud-based SRSs within the GEM classroom, we did not select a randomized control group to experience the same classes without utilising the Socrative quizzes. However, prior to the module in which Socrative was employed, the same students surveyed for the current study did complete another fourteen week 'basic' sciences module (GM1001) in which the Physiology content was delivered in a traditional didactic format, without utilisation of Socrative. Indeed, most students on the GEM program entered the course having completed undergraduate degrees in which didactic lectures are the norm. Additionally, prior to commencing the present study, when asked, none of the students in the class indicated that they had previously even heard of Socrative. Thus, students' extensive experience of lectures without Socrative, effectively enabled them to act as their own controls (i.e. the vast majority clearly indicated that they preferred lectures in which Socrative was, rather than was not, used. Finally, although students did perceive that using Socrative did improve their understanding of the taught material, we did not empirically assess whether or not this did actually translate into an improvement in examination performance. However, although this is a future study goal, it is likely that we would continue to use Socrative as a teaching tool, almost irrespective of the outcome of such a study (unless it significantly negatively affected examination scores), such was its popularity and success in engaging students within lectures.

Future plans
Given the success of this initial pilot study we plan to utilise Socrative in all future Physiology lectures on the GEM course, but possibly with some minor alterations in its delivery. For example, given the reported attention spans of only seven minutes (Baker et al. 2011) (or even shorter; Bunce et al., 2010 ) of the current generation of learners, a useful proposal is to break the lecture itself up into 'digestible blocks', each separated by a question or two related the completed section (Liu and Taylor 2013) as an aid to consolidate that material.
Equally, it may be instructive to run Socrative at the start of lectures either to assess students' understanding of the material to be presented so that the lecture can be tailored accordingly, or to test retention of material from previous information sessions. Finally, and as described by Liu and Taylor (2013), Socrative could also be used to gather immediate student feedback on the teaching session itself, such that subsequent lectures could be revised or modified to reflect the feedback received.
Finally, as discussed above, we wish to assess whether the deployment of Socrative within classes actually results in a measurable improvement in grades.

Conclusions
The evidence presented here strongly supports the use of SRSs such as Socrative as an aid to enhancing perceived student learning outcomes even in accelerated graduate entry programs such as GEM. Whereas lecturers might have previously been concerned that the use of such technology would be perceived as gimmicky or a waste of time by the more mature students enrolled upon these types of professional courses, who will also have generally come from previous degrees where traditional course structures prevail (Knight and Wood 2005), our study suggests that, in Rae M, O'Malley D MedEdPublish https://doi.org/10.15694/mep.2017.000004 Page | 12 contrast, these students wholeheartedly embraced this increased classroom interactivity. However, as discussed by others, in order to be effective, SRSs like Socrative and, more specifically, the questions that are utilised to engage students, need to be designed appropriately such that the technology always supports, rather than distracts from, pedagogy (Gray and Steer 2012, Liu and Taylor 2013, Watson 2001).

Take Home Messages
1) Little is known about how graduate entry to medicine students perceive the use of cloud-based student response systems like Socrative in the classroom.
2) The integration of Socrative into lectures in a pre-clinical physiology module was overwhelmingly popular with students.
3) Students indicated that Socrative use improved their understanding of the taught material even though that was not empirically tested here.

Notes On Contributors
Mark G. Rae is a lecturer/principal investigator in the Department of Physiology at UCC and holds a Ph.D. in Physiology and a Masters degree in Teaching and Learning in Higher Education.
Dervla O'Malley is a lecturer/principal investigator in the Department of Physiology at UCC and holds a Ph.D. in Neuroscience and a Diploma in Teaching and Learning in Higher Education.