Assessment of students’ satisfaction with a student-led team-based learning course

Purpose: To date, no studies in the literature have examined student delivery of team-based learning (TBL) modules in the classroom. We aimed to assess student perceptions of a student-led TBL elective. Methods: Third-year pharmacy students were assigned topics in teams and developed learning objectives, a 15-minute mini-lecture, and a TBL application exercise and presented them to student colleagues. Students completed a survey upon completion of the course and participated in a focus group discussion to share their views on learning. Results: The majority of students (n=23/30) agreed that creating TBL modules enhanced their understanding of concepts, improved their self-directed learning skills (n=26/30), and improved their comprehension of TBL pedagogy (n=27/30). However, 60% disagreed with incorporating student-generated TBL modules into core curricular classes. Focus group data identified student-perceived barriers to success in the elective, in particular the development of TBL application exercises. Conclusion: This study provides evidence that students positively perceived student-led TBL as encouraging proactive learning from peer-to-peer teaching.


culty roles i
volving course design and delivery have reported conflicting data on student satisfaction and learning outcomes [2].Additional research in this area is warranted, and in the setting of TBL, it is particularly novel.Furthermore, the implementation of TBL involves limitations such as an increased workload and time constraints for the supervising faculty mem-bers [3].It is important to note that research has indicated that students that can provide effective learning experiences to each other [3].Exploring ways to deliver TBL courses with more student involvement may help alleviate faculty workload while maintaining a high-quality, satisfying student learning experience.This study aimed to assess student perceptions of an elective course with a TBL format, in which the students generated and delivered the majority of the content with faculty facilitation.We expected that by having a greater involvement in the delivery of the course, students would gain a better understanding of TBL pedagogy.Additionally, we hypothesized that student-generated TBL modules could potentially contribute to improving students' self-perceived understanding of their assigned teaching topic, as well as their self-directed learning skills and overall satisfaction with the course.


METHODS

In 2008, the California Northstate University College of Pharmacy was established, and was one of the first pharmacy http://jeehp.orgJ Educ Eval Health Prof 2015, 12: 23 • http://dx.doi.org/10.3352/jeehp.2015.12.23 programs to include a fully integrated TBL curriculum.The elective involved in this study was a two-credit hour course called 'diabetes management and the ambulatory care setting.' The course utilized both

culty an
student-led TBL presentations throughout the curriculum to achieve student learning outcomes.When students were presenting TBL modules, faculty primarily served as facilitators to augment or clarify information.In addition, students performed a variety of specific active learning activities throughout the semester to help achieve student learning outcomes, including a patient simulation activity, patient interview exercises, and physical assessment labs.Students were also required to reflect on their class experiences by completing weekly journal entries and a final reflection paper.A summary of the course activities is provided in Table 1.An example of a rubric used in the course is shown in Appendix 1.

A major component of the course was the development and delivery of student-led TBL modules on a variety of topics related to diabetes mellitus.The first day of class included a faculty-led 'how-to' presentation on constructing TBL modules, which included a discussion of TBL background information, essential elements of TBL, Bloom's taxon my, techniques for building mini-lectures, and the construction of effective application exercises.The presentation was supplemented with time for discussion, in which student teams asked any questions that may not have been answered by the presentation.In order to evaluate student perceptions of student-led TBL, a ques-tionnaire was administered and a focus group was conducted.All students enrolled in the elective were eligible to participate in the questionnaire.In addition, seven student volunteers participated in the focus group.The use of both questionnaires and focus groups allowed the investigators to obtain a more complete understanding of the students' perspectives regarding student-led TBL.

A survey instrument was developed specifically for this study, containing nine four-point Likert-type items (Table 2).The items assessed the benefits and barriers that students perceived while creating their own TBL modules, as well as their perceptions of the impact that student-led TBL sessions had on their learning experience.A focus group g ide was also developed (Table 3) to elicit specific examples of how having students lead TBL sessions affected their understanding of the topics covered during the elective.

Volunteer students anonymously completed the survey during the last day of classes.The focus group session was held a week after the final exam, and was moderated by an investigator not involved in the class in order to minimize bias.The focus group discussion lasted 45 minutes, was audio-recorded, and was transcribed verbatim for analysis.All focu group attendees received $5.00 gift cards.This study was approved by the California Northstate University College of Pharmacy Institutional Review Board.

The questionnaire responses were summarized as frequencies and percentages.Thematic analysis was conducted on the focus group data once accuracy of transcription was ensured.Two independent investigators (AH and VG) analyzed the focus group data and subsequently met to resolve discrepancies in the topics identified.Descriptions were generated in the re evant sections of the data (codes), and the descriptions were interpreted to identify major themes.Due to the exploratory nature of this study, themes were generated inductively.The focus group data analysis was conducted by hand, given the small amount of data.


RESULTS

A total of 30 students enrolled in the class, all of whom completed the questionnaire.Seventeen respondents were female.The results of the questionnaire responses are shown in Table 2. Fifty percent of the students agreed, and another 50% strongly agreed, that they would recommend this elective to another pharmacy student.More than 80% of the resp

dents ag
eed or strongly agreed that the class helped them better understand TBL pedagogy and improved their self-directed learning skills.While 76.6% of the students felt that generating the TBL modules helped them better understand the topic of the module, 62% felt that generating TBL modules resulted in too much extra work.Seventy-three percent of the students did not agree that they were more satisfied with student-led TBL modules.Still, less than a third of the respondents felt that the student-led TBL format was a barrier to achieve learning outcomes.Moreover, 96.7% of students agreed or strongly agreed that they achieved the student learning outcomes.Individual, team, and overall course grades were positive and indicated that students comprehended the content of the course.The average scores on the midterm examination and final examination were 83% (range, 66% to 94%) and 91% (range, 82% to 100%), respectively.The average course grade was 91%.

Four female and three male students volunteered to participate in the focus group discussion.Thematic analysis of the focus group data identified four common themes.The next sections describe these themes in detail with supporting quotes.


Theme 1: students experienced challenges in developing team-based learning applications

During the focus groups, discussion part cipants stated that they experienced difficulties developing the team application exercises for their classmates and indicated their frustration in  completing the applications developed by other teams.Some of the applications were perceived

o measure lower-level thinking skills.

"One thing that I was hoping to be a little bit
ore challenging was the applications.I thought that the applications were pretty easy and I was hoping that they were a little bit more challenging and more engaging."

"I guess for me coming up with applications was really hard.Making a good application is really hard.It's just not a basic answer whereas we had come up wi h applications and a lot of times it was just crossword puzzle multiple choice or true or false, because we don't know how to get people thinking outside the box."


Theme 2: students valued delivering the team-based learning modules

Students identified presentation skills and public speaking as areas that were improved by delivering a TBL class.Participants expressed their willingness to recommend or retake the class as it was.

"To be honest also if there was ever a time where I did have to present something I would learn the materia

a little bit better than the other material."

"It gives you an idea
to increase your public speaking skills because you are always there presenting."

"I did learn more about my specific subject at that time, once it came down to reviewing and taking the test the I di learn them all at the same level but for just for my TBL session I did master my material before I probably would've mastered other material."


Theme 3: students needed faculty reassurance to help reinforce major concepts

Students expressed their desire for irect faculty involvement with the TBL learning objectives and summary development.The participants considered that refining learning objectives requires closer faculty input for the students to feel assured that the objectives focus on the most important concept

and align with the goals of the class.Likewise, the participants indicated a p
eference to have faculty provide a summary after students delivered the class or for faculty to help students prepare a summary to conclude each class.


"I would also myself have a summary at the end printed or whatever to draw everyone back at what is the major focus here. I think that would take care of not only of the rushing but also getting us focused back to what is it that it's really important. " "Students doing the learning objectives sometimes it is not enough. They don't recognize which one is important. " Theme 4: st

as the journal and patient simulation exercise.


"Because of the day it fell onto we had other
two classes that we had to study for so it always seemed that I was always playing catch up in this class and there was too much reading even with the other turns group being on top of their presentation in this and I literally never had time to catch up. "

"I think that was more problem of the fact that it was on Wednesday morning tha

we really didn't get to really focus on times when we wanted to on a day that where we had to prepare for three classes so we never got to really spend the time that we wanted to."

"There was actually a lot of outside work to do.For example the journal entries that I always almost forget to do unless s
meone posted it on Facebook.You're testing your diabetes, you're preparing the TBL, you're reading there were so many outside thing I kept on forgetting and the it was just like oh my gosh really?"


DISCUSSION

The objective of this study was to assess students' perceptions of student-led TBL in an elective course, with the hypothesis that students would gain a better understanding of TBL pedagogy, as well as improving their self-perceived understanding of the assigned topics and their self-directed learning skills.While these objectives are certainly valuable in terms of achiev

g success i
curricular outcomes, meeting these goals may have the added value of potentially reducing faculty workload in course development and in the classroom.Survey results suggested that this student-led TBL course helped improve students' understanding of TBL pedagogy and their self-directed learning skills, as well as enhancing their understanding of the topics, thus achieving the goals of incorporating student-led TBL into the classroom.The survey results showed that the majority of students felt that they understood TBL pedagogy better and gained a deeper understanding of the topics.Performing TBL compels students to gain mastery of basic content in order to teach their peers higher-level applications, which may translate into a deeper understanding of http://jeehp.orgJ Educ Eval Health Prof 2015, 12: 23 • http://dx.doi.org/10.3352/jeehp.2015.12.23 various topics.The majority of students also felt that they improved their learning skills, which may have been because TBL requires a higher level of organization prior to implementation.

While our study did not specifically measure the impact of student-led TBL on faculty workload in the classroom, one could speculate that there is potential for student-led TBL to have a positive impact on the time spent on building modules and preparing for classroom teaching.In our elective, while faculty co tinued to provide support regarding the identification of key topics, required readings, and individualized readiness assurance test questions, the students were responsible for background research in addition to the readings provided and the generation of learning objectives, mini-lectures, and application exercises, which substantially reduced the workload of the course coordinators.This was somewhat compensated for by the faculty time required to effectively review student-generated content before delivery, although the workload was reduced overall.Students identified the most beneficial parts of the course, as well as the most significant challenges.In the focus group, the students agreed that incorporating the simulation activity into the course provided them with a valuable hands-on learning experience.Multiple previous studies evaluating the inclusion of a patient simulation component in courses have established the value of such exercises in improving student empathy, which is consistent with the results of our study [4][5][6].The focus group results also showed that students identified time constraints in delivering their TBL modules and their overall workload as their biggest challenges.The authors allotted one hour of each two-hour class for student groups to present their modules, and students consistently noted that they felt rushed in delivering their material, especially their application exercises.

The focus group participants also reported that another challenging aspect of the course was creating TBL applications.These exercises are an essential component of TBL, designed to foster higher-level critical thinking and team problem solving [7].The focus group participants also experienced difficulties in identif ing the most important concepts to extract from the assigned readings in order to build effective learning objectives.They experienced a similar difficulty in creating the applications due to their limited experience in applying clinical information.Moreover, students noted their preference to have faculty input during these processes and to have faculty members review important material to help ensure that learning outcomes were met.

The results of our research were similar to the results that have been reported for other peer-to-peer models of teaching [8].A previous study has found that peer-to-peer teaching allowed students to better perceive the challenges that other students face when learning and to provide better feedback.More-over, perception regarding decision-making dynamics and group support have been reported to be more positive in student-led groups compared to faculty-led groups [8].In addition, the students perceived improvements in their interpersonal communication, presentation, teamwork, leadership, and evaluation skills, which are all essential for future pharmacy graduates to function as a cohesive team in a wide range of healthcare fields and to provide the most competent care to patients.

The results of this study should be interpreted in light of its limitations.While the study suggests that student-led TBL contributed to positive student perceptions relating to understanding and satisfaction, without a comparator group, the authors cannot conclude that the TBL pedagogy was solely responsible for these obse vations.Second, the focus group included students who had completed the survey assessing their perceptions regarding student-led TBL, which could have influenced their responses.Selection bias should also be considered, since the focus group participants were volunteers and may not have represented the views of the entire class.As we did not anticipate the challenges associated with the development of application exercises, information was only acquired about this issue through the focus group.Future questionnaires should include this dimension in order to ascertain whether a comprehensive discussion illustrating how to create valuable application exercises would improve students' understanding of how to create complete TBL modules.In addition, future focus groups should inquire about how students approach the development of applications, as well as identifying the resources that students used and whether they modeled their applications on those that they had previously seen in the classroom.Finally, the sample size for the survey was small.Therefore, it may be difficult to generalize the results of this study widely.

Moreover, while peer-to-peer teaching in academia is relatively well studied, its use within TBL pedagogy itself is novel; however, more robust studies are needed to shed light on the connection between student-led TBL and improvements in learning, understanding, and satisfaction.This pilot study suggests that students may perceive student-led TBL positively, and that further studies are warranted.

ORCID: Justin W. Bouw: http://orcid.org/0000-0002-8625-1841; Vasudha Gupta: http://orcid.org/0000-0003-0169-3120;Ana L. Hincapie: http://orcid.org/0000-0002-6142-1744Team develops quality PowerPoint presentation with content that displays complete understanding of topic, and formatting and referencing are professional and consistent.

Team application Team develops effective application to enhance understanding of topic and challenge critical thinking.

A plication exercise lacked creativity and effort and did not challenge critical thinking or enhance understanding of topic.

Application exercise was developed but was minimally effective in challenging student' s critical thinking and/or enhancing understanding of topic Application exercise is acceptable and challenges student' s critical thinking skills to enhance understanding of topic.

Application exercise demonstrates creativity and innovation to challenge student' s critical thinking skills and enhance understanding of topic.Does not communic te the presentation and module effectively.Team is unfamiliar with case; reads the handout or slides.Team cannot answer questions appropriately.

Communicates the pr sentation and module in an unorganized and unfamiliar manner.Team answered questions with lack of confidence or little detail.

Communicates the pre entation in an organized, familiar manner that does not impede understanding.Answered questions with sufficient detail and accuracy.

Clearly and effective y communicates the presentation and module in an organized manner that enhances understanding.Answered questions thoroughly, accurately, and with confidence.Time management Team displays effective use of time to administer topic module.

Team either rushes thro gh presentation and activity (completed < 30 min) or not able to start application due to length of presentation.

Team completes presenta ion but did not deliver effective activity due to lack of appropriate time management.

Team completes all port ons of module and exercise but had to rush through portions of the activities in order to complete.

Team utilizes time effe tively to complete all portions of module and active learning exercise.


Exam questions Develop

appropriate exam questions based on module content

Assignment not complete
.

Team develops < 10 exam questions and/or question structure significantly lacks ability to evaluate basic knowledge.

Team develops 10 exam q estions but structure of some questions needs to be revised to more appropriately evaluate basic knowledge.

Team develops 10 exam q estions based on module content that appropriately evaluates basic knowledge.



niques to deliver mat