Student Response to Team-Based Learning and Mixed Gender Teams in an Undergraduate Medical Informatics Course

Objectives: Team-based learning (TBL) is increasingly being used in medical and medically-related courses. The aim of this study was to evaluate student responses to a TBL-based course in medical informatics at the Sultan Qaboos University, Oman. Methods: A total of 11 TBL sessions were run during a 14-week semester of the Medical Informatics II course at SQU. An online student evaluation was performed in week 13 of the course. Results: Of the 108 students on the course, 96 (88.9%) of the students responded to the survey. For the most part, the students regarded TBL favourably, and derived great benefit from the TBL sessions. Cultural norms, however, appear to have impacted negatively on the females’ experience of the TBL sessions. Conclusion: TBL’s benefits in the medical informatics course were recognised by the students. In such an environment, however, facilitators will have to bear in mind and continually address cultural issues.

2][3][4][5][6][7][8][9] Nevertheless, it is useful to note that a TBL session emphasises active learning and follows a process during which students receive stimulus material (e.g.readings), perform a short multiple choice question (MCQ) test taken individually, followed by the same MCQ test taken in teams of 5-8 students.The two tests are frequently referred to as the individual readiness assessment test (IRAT), and the group readiness assessment test (GRAT) respectively. 2,4,7,10he scores from the tests usually contribute significantly towards the final course assessment.Feedback on the GRAT is given immediately in class, and the TBL session continues with a more in-depth question in which the information learned is applied.This is followed by further feedback and clarifications from the facilitator.
][18][19][20][21][22] In 2010, the College of Medicine and Health Sciences at Sultan Qaboos University (SQU) in Oman hosted TBL workshops for staff who were interested in using TBL in their courses.In 2011, the author implemented TBL in the Medical Informatics II course.This course has 4 contact hours per week, builds on the more basic Medical Informatics I course, runs through a 14-week semester, covers a range of theoretical and practical aspects of medical informatics, and is designed to equip the students with the foundational attitudes, knowledge, and skills in information systems required by doctors.
In the first week of the Medical Informatics II course, students were given an orientation to TBL, and were then able to run through the TBL process twice.As part of the course orientation, the need for teamwork in medical practice was described, with reference to common practices in Oman and internationally, and the link between this teamwork and TBL was made.During the course, students were frequently reminded of this by reference back to the discussions in the orientation week; the overarching philosophy was that TBL was part of their training as members of a health care team.
As part of this orientation, a small but significant change was made to the nomenclature.In much TBL writing, the words "group" and "team" appear to be used interchangeably. 9Reference has already been made to the fact that the assessment taken by the team is usually referred to as a group readiness assessment test (GRAT). 2,4,7,10The confusion of "group" with "team" is antithetical to the concept of teamwork in general, and it was felt would undermine the team aspect of TBL.As a result, the instructor never used the word "group, " and the team assessment was always referred to as a team readiness assessment test (TRAT) to emphasise that it was an assessment taken by a team.The term TRAT is used in the rest of this paper.
Because the first two TBL sessions held in the first week were practice sessions, they did not contribute to the course grade (this work was examined in the final examination).During the orientation, the instructor divided the students into 14 teams of 7-8 students each.Students were randomly chosen except in the case of gender: because of the high female to male student ratio on the course, there was a need to ensure that each team had at least two males.The teams of students remained fixed for the duration of the course.The objectives and readings for each TBL session were made available in the Learning Management System (LMS) at the start of the semester. 5In preparation for each TBL session, students had to read approximately five pages of text.
During the rest of the semester, a 2-hour TBL session was run every week for 11 weeks, while the other 2-hour weekly contact time of the course consisted of student presentations on advanced topics.During each session, the IRATs and TRATs consisted of 5 MCQs covering the basics of the topic.Students had 8 minutes to complete the IRAT, and 15 minutes to complete the TRAT.Because the attendance policy at SQU allows students to be absent from some classes, only the highest nine IRAT and TRAT scores would contribute to the students' scores.
The IRAT and TRAT scores each counted for 20% of the total score for the course.This proportion is in line with practices elsewhere, and is considered a substantial contribution. 1,3,4,10 ecause the course was a medical informatics course, classes were held in computer laboratories.As a result, the IRATs were taken online.The students then broke into their teams in the computer laboratory and continued with the TRAT.After the TRAT, application questions were given, and students had approximately 45 minutes to solve the problems.This was followed by feedback and wrap-up.
When running TBL sessions at SQU, however, one needs to be sensitive to a particular aspect of Omani culture that may conflict with a central tenet of TBL.TBL expressly relies on the diversity of the teams in respect to culture, background, and gender. 9In Oman, classes in public schools after 5 th grade are usually single-gender (although not all students at SQU are from public schools), and interaction between genders outside of one's family is generally discouraged.In the College of Medicine & Health Sciences, while classes are of mixed gender, the students separate themselves by gender in the classroom.The result is that the majority of the college's undergraduate students are not used to working in mixed gender teams.It was expected that this would have an impact on the running of the TBL sessions.Given the professional needs within this complex setting, this study set out to gauge the students' reactions to TBL in their course.

Methods
The study was conducted among the 108 medical students at SQU who were taking the Medical Informatics II course in the spring semester of 2011.An in-course evaluation of the TBL process, conducted three weeks into the course, indicated that there had been a general acceptance of the new teaching model amongst students, although there was also some reluctance and uncertainty.
The literature indicates that this is common in the early stages of TBL introduction. 7The issues and problems raised by the students regarding the process were addressed, and the process continued.
While the results of the preliminary evaluation were useful because the evaluation had been conducted early in the semester, there was the risk that results may have been influenced by the novelty of the teaching method.There was a need to determine students' attitudes towards TBL after 11 sessions of TBL across the semester.
For this study, a survey form and consent form were created.The core of the survey form was based on the 12-item survey form by Wiener, Plass and Marz. 8Students responded to statements with a Lickert scale running from 1 (strongly disagree) to 6 (strongly agree).Small language changes were made to some of Wiener et al.'s questions to account for local interpretations, and additional questions were added to gather further information about students' attitudes towards TBL.Because of the cultural issues discussed above, a crucial question regarding the comfort of students working in mixed-gender teams was also added.Finally, an open text question allowed students to add any further comments.
The in-course TBL evaluation used this revised form and also served as a pilot for students' feedback on the questionnaire.As a result of students' feedback, some minor language alterations were made to the questions.A consent form was designed, based on a standard consent form used in previous research by the author, and approved by SQU.The consent form indicated that the survey was anonymous, and students were under no obligation to participate.
The language of instruction at SQU is English, but many of the students do not have English as their mother tongue.As a result, the language of the consent form and the survey form needed to be easily accessible.Common (albeit not absolutely perfect) scales of measuring English language levels are the Flesch Reading Ease and the Flesch-Kincaid Grade Level. 2,31Using these scales, the level of English of the consent and survey forms was determined as shown in Table 1.The figures indicate that the forms would be understood by native English speakers in middle school, so the students in this study would be able to understand the forms.Ethical approval for the study was granted by the SQU College of Medicine & Health Sciences Medical Research & Ethics Committee (MREC #417).
The survey was conducted electronically near the end of the semester (May 2011), in week 13, by which time 11 TBL sessions had been run.The survey remained open for a week.All data were collected and stored anonymously and electronically, and were secured by means of passwords and 256-bit encryption.The data were placed into a Microsoft Excel 2010 spread sheet (Microsoft, Redmond, Washington, USA), and descriptive statistical analyses were performed.Wiener et al. had used the Mann-Whitney U-test to determine the differences in responses based on gender, so, for consistency in the comparison of the two studies, the Mann-Whitney U-test was also chosen for analysis of the data in this study (QI Macros SPC Software for Excel, KnowWare International, Denver, Colorado, 2012), and statistical difference was taken at P <0.05.

Results
Attendance at the TBL sessions was 96.97%, with only three students missing the maximum allowable number of two TBL sessions.A total of 96 students There was no statistical difference between the female/male ratio of the respondents and the total class population.The students' responses to the questions are given in Table 2.For purposes of reporting, following the process given by Wiener et al. 8 , the responses to the Lickert scale were reduced to three groups (1-2; 3-4; 5-6).
On the organisation and facilitation of the TBL sessions, the students were generally satisfied, and there was consistency of satisfaction across both genders.In addition, there is general acceptance of the effectiveness of the TBL process (including the discussion after the TRATs), and the belief that TBL is more valuable than normal lectures.Overall, more than half the class enjoyed the TBL sessions, but 42% were neutral.The application questions, however, were not seen to be as valuable as the IRAT and TRAT.
Obvious, however, was the significant differences between a) females and males in the enjoyment of the TBL session (4.3 vs. 5.1); b) the likelihood of recommending the method to other students (4.7 vs. 5.2), and c) the interest in the College offering TBL more frequently (4.3 versus 5.0).The greatest difference between female and male scores, and possibly the root for the other differences, is that females were far less happy than males to work in mixed-gender teams (3.7 versus 5.2).
Superficially, one might be tempted to believe that this lower figure amongst females is simply a gravitation towards the central options (3 and 4)  on the Lickert scale, and that the females did not have particularly strong feelings either way.When inspecting these figures in more detail, however, we find that this is not the case.Figure 1 indicates that, while there are fewer females agreeing with the statement, the central gravitation had not occurred.The reason for the low mean score was that a large number of females appeared directly opposed to the mixed-gender groups.A total of 19.2% of the females selected "strongly disagree" and 15.4% selected "disagree" in response to questions about working in mixed-gender teams, giving a combined score of 34.7%, as opposed to only 5.3% for the males [Figure 1].In the open text response question, two female students commented negatively on the mixed-gender teams.One of them wrote: "We are not happy to work as males and females together, so we prefer to separate us as groups of males and other groups of females." In addition, during the running of the TBL sessions, almost all the teams formed themselves into sub-groups, split on gender lines.The facilitator had to tread a delicate line of encouraging greater cohesion while ensuring that cultural sensitivities were not offended.This was usually done with reference to the need for cohesion in clinical teams.Students were informed that they were already doctors, but in their early stages of their training, and that the problems to be solved should be viewed as if they were those of actual patients in need of the best possible medical care.It was underlined that this could be provided only by a proper team of medical personnel.By the end of the semester, there was greater cohesion within teams, with most of the teams forming a circle around a computer or two.They were still, however, divided along lines of gender, and some teams remained in a single row, refusing to form a cohesive circle of any type.
An added factor that may have led to the dissatisfaction amongst the females is that the results indicate that the males did less reading preparation than the females, and many females may have felt that they were being forced to carry the load in the TRATs.
This paper focuses on student responses, and so does not report on student performance in any detail.Nevertheless, because an important reason for emphasising teamwork is to improve on individual performance, it is appropriate to report briefly on aggregated data.For the class, the mean IRAT score was 87.9% (range: 60-100%) and the mean TRAT score was 99.7%, with most teams scoring 100%.Similar scores have been found elsewhere. 10

Discussion
For the most part, there was a positive response to the value of TBL, its ability to increase knowledge, and its benefits in comparison to the value of lectures.This positive attitude has been found by several other researchers. 3,4,8,24The increase in scores from IRAT to TRAT certainly demonstrated the value of working in teams.In addition, the students mostly enjoyed the sessions.Specifically, 51% of the students enjoyed the TBL sessions, 42% were neutral, and only 7% did not enjoy the sessions.
Although the facilitator would have preferred a greater indication of enjoyment, these figures are encouraging, especially when viewed in the light of other research. 1(The issue of the male/female dichotomy is discussed below).
There may be many factors that increase student enjoyment of TBL.The results certainly indicate that the students recognise that the course was well-organised, and this would have an impact on their perception of the course.In addition, having the TBL so early in their medical training might have helped because they had not yet come to believe that PowerPoint-driven lectures are the only method of teaching at university, as has been suggested by other researchers. 1his early offering of TBL, however, may also have a great disadvantage.At this early stage, the students are not convinced of the need to work in teams, especially as so much of their other course work is individually assessed.Trying to convince students that they will need to learn to work in teams for their later student years, as well as during their medical careers, may sound like trying to sell a concept rather than preparing them for that scenario.
In this cultural setting, this would have been a particular problem and was possibly exaggerated by the fact that the course convenor and facilitator was a non-Omani male who may have been perceived to be importing foreign and culturally-insensitive practices into the curriculum.
Readers familiar with TBL may have noticed that these sessions did not have peer-evaluation.
A system of peer-evaluation is recommended as a crucial part of TBL, primarily because it encourages individual accountability, and has been implemented by several researchers. 4,5,7,9Some of these researchers, however, have reported that the peer-evaluation system is manipulated by students and also causes great dissatisfaction amongst them. 4,7For these reasons, the facilitator decided that having both the IRAT and TRAT each counting 20% of the course grade would be enough incentive for individual accountability.As a result, at this stage, there is no plan to introduce peer-evaluation; however, the course could probably benefit from future course evaluation questions on participation in teams.This would be especially important given the apparent differences between males and females in the reading preparation described above.
There is also the argument that, as TBL contributes as much as 40% of the overall grade, this could inflate the end-of-year grade.A counter argument, however, is that it makes little sense to introduce TBL, and to emphasise the need to work in teams, unless one is prepared to have it contribute significantly to the students' final marks.Allowing the students to skip 2 out of 11 sessions possibly inappropriately boosted their grades, so this allowance may be reduced to only 1 in the future.
The greatest problem encountered was that of the mixed-gender teams.9][30] This is a shortcoming that has been investigated in other cultural settings. 31As a result, this would appear to be an area for more detailed future research.
At this stage, when we compare the results of this study to Wiener et al. 's results, we can see there are similar male/female trends in the response to many of the questions. 8Simultaneously, however, when one looks at questions dealing with recommending TBL to other students, offering TBL more in the curriculum, and overall satisfaction with TBL, one sees that there is a reversal: females in this study were less satisfied than the males.
Until further research is conducted, it will be prudent for the facilitator of such a course to be vigilant about the gender issue.However, as TBL becomes used more frequently in the students' courses, the problems associated with mixedgender teams may subside.Although this study covered the impact of 11 TBL sessions, it was conducted in one course in one university only.The results will benefit from repeated studies to determine the overall student response to the use of TBL in medical informatics classes.In addition, because the cultural impact had not been foreseen, there were too few questions regarding students' backgrounds for a detailed understanding of possible reasons behind the students' responses.A repeat study should include questions on students' ages, ethnic background, schooling, and whether their family is from an urban or rural setting.

Conclusion
Team-based learning is being increasingly used in medical and medically-related courses.This paper has described the student response to the use of TBL in a medical informatics course in the College of Medicine & Health Sciences at SQU in Oman.For the most part, the students accepted the use of TBL in the course and perceived the benefits; however, issues pertaining to Omani cultural norms may have impacted negatively on the females' perceptions of TBL.Because working in mixedgender teams is crucial to the delivery of health care, facilitators working in such sensitive cultures will need to address the issue continually so that that the students' experience of TBL is as positive as possible.The author is the sole author of the paper.There are no known conflicts of interest.

Figure 1 :
Figure 1: Graph showing percentages of responses to the question "I am happy working in a TBL team with males and females together." 1= strongly disagree; 6 = strongly agree.

A
C k N O w L E D G E M E N T S I am indebted to Dr. Nadia Al-Wardy for comments on an earlier draft of the paper, to my students for participating in the TBL sessions and the survey, and to the reviewers who commented on an earlier version of this paper.D I S C L A I M E R A N D C O N F L I C T O F I N T E R E S T

Table 1 :
Flesch Reading Ease and Flesch-Kincaid Grade Level of the consent and survey forms

Table 2 :
Student responses to questions in the TBL survey (N = 96)