Does curricular change improve faculty perceptions of student experiences with the educational environment? A preliminary study in an institution undergoing curricular change

Purpose: College of Medicine, Gulf Medical University, United Arab Emirates, underwent a major curriculum change from a discipline-based to an organ system-based integrated curriculum. However, it was not known how the faculty perceived the changes in the educational environment as experienced by the students. In this context, we aimed to compare the faculty perceptions of the student experiences in the discipline-based curriculum with those in the organ system-based integrated curriculum. Methods: The Dundee Ready Education Environment Measure (DREEM) questionnaire was modified to assess faculty perceptions of the student experiences, pilot-tested, and administered to all faculty members (n=28) involved in the discipline-based curriculum (FDC) in January 2009. In the subsequent year, data were collected from the same faculty involved in the new integrated curriculum (FIC). Collected data were transferred to Predictive Analytics Software version 18. Total, domain, and individual statement scores were assessed with the Wilcoxon signed rank test. Percentage agreement, disagreement, and uncertainty were assessed by the McNemar’s test for proportion. Results: The mean total DREEM score was significantly higher (P<0.001) for FIC (139/200) as compared to FDC (119/200). The FIC perceived significantly more positive student experiences with the educational environment as indicated by the domain scores and statement scores. The differences in proportions of agreement between FIC and FDC also reinforced that the FIC perceived more positive student experiences with the educational environment. Conclusion: The study showed that the faculty perceived the organ system-based integrated curriculum as providing a better educational environment for the students than the discipline based curriculum.


INTRODUCTION
The importance of gauging the educational environment of a medical school has been well recognized. Positive environ ment and student achievement, satisfaction and success are intimately linked. Learning environment and curriculum are also closely associated [1]. The College of Medicine, Gulf Me dical University, United Arab Emirates, had a traditional dis ciplinebased undergraduate curriculum for over 10 years. However, reviews of the traditional curriculum, guidelines from Ministry of Higher Education (accreditation body in United Arab Emirates) and a desire to keep up with the global changes in medical education resulted in a major curricular change from the traditional curriculum to an organsystem based integrated curriculum with elements of problem based learning. A more studentcentered approach using less didac tics and more interactive sessions (small group learning, com puter aided learning, case based learning, problem based learn ing) was the focus of the new curriculum. The new curricu lum, in place from 2008, is imparted in three phases. The first phase (of one year duration) involves an introduction to med ical sciences with courses like Cells, Molecules and Genes, Tissues and Organs, Embryogenesis and Life Cycle, Metabolism and Nutrition, Internal and External Environment, Language and Communication Skills, and Psychosocial Sciences. The next two years of Phase II are devoted to the integrated study of all organ systems and the final Phase III (of two years duration) includes the clinical clerkships. Subsequently, the students un dergo one year of compulsory rotating internship.
While a lot of attention has been given to the students' per spective about the educational environment [25], very few studies have looked into perspective of the other stakeholders such as the teaching staff. Previous studies comparing the fac ulty perceptions with those of the students revealed that the faculty and students had similar perceptions about the student experiences in an integrated curriculum [6,7]. The perceptions of the educational environment by medical teachers are sig nificant not only for themselves but also for the students. The environment of any medical school will be a determinant of the behavior of the students and teachers of the medical school [1]. Hence, the quality of the educational environment in any institution can be improved by studying the teaching faculty's perceptions. The faculty viewpoints about the student experi ences in the educational environment prior to and after the curricular change at College of Medicine were not explored in depth. Hence, the objectives of our study were to measure and compare the faculty perceptions of the student experiences with the educational environment in the disciplinebased cur riculum with that in the newly introduced integrated curricu lum at College of Medicine.

Instrument
The validated and highly reliable Dundee Ready Education Environment Measure (DREEM) has been used by a number of medical colleges to assess the learning environment as per ceived by the students. DREEM has been used to diagnose the positive or negative aspects of individual institutions; to iden tify the perceived weaknesses of a new curriculum and to com pare perceptions of medical students in disciplinebased cur riculum with those in innovative curriculum [25]. A modi fied version of the DREEM inventory also assessed the per ception of teachers and compared it with the students' percep tions [6,7].
The DREEM inventory involves 50 statements divided into 5 domains which are: imum score is 28. The total score for all domains is 200. Each statement is scor ed from 04 with 4, strongly agree; 3, agree; 2, unsure; 1, dis agree; 0, strongly disagree. Nine negative statements are scored in reverse for analysis.
The main aim of this paper was to assess the faculty percep tions of students' experiences with the educational environ ment in a curriculum. However, to the best of our knowledge there is no appropriate instrument available to assess the views of the faculty. As the DREEM questionnaire was originally developed to give only the students' perceptions of the educa tional environment, hence, in our study it was modified so that certain statements in the first person (e.g., I am encouraged to participate in class.) were rephrased to allow the faculty to give their opinions about the student experiences in our in stitution. (e.g., The students are encouraged to participate in class.) Permission to modify the instrument was obtained from the authors of the original DREEM. The content validity of the modified DREEM questionnaire was obtained through a review process with a medical education expert and a socio psychologist. The questionnaire was pretested on a group of faculty members, to ensure face validity and this data was ex cluded from final analysis. Following the pilot study, descrip tive phrases were added to some statements of the question naire for better understanding. In addition, openended re sponses about the students' experiences were also encouraged.

Sample
The 30 faculty members of College of Medicine were con tacted personally by the authors. Explanations of the objec tives and the method of filling out the questionnaire were giv en. The faculty was specifically instructed that their percep tions should be about the students' experiences in the previ ous academic year. Any statements dealing with students' per sonal aspects of the environment (e.g., Students have good friends in this school.) or outside of the context in which some of the staff encountered the students (e.g., Teachers are patient with the hospital patients.) were encouraged to be responded as unsure. Voluntary participation was stressed upon with informed consent obtained from the faculty.
DREEM questionnaire was answered anonymously by the faculty involved with students of the 2007 batch (last batch of disciplinebased curriculum; n = 28; FDC) in January 2009 http://jeehp.org J Educ Eval Health Prof 2014, 11: 7 • http://dx.doi.org/10.3352/jeehp.2014. 11.7 and in the subsequent year (January 2010), by the same faculty who were now teaching students of the 2008 batch (first batch of integrated curriculum; n = 28; FIC). Data from those staff involved in teaching in both the consecutive academic years (2009 and 2010) were selected for final analysis.
The study was approved by the Ethics Review Committee of the institution.

Statistical analysis
Data was analyzed using the PASW ver. 18.0 (SPSS Inc., Chi cago, IL, USA). The mean total, domain and individual state ment scores were expressed as mean± SD. Comparison of scores was done using Wilcoxon signed rank test. Pvalue less than 0.05 was considered as statistically significant.
Due to the anticipated large number of unsure responses, the faculty responses to individual statements were also classi fied into three categories: agreement (strongly agree or agree), disagreement (strongly disagree or disagree), and unsure. The McNemar's test for proportion was used to compare the per centage agreement of staff giving responses in each category using a significance level of 0.05.

RESULTS
A total of 28 questionnaires were returned out of 30 subjects. Majority of the faculty were female (64%), senior faculty (61%), those with teaching experience of more than 10 years (68%), and from preclinical/paraclinical departments (82%). The faculty had multiple teaching roles as lecturers/seminar lead ers and as tutors for tutorials, casebased learning (CBL)/prob lembased learning (PBL) groups.
The mean total DREEM score for FDC and FIC was 119/200 and 139/200, respectively (P< 0.001). The mean domain scores obtained by FDC and FIC are shown in Table 1. Interpreta tion of domain scores revealed that both groups perceived moving in the right direction for the teachers as the scores for the domain Students' Perceptions of Teachers ranged from 23 to 33; feeling more on the positive side for the academic self perception as the domain scores for Students' Academic Self-Perceptions were between 1724; a more positive atmosphere for the atmosphere as scores for domain Students' Perceptions of Atmosphere were between 2536 and not too bad for the so cial selfperceptions with scores of Students' Social Self-Perceptions ranging from 15 to 21. FDC perceived a more positive approach whereas FIC perceived teaching highly thought of for the learning as the scores for domain Students' Perceptions of Learning were 28.3 and 36.9, respectively. However, all mean domain scores for FIC were significantly higher as compared to FDC.
FDC identified Students' Perceptions of Teachers as the do main with highest mean score, whereas FIC gave highest scores to the domain Students' Perceptions of Learning. Both groups gave lowest mean scores to the domain Students' Social Self-Perceptions. On analysis of the mean scores of individual state ments, mean scores 3 and above were considered as areas of strength; mean scores greater than 2 and less than 3 were con sidered as areas that could be improved and mean scores equal to or less than 2 were areas of weaknesses. FDC and FIC gave the highest score for the statement the teachers are knowledgeable. FDC did not feel that the teaching is student centered and so this statement had the lowest score. FIC gave the lowest scores to the statement stating the students' accommodation is pleasant (Tables 2, 3).
For FDC, there were 10 statements (20%) indicating the ar eas of weaknesses; three statements in the positive area (6%), and the remaining statements were in the areas that could be improved (74%). However, FIC showed a more positive per ception as only 1 statement had a mean score of 2 or less (2%); 17 statements had a mean score above 3 (34%), and the rest were in the areas that could be improved (64%) (Fig. 1). On comparison of percentage agreement, FIC tended to agree significantly more than FDC for 4 statements (Table 4). Anal  The teaching is often stimulating.
1.82 ± 1.020 2.54 ± 1.071** 38 The students are clear about the learning objectives of the course.
2.54 ± 0.881 3.21 ± 0.568*** 48 The teaching is too teacher-centered. Values are presented as mean ± SD scores of statements with significant differences. Negative statements are in italics with the reversed scores inserted into the Table. Higher a score for these statements, the more the faculty members disagreed to the statement. FDC, faculty teaching in discipline-based curriculum; FIC, faculty teaching in integrated curriculum. *P < 0.05, **P < 0.01, ***P < 0.001.

ysis of percentage disagreement identified one statement with which FIC significantly disagreed. (The teaching is too teachercentered.)
The unstructured qualitative data from the openended re sponses also provided insight into the respondents' views. Most of the faculty supported the curricular change. However, one of them felt that there was less emphasis on basic sciences knowl edge in the new curriculum.

DISCUSSION
The success of an effective curriculum depends on a positive educational environment [1]. Any change made to the curric ulum will invariably result in changes to the environment. To obtain the best outcomes, teachers must also be committed to any changes in the curriculum. Hence, we have studied the changes in the educational environment, as perceived by the teachers, after a shift to a more student centered curriculum.  Values are presented as mean ± SD scores of statements without significant differences. Negative statements are in italics with the reversed scores inserted into the Table. Higher a score for these statements, the more the faculty members disagreed to the statement. FDC, faculty teaching in discipline-based curriculum; FIC, faculty teaching in integrated curriculum. Since we could not find an appropriate instrument to gauge the faculty perceptions of the changed educational environ ment, we had modified the DREEM questionnaire and used it to assess the perceptions of the faculty about the student expe riences in our school. This would indirectly allow us insights into the faculty perceptions of our curricular change. The find ings from this study will also help in ascertaining the weak nesses and strengths of our curriculum and in turn in improv ing our newly introduced curriculum. We had an acceptable response rate of 93%. We found that the mean total DREEM score of our faculty (139/200) was lower than that (144/200) previously reported [7]. The posi tive response observed by us is similar to studies which show teachers' views on integrated; problembased medical curri cula to be positive [8]. The positive response by FIC to the stu dent centered curriculum, which emphasizes long term learn ing and problem solving skills, are reflected in the higher mean scores in the domains Students' Perceptions of Learning and http://jeehp.org  Students' Academic Self-Perceptions. Low scores in domains Students' Social Self-Perceptions may not reveal perceptions of weaknesses, instead could be due to staff unfamiliarity with certain aspects of the students' experience since faculty were asked to select unsure when the statements dealt with aspects with which they were unfamiliar. The individual statement analyses identified the teachers' medical knowledge and competence as the strength of our college. The FIC perceive that significant endeavors have been made to encourage student participation in class, make teach ing stimulating, develop the students' competence, confidence and interpersonal skills; clarify learning objectives to students, encourage active and long term learning and encourage the students to ask questions they want. The relevance of learning to a career in medicine (No. 45) has also been perceived to be enhanced which may lead to enhanced motivation and reten tion of learning. The significant positive responses to the stu dentcenteredness of teaching and the development of prob lem solving skills in the students reassure the curriculum plan ners that the curricular change has been well discerned by the faculty. This is vitally important as faculty contributions and positive views of planned curricular changes lead to successful curricular renewals [9].
The highlighting of the teachers strengths such as well fo cused teaching, teaching time put to good use and teachers being well prepared for their classes are common to both the groups and this reflects the selfconfidence of the teachers. The FIC perceived positive changes with regard to construc tive criticism and teachers' behavior towards the students as significant. However, scores of statements teachers are authoritarian (No. 9); teachers get angry in class (No. 39), teaching is too teacher-centered (No. 48), and students irritate the teachers (No. 50) reveal FIC's perception of being unsuccessful in re linquishing their conventional roles. FIC also perceive that there is room for improvement with regard to feedback by the teachers and teachers giving clear examples. Though literature reviews have revealed that there are substantial differences be tween the students' perceptions about their lecturers when compared with the perceptions of the faculty about their col leagues [10], a previous study comparing faculty perceptions with those of the students showed similar perceptions of the teachers by both the groups [6].
The traditional medical curriculum has been deemed to be overloaded with information and previous studies and meta analytical reviews have shown PBL based curriculum associ ated with less factual knowledge [8]. However in our study, scores of statements associated with importance to factual learning, inability to memorize, appropriateness of the timeta ble of the school, stress of studying medicine and weariness of students may indicate that the teachers still perceive a curricu lum overload. Though significant endeavors have been made to prepare the students for the next level, there is still room for improvement as indicated by the scores being less than 3 for all above statements. Cheating has also been identified as an area requiring intervention. These common issues of concern have also been identified by studies conducted in other insti tutions [2,5,11] as the areas warranting remediation. The rest of the statements for which the faculty were probably unsure have garnered scores below three.
As the DREEM was not created to acquire other stakehold ers' opinions of how the students experience their educational climate, it was expected that the faculty would not be able to respond to certain statements. The respondents, therefore, were encouraged to mark these statements as unsure. Conse quently, the comparison of percentage agreement (faculty who agreed/strongly agreed), identified significant differences be tween FIC and FDC which reinforced the positive perceptions of FIC about the studentcentered curricular change. The fact that FIC tended to disagree that the teaching is too teacher-centered (No. 48) significantly more than FDC, has also support ed the previous findings of this study. We were unable to iden tify any major problem areas with the new curriculum based on the percentage analysis.
The faculty strength of College of Medicine is rather small and most faculty members are well informed about the meth ods used in the curriculum due to the continuous faculty de velopment programs. Faculty buy-in is essential for any suc cessful major curriculum reform effort [9]. While many of the http://jeehp.org J Educ Eval Health Prof 2014, 11: 7 • http://dx.doi.org/10.3352/jeehp.2014. 11.7 faculty teaching in the integrated curriculum were not involved in the developmental stages, they have been actively involved in the implementation of the curriculum. These may have contributed to most FIC teachers finding the experiences of the students in the new curriculum more favorable. Faculty perceptions reveal the need for undertaking certain interven tions to improve our curriculum. Substantial reduction of core curriculum, introduction of study guides, curriculum maps and providing conceptual frameworks for learning [12] may circumvent curriculum overload. An institutional culture of integrity can be established with clearcut institutional poli cies: introduction of medical ethics, formative selfassessment, and newer strategies in assessment like portfolio assessment. These may help in thwarting the problem of cheating encoun tered. A collegiate, cooperative staff student relationship will reduce the authoritarian leaning of the faculty [1]. Construc tive and effective feedback techniques should be reinforced through our faculty development programs [13].
We are aware that our sample size is too small to generalize the results of this study to other schools, and that a modifica tion of the instrument may have had an effect on its reliability. Moreover, the method we have used is indirect, subjective and dependent on teachers' recollection of students' experiences from the previous year. However, faculty members' perceptions are frequently dependent on these subjective factors and may influence the faculty' s resultant behavior. More structured qual itative analysis conducted in focus groups and using openend ed questions can supplement our findings [14] and deal with the limitations of this questionnaire based data [15].
In conclusion, our study provides preliminary data about the faculty perceptions' of the student experiences in the new curriculum in the College of Medicine. The modified DREEM questionnaire has helped in identifying the strengths and weak nesses of the new curriculum from the faculty's point of view. The faculty teaching in the integrated curriculum perceived the organ systembased integrated curriculum as providing a better educational environment for the students than the dis cipline based curriculum. However, authoritarian leaning of the faculty, lack of effective feedback, curriculum overload and cheating have been identified as areas of concern. Conse quently, a need to fine tune the new curriculum in these areas surfaces as a means of improving the new curriculum and the learning environment in our institution.