Comparison of Dundee Ready Educational Environment Measure with An abridged version at a Dental School

Background: educational environment refers to a group of factors that have an impact on students' performance. Measuring these factors help redesign curricula, instructional methods and impact teacher and student performance. The 50 item Dundee Ready Educational Environment Measure is a widely used tool for measuring the educational environment. An abridged version of DREEM was recently postulated with 12 items (DREEM-12) derived from DREEM. Aim: to measure the educational environment as perceived by third year Dental School PBL students at the Libyan International Medical University and comparing the scores of DREEM with the 12-item version. Methodology: DREEM questionnaire was administered anonymously through Goggle Forms to all third-year dental students at the Dental School of the Libyan International Medical University, Benghazi, Libya. Data for the 12-item version were extracted from responses to DREEM and the results were compared. Results: All third year students participated in the study n=59). 58 responses were eligible for analysis. Mean score for DREEM and its abridged version were 127.5±24.5 and 32.9±7 respectively. The correlation between the total scores of the two versions was highly signiﬁcant(r= 0.935, P <0.01). 83% of items of DREEM have mean scores ranging between two and three. Gender didn't aﬀect any of the items, domains or total scores of both versions. Conclusion: Both versions reached to the same conclusion i.e. the educational environment in the studied setting is perceived by students more positively than negatively, and that the correlation between the two questionnaires is highly signiﬁcant. However, most of the items need to be improved. Further studies are needed to determine whether the tested abridged version is performing comparably to the regular one in similar settings.


Introduction
Studying the educational environment (EE) is an important concept in the evaluation of the appropriateness of the situational context in which students learn and interact. The importance of measuring the EE stems from the information that can be gathered and used to identify potential areas for improvement. It has also been shown that academic performance is affected by the student's learning style, cognitive strategies used and regulation of learning, and that these factors are interdependent (Heikkilä and Lonka, 2006). The educational environment includes physical as well as non-physical components. These include, among many, the physical atmosphere, curriculum, methods of the instruction, instructors, methods of communication, methods of assessment, motivation factors, psychosocial factors and last but not least students.
The concept of evaluating the educational environment is so crucial that a number of EE evaluation tools were developed such as the Dundee Ready Educational Environment Measure (DREEM), the "Learning Environment Questionnaire" and the "Medical School Learning Environment Survey" (Roff et al.,1997;Soemantri, Herrera and Riquelme, 2010). Others produced tools that measure the educational environment in specific educational contexts such as the surgical operating theatre, anaesthetic theatre, practice-based general practice, and evidence-based medicine, osteopathy teaching program, diagnostic radiology and postgraduate studies (Cassar, 2004;Holt and Roff, 2004;Mulrooney, 2005;Bloomfield and Subramaniam, 2008;Malau-Aduli et al., 2019).
Dentistry is a highly competitive field since thousands of students apply for a limited number of seats. During study, dental students get challenged with a heavy-loaded curriculum, training on fine techniques, early contact with patients and continuous assessment. These factors, in addition to high expectations put formidable amount of stress on students (Till, 2004;Alzahem et al., 2014). DREEM is widely used by researchers in evaluating how health profession students perceive their educational environment (Roff, 2005).
Studies have been carried out using DREEM to help produce recommendations that can be used to make changes to curricula and assessment methods. These studies have also shown a positive impact of feedback given by teachers in response to such recommendations (Foster Page et al., 2012;Al-ansari and Tantawi, 2016).
An abridged version composed of 12 items (DREEM-12) covering the five domains of DREEM was reported showing that DREEM-12 explained 77.4% of the variance in DREEM scores with a correlation of 0.88 at p value of < 0.001 between the total scores of both instruments (Jeyashree, Shewade and Kathirvel, 2018).
The dental school at the Libyan International Medical University (LIMU) was established in 2007 and has graduated 7 batches up to date. Students belong to different sectors of the community and the school provides its services with a mission to graduate competent life-long learning graduates. The aim of this study was to assess how third year students at the dental school of LIMU perceive their educational environment using the DREEM instrument and compare it with DREEM-12. This aim was further divided to: Determine whether there are any gender differences in students' perceptions. 1.
Identify potential areas for improvement.

Setting
This study was carried out at the dental school at LIMU. The dental school offers a Bachelor in Dental Surgery after a 5 year-program followed by a 9 months internship. Students are instructed during the first two years of their study Altawaty A, Othman E, Alkuwafi R, El-kilani S MedEdPublish https://doi.org/10.15694/mep.2020.000234.1 Page | 3 using problem-based learning (PBL) in addition to practical teaching, skill lab sessions, seminars, tutorial and a limited number of interactive lectures. Once they move to the third year, PBL is replaced with interactive lecturing, seminars, tutorials, appraisals, community visits, clinical teaching etc. So the perception of students in this study reflects their first two year PBL-driven studies.

Ethical issues
Approval was granted by the Ethical Review Committee at LIMU. Students were totally free to participate, abstain or withdraw from participation. The questionnaire was anonymous. None of the authors is aware of the identity of the participating students.

Participants
All third-year dental students of the academic year 2018-2019 were eligible to participate. Students were invited to voluntarily participate in answering the DREEM questionnaire. The questionnaire was made available using Google Forms and delivered to students in the computer lab during a predesigned session.

DREEM
DREEM questionnaire is composed of 50 items covering five domains. These are students' perception of learning (SPL), students' perception of teachers (SPT), Students' academic self-perception (ASP), students' perception of atmosphere (SPA), and students' social self-perception (SSSP). Items are graded using a 5-point Likert's scale: 0 for strongly disagree, 1 for disagree, 2 for unsure, 3 for agree and 4 for strongly agree. Nine of the items were negatively worded and so reversely scaled before data analysis.
DREEM-12 is composed of 12 items covering the same five domains of DREEM. These 12 items are extracted from DREEM items (Jeyashree, Shewade and Kathirvel, 2018). Therefore, data for DREEM-12 are extracted from students' answers to DREEM.
In this study, the only demographic data added to this questionnaire were the gender and age.

Data analysis
Data were analyzed using SPSS version 23 (IBM SPSS Statistics for Windows, Version 23.0. Armonk, NY: IBM Corp., USA). All of the 59 students anonymously and voluntarily participated in filling the questionnaire. All filled questionnaires, but one, were eligible for analysis. One response was excluded from analysis because the response form was almost blank. Therefore, only 58 were included in the analysis. Missing values were replaced by medians when their number is two or less for each participant. Data for DREEM-12 was extracted from the data of DREEM. Scores and descriptive statistics were calculated for each item and for each of the five subscales for both DREEM and DREEM-12. Descriptive statistics included numbers, means and standard deviations. Cronbach's alpha was calculated for the two versions of DREEM and for their domains without and with item deletion. Correlations between items, totals and subscales within each instrument and between the two instruments were calculated using Pearson's coefficient. The impact of gender of the scores was estimated using one-way Anova.

Interpretation
Interpretations for DREEM and DREEM-12 versions of the questionnaire are shown in Tables 1 & 2 respectively.

Results/Analysis
All third-year dental students participated in filling the questionnaire giving a response rate of 100%. Thirty-one (53.4%) of participants were males. Mean age in years was 20.7±1.27.Crohnbach's alpha for DREEM was 0.93 while that of DREEM-12 was 0.835. Reliability of DREEM didn't change much by deleting any of its items (see Table 3).The mean DREEM total score was 127.5±24.5 while that of DREEM-12 was 32.9±7. The correlation between the two scores was 0.935 at a level of 0.001. Gender didn't affect the scores of any of items, subscales or totals for both versions. Tables 3 and 4 show means, standard deviations, standard errors, 95% confidence intervals and Crohnbach's Alpha values for the subscales of DREEM and DREEM-12 as well as the effects of item-deletions on the scores of DREEM. Of notice is the low reliability coefficient of the social subscale of DREEM-12.   All items of DREEM scored a value of less than 3 except two items which scored 3.08 and 3.06. These are "The teachers are knowledgeable" and "I am confident about passing this year" respectively. Four items scored less than two. These are "Cheating is a problem in this course", "I seldom feel lonely", "I find the experience disappointing" and "The teachers get angry in the class". Table 5 shows that the correlations between the total score of both versions of DREEM and their subscales are significant at a p value of 0.01. Table 6 shows the correlation coefficients between totals and subscales of both versions of DREEM. All correlations were significant at 0.01 level. All items of DREEM-12 scored a mean of less than two except "The teachers are knowledgeable" which scored 3.08.  Most of the items in both versions scored means between two and three. The item on teachers' knowledge was an exception and scored 3.08 while item "I seldom feel lonely" scored 1.94 (Tables 7a-7e and Table 8).

Discussion
This study explores how third year dental students at the Libyan International Medical University perceive their educational environment and compares the perception scores of a short 12-item version of DREEM with the original 50-item one. Problem-based learning is the educational strategy adopted during the first two years of study in the dental school at the Libyan International Medical University. The students in this study will move to their third year of study where the adopted educational strategy is not problem-based. The aim of this study was to assess the students' perception of the educational environment during the first two years of their PBL-driven study with an intention to have a second look at the same cohort once they finish two more years of learning through non-PBL driven instruction during their third and fourth years of study. So, this study reports the first part of our planned investigation.
The overall score of DREEM was 127.5±24.5 which indicates that students perceive their EE more positively than negatively. For the score to be higher, more work is needed to improve less-performing areas in the educational environment. This result also denotes that there are areas which need focus on, particularly the student's social selfperception (SSSP) which read as "Not too bad". Comparing this score with that of some other studies shows similar results (Table 9) (Zawawi and Elzubeir 2012;Hamid et al. 2013;Doshi et al. 2014;Jnaneswar et al. 2016;Khan et al. 2016;Jamali, Saberi and Taheri, 2019;Stormon, Ford and Eley, 2019).  Khan et al. (2016), studied the educational environment across four dental schools in Islamabad, Pakistan and reported mean total scores ranging from 109.7±24.2 to 125.4±18.6 with an overall mean for the four schools of 120±18.9.Other studies reported much lower figures even though they are still within the third level of the interpretation ladder (Khan et al., 2016). A review of 106 studies reporting DREEM total score showed that 80.6% of them read a score between 101-150 (Chan et al., 2016). We conclude from the above that the score reported by the present study lies within the usually reported range. Only few studies addressed how these results could be used to improve the learning environment. Most just mention the importance of making a change without specifying what actual steps to be taken. Prospective studies are needed to assess the impact of any interventional actions directed towards making the desired change. A report from Iran exemplifies such kind of interventions which included professional development programs for the faculty accompanied by monitoring the change in the educational environment perception (Bakhshialiabad et al., 2019). The mean score of DREEM-12 was 34.7/48 (67.5%) while that of DREEM was 127.5/200 (63.75 %) indicating almost no difference. This marked similarity indicates that using the twelve-item version of DREEM likely gives comparable results as far as the current study is concerned.
Altawaty A, Othman E, Alkuwafi R, El-kilani S MedEdPublish https://doi.org/10.15694/mep.2020.000234.1 Page | 11 Males and females are known to have different learning styles and so we expect to find differences in their perception of the EE (Philpin et al., 1995;Katz, 2017). In the current study, gender had no effect on the total or subscale scores of both versions of the questionnaire which is in contrast to what has been reported by studies carried out in Australia, Iran, Malaysia India, Pakistan and Saudi Arabia (Brown, Williams and Lynch, 2011;Rahman, et al., 2015;Jnaneswar, et al., 2016;Rehman, et al., 2017;Al-Saleh, et al., 2018). In all these studies females outnumbered males while in the current study males marginally outnumbered females.
The reliability coefficient of DREEM was 0.838 indicating a high level of consistency between items. It is generally considered that a reliability coefficient (alpha value) of 0.65-0.80 indicates that the scale is adequate (Vaske, Carley and Jay, 2017). However, Alpha coefficient is known to be affected by the number of items in the questionnaire. Though the required number of items is debatable, the larger the number, the higher the consistency even though the relationship between the alpha coefficient and the number of the items is non-linear (Komorita and Graham, 1965). One of the problems of alpha coefficient is that it only tests the overall consistency of the questionnaire without due concern of the relative loads of the items. The reliability coefficient of the subscales in this study ranged from 0.396 to 0.838 with the Student's social self-perception scoring the lowest. The student's social perception alpha coefficient could be increased to 0.416 if "I am too tired to enjoy this course" and to 0.427 if "My social life is good" and to 0.45 if "If seldom feel lonely" were deleted. In spite of that, the social self-perception score was above 60% of its possible maximum i.e. 17.4 out of 28. The four other subscales i.e. Student's perceptions of learning and Student's academic self-perceptions reliability coefficients were not affected by item-deletion while Student's perceptions of teachers and Student's perceptions of atmosphere were only marginally increased by item deletion (Table 3). Factors that enhance students' perception of their social life includes involving students into group-based learning tasks and extracurricular activities. The dental students in this study have learned their first two years through problem-based learning which is expected to enhance socializing while learning in groups. Measures to make sure that students learn and interact in and outside classrooms might help improving the students' social perception. Enhancing student support systems and providing individual-tailored feedback is likely to help both the social, learning, academic, atmosphere and teacher perceptions. Feedback in particular is highly valued by students (Whittle, Whelan and Murdoch-Eaton, 2007).
All subscales of DREEM scored above 60% of their maximum with a range from 60-67. This represents achieving the third level of each subscale indicating a positive trend in students' perceptions across all five domains. However, this also shows that there are areas for improvement. These areas are identified by having a look at subscale and individual items' scores. Two items scored ≥ 3 on a scale of 4. These are "The teachers are knowledgeable" and "I am confident about passing this year". This satisfaction with teachers' knowledge can occur even in traditional teaching because it assesses student perception of the teacher's knowledge which can be expressed more powerfully in teacher-centered teaching. Therefore, the interpretation of individual items of the questionnaire should be done with an eye on the relevant subscale to which the item belongs. Students in this study responded almost equally to two contradictory items; "The teaching is student-centred" and "The teaching is too teacher-centred" with mean scores of 2.6 and 2.36 respectively. This neutrality in the response to these two opposing statements calls for helping students to get orientated to the importance of learning in a student-centered manner. This might be achieved through orientation courses on the advantages of student-dependent learning strategies. Teaching students through student-centered approaches doesn't guarantee that they will largely learn more independently. For such learning to happen, students need to believe in student-centered learning strategies to get the most of it.
Seven items covering the social domain, of these two items scored lowest scores. These are "I seldom feel lonely" and "I am rarely bored during this course" with scores of 1.94 ±1.30 and 2.05±1.08 respectively. This result calls for more work on enhancing interpersonal relationships and making educational environment more attractive. Establishing and sustaining learning environments that nurture healthy and humanistic interpersonal relationships is Altawaty A, Othman E, Alkuwafi R, El-kilani S MedEdPublish https://doi.org/10.15694/mep.2020.000234.1 Page | 12 important in motivating students to learn and sense a feeling of belonging to the institution, peers and teachers. It has been shown that adolescents with positive relationships with peers are more motivated and gain higher achievement levels (Furrer and Skinner 2003;Martin, 2013). The social relations at the institution can take a variety of forms. These include dominance, influence, friendliness, hostility etc. Poor interpersonal relationships are detrimental to students and teachers alike, because they adversely affect the academic and non-academic development of students (Martin, 2013). Regarding students' perception of teachers, the item "the teachers get angry in the class" achieved a very low score (1.55±1.06). This score points to the need for training instructors on how to deal with misbehaving students in the class but also calls for the school to produce instructor selection criteria that guarantee hiring instructors with sufficient interpersonal tolerance. Teaching communication skills to instructors through simulation workshops could help. Two items under the perception of atmosphere achieved low scores "cheating is a problem in this course" and "I find the experience disappointing" with mean scores of 1.93±1.22 and 1.81±1.19 respectively. Even though the total score of this subscale was 60% of its maximum possible value, the very low scores of these two items need immediate attention. Cheating is a problem in education in general and it is a complex one. An article on this issue showed that adherence to "Neoliberal" values that enhances focusing on self enhancement and competitiveness provides the seeds for cheating (Pulfrey and Butera, 2013). Since these values are based in the society in general; management of cheating needs a wider approach due to the close link between societal and individual values. Students strive to achieve high marks in order to win and be recognized by the society as successful persons. This would naturally lead to a competitive rather than collaborative attitude between students and to cheating. Changes in the assessment methods at the institutional level might help in mitigating this trend. The mean score of the item "I find the experience disappointing" is disappointing. Unfortunately, DREEM doesn't explain responses to its items. Some researchers followed DREEM results with interviews and focus groups in order to know the explanations behind students' responses (Whittle, Whelan and Murdoch-Eaton, 2007). Rest of the items in DREEM scored means between 2 and 3 which indicates that students are somewhat neutral about them indicating the need for improvement. All items but six have strong correlation with the total score at a p value of 0.01. The items with insignificant correlation with the total score were "The teachers ridicule the students" "The teachers get angry in the class", "This school is well timetabled", "I find the experience disappointing"," I am too tired to enjoy this course", "I seldom feel lonely". Four of these are negatively stated items which might have a relationship to the lower scores of students' perceptions to them. The language of instruction at LIMU is English even though the primary language of most students is Arabic. This bilingualism might have an effect on the students' interpretation of the items especially the negative ones. Arabic versions of DREEM were used in some studies (Al Ayed and Sheik, 2008). We are not sure whether the bilingualism of our cohort has something to do with the poor mean scores of the negative statements.
The mean score of DREEM-12 is 32.4± 7.0 which is 67.5% of the maximum indicating a more positive than negative trend. This is quite comparable to that of DREEM. The percentage of mean scores of the subscales of the abridged version ranged from 66.2%-67.5% which is also quite comparable to the regular version. The reliability coefficients of DREEM-12 was 0.835, much similar to DREEM and that for the subscales were reasonably good except for the social subscale which has a value of 0.08 indicating poor internal consistency. The subscales of this version had a highly significant correlation with the total scores at a p value of 0.01 as shown in (Table 5). The total and subscales' scores of both versions have strong correlation at a p value of 0.01.
Limitations of this study include the relatively small number of participants and possibly the use of the English version of DREEM to students whose primary language is non-English.