Evaluation of Competency Based Medical Education Curriculum*

There is an increasing interest in competency based medical education (CBME) due to the developments and changes in medical education and many medical schools have begun to apply it recently. Although its popularity in both practice and theoretical studies, there is a lack of studies focusing on evaluation of CBME curriculum. In that study, it is aimed to evaluate competency based medical education curriculum. This qualitative study is designed as a case study. It is also a curriculum evaluation study carried out systematically via Stufflebeam’s CIPP model. The study group consisted of fifteen faculty members and fifty students of a medical school in Turkey. The data was collected via semi-structured interview forms and analysed applying content analysis. Some of the important results reached in that study are the followings: The students have had mostly negative ideas about the competencies and the teaching-learning process while the faculty members have been satisfied with their practices. Both the students and the faculty members thought the CBME contributed to students’ academic and vocational developments. It is concluded that in order for CBME to provide the opportunities in medical education context, it needs to be developed, implemented and then evaluated in the frame of theoretical backgrounds and principals. Otherwise, because of the problems/deficiency in the development and implementation process, the efforts might end in vain. In the light of the results, it can be suggested that the faculty members should be supported on how to determine/write the competencies and the content; on effective teaching/learning methods. Also, CBME should be evaluated and revised regularly involving the faculty members and the students in the process.


INTRODUCTION
Higher education is placing a growing emphasis on affordability and accountability (Burnette, 2016;Fiddler, Marienau, & Whitaker, 2006). Many stakeholders, including academic accreditation organizations, employers, and academic institutions, desire to find a way to define and assess students' attainment of educational outcomes and competencies related to their academic programs (Mintz, 2015). Due to these attempts, competency-based education (CBE) has become widespread in recent years, even though it had begun in adult-focused degree programs during the 1970s (Burnette, 2016). Interest in CBE has increased in several fields of higher education, especially in health professions education. Some think that CBE is an emerging discourse in health professions education while others believe that competency-based medical education (CBME) has gained renewed interest among health educators and policymakers in recent years because of their increased focus on outcomes related to patients, populations, and health professions education programs Frank, Snell, et al., 2010). CBME is believed to have become popular due to the need to reduce unacceptable variability in graduates' skills after finishing medical training (Raymond, Kerschner, Hueston, & Maurana, 2015). With the introduction of Tomorrow's Doctors in the UK, medical education began the transition from a time-and process-based system to a competency-based training system. The international acceptance of this paradigm shift is shown by the releases of the CanMEDS framework, The Scottish Doctor (Simpson et al., 2002), the ACGME Outcomes Project (Swing, 2009), Good Medical Practice (General Medical Council, 2006, the Australian Curriculum Framework for Junior Doctors (Graham et al., 2007), the 2009 Framework for Undergraduate Medical Education in the Netherlands (Van Herwaarden, Laan, & Leunissen, 2009) and the National Core Curriculum (NCC) Framework in Turkey. CBME was defined by the International CBME Collaborators as [a]n outcomes-based approach to the design, implementation, assessment, and evaluation of medical curricula, using an organizing framework of competencies . It is also an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. It deemphasizes time-based training and promises greater accountability, flexibility, and learnercenteredness . In CBME, untethered from course material and credit hour, learners demonstrate -clearly defined and measurable-(Klein-Collins, 2012) competencies particularly at mastery level and their own pace.
Despite the growing interest in the CBME, how to plan and implement CBME has remained a big question to reach the supposed opportunities and dismiss the drawbacks: Frank, Snell et al., (2010) summarized the steps in developing CBME, namely identify the abilities needed for graduates, explicitly define the required competencies and their components, define milestones along a development path for the competencies, select educational activities, experiences, and instructional methods, select assessment tools to measure progress along the milestones, and design an outcomebased evaluation of the program. CBME curricula developed via that process can reflect a spectrum in terms of structure and time flexibility (Frank, Snell et al., 2010), which can be interconnected with different approaches of CBE (Book, 2014) like course/credit, direct assessment and hybrid approaches. Although The Council of Regional Accrediting Commissions (2015) identified these three approaches, the Carnegie Foundation, in an effort to re-examine the use of the credit hour, acknowledged that competency-based approaches occur in various contexts, and when comparing different models there are "huge variations" (Silva, White, & Toch, 2015).
The related literature mainly focuses on different subjects apart from curriculum evaluation. Nevertheless with the increasing number of medical schools applying CBME, it gets more important to determine effective practices of CBME and to share best practices, which makes curriculum evaluation a must. What's more, as a field of education, medical education is needed to be revised and updated in accordance with scientific, technological, and social developments, which makes it relevant to evaluate curricula. Curriculum evaluation is defined as the process of delineating, obtaining, and providing useful information for judging decision alternatives (Stufflebeam, 2005). According to various accreditation councils around the world (Accreditation Council for Graduate Medical Education, Liaison Committee for Graduate Medical Education, Association for Evaluation and Accreditation of Medical Education Programs, World Federation for Medical Education), it is a fundamental responsibility of medical schools to make comprehensive, multifaceted, model-based, data-driven curriculum evaluation studies. In order to establish that responsibility properly and systematically, medical schools have to evaluate their curricula in accordance with one or more evaluation models. Among plenty of curriculum evaluation models, the use of the Context, Input, Process and Product (CIPP) evaluation model has been thoroughly recognized in a variety of educational and non-educational evaluation settings (Tokmak, Baturay, & Fadde, 2013;Zhang, Zeller, & Griffth, 2011). Additionally, several studies that applied this model to evaluate curriculum in the context of health professions have attracted attention in the literature in recent years (Singh, 2004;Steinert et al., 2005). However, to date, no comprehensive study has used the CIPP evaluation model to facilitate the evaluation of CBME in an undergraduate medical education curriculum so far.
The evaluation of CBME guides decision-makers on university/national/international level to determine what kind of curriculum to improve in order to implement CBME more efficiently. On the university level, the curriculum development units at the medical schools should benefit from comprehensive results of curriculum evaluation studies because curriculum evaluation must be used as a first step in order to develop a new curriculum. And it also carries importance for the curriculum evaluation unit at the faculties illustrating how to conduct an evidence based curriculum evaluation based on the theoretical foundations and appropriate evaluation models. On national level (namely in Turkey), adoption of CBME properly is likely to be slow and incremental due to the regulatory environment, so research showing what does and does not work is important as competency-based curricula continue to expand at universities. This evaluation of the CBME curriculum could not only provide insight into the effectiveness of a CBME in the evaluated context, but it might contribute to the growing field of knowledge that could help move the current regulatory environment in other national/international contexts. Additionally, leaders at medical education can use of evaluation results to determine whether CBME can offer opportunities to some/all students to improve themselves academically/vocationally. Finally, the evaluation could also serve an international role in contributing to the research on competency-based education. Even though CBME have been in practice in certain areas, few studies are available that provide evidence about their effectiveness (Barman, Silèn, & Bolander Laksov, 2014).

Aim of the study
In that study, it is aimed to evaluate the competency-based medical education curriculum. To the end, the following questions were answered: 1. What are the views of the faculty members and the students about the CBME curriculum? 2. What are the views of the faculty members and the students about the implications of the CBME curriculum?
3. What are the opinions of the faculty members and the students about the effects of CBME in terms of academic/vocational developments of students?

Research Design
The study was designed as a qualitative case study, because case studies are one of the most appropriate qualitative research designs that can be used to understand complex phenomena (Yin, 2003). To understand the CBME, as accepted as a complex phenomenon, and to evaluate it, the CIPP model was used. For that reason, that study is also a curriculum evaluation study carried out in the Input, Process, and Product evaluation in the frame of CIPP model.

The Setting
In Turkey, the National Core Curriculum (NCC) was developed in 2001-2002 and revised in 2014 to identify standards for medical degrees, stating what a graduate is supposed to know, able to do, and competent in, within the context of local needs and realities (Bulut, 2013). However, in 2014, there made some changes and instead of content and learning objectives, NCC was grounded in competencies and tasks (Gülpınar et al., 2014). In the revised version, it is advised that pre-graduate medical education should be developed and implemented within the framework of an educational approach based on educational outcomes. In this framework, the competencies of the medical school graduates are determined and the whole education process is to be carried out in accordance with the determined competences framework. Although the NCC is an advisory document, many medical schools in Turkey have revised their curricula in that framework. One of the universities conducting CBME is University A (The university is coded like that.). That state university, situated in a small city in the middle of Turkey, has a newly founded medical school. This study was conducted in that medical school at University A. Since its beginning to accept students in 2016, the medical school has been conducting CBME. The founder dean of the medical school advocates the CBME uttering "The science of medicine, with its objective structure and practical feature, makes it a must to determine competences and measure them not only on knowledge level but also high level skills." Table 1 explains the details of the CBME implemented in the medical school of University A to provide a context for this study. As Table 1 shows, there are inconsistencies between theory and practice in terms of the path of learning, typical assessment tool, timing of assessment and program completion, which can be regarded as a limitation for that study.

Table 1 A Comparison of CBME in Theory and Its Practice in the Medical School of University A
Variables CBME (theory)

CBME (practice at University A) Driving force for curriculum
Competency-knowledge acquisition All of the faculty members have to determine competencies and share them before the courses so that students can study on that related theoretical background and knowledge acquisition related to competencies can be accomplished. Driving force for process

Learner
All of the preparation and application are done based on learners' needs and expectations. The students are always welcomed to share their ideas in formal and informal ways. There are also section representative students to meet the faculty members regularly. Their ideas are evaluated and reflected in the following educational/other regulations. Path of learning Non-hierarchical The path of learning is organized based on learning outcomes in a hierarchical way.

Student and teacher
The faculty members are mainly responsible for content selection/order, but the students are expected to have a look at the content uploaded onto their system by the faculty and share their views on the content.

Goal of educational encounter
Knowledge application The main goal is to train qualified future doctors to apply what they learn at the faculty. Typical assessment tool Multiple objective measures /evaluation portfolio Students' learning levels are assessed mainly via multiple choice/truefalse and fill in the gaps questions in the module exams. On the one hand, they are assessed via open-ended questions in the committee exams. Assessment tool Authentic (mimics real tasks of profession) In the second half of the six-year medical education, students will be assessed via real tasks of the profession.

Setting for evaluation
In the trenches (direct observation) Especially in the last year of their education, students will be assessed via direct observation by the intern responsible faculty team. Evaluation Criterion-referenced Students need to have many exams during each year, the evaluation criteria was determined onwards. Timing of assessment Emphasis on formative Students are examined regularly and small intervals via module and committee exams. These are mostly summative exams. Program completion

Variable time
Program completion time is fixed for all the students.
Note: The table (based on Carraccio et al, 2002) was revised and adjusted by the researcher.

Study Group
The study group included fifty students and fifteen faculty members. The students at the medical school determined via convenience sampling method included in that study. Thus, 30 female and 20 male students were interviewed, which resulted in a demographic spread of 60% female and 40% male, because of the high number of the female in the total students. Furthermore, fifteen faculty members were identified by the researcher for the interview by maximum variation sampling method among purposive sampling methods in terms of their expertise area. The aim of choosing this sampling was to form a small working group that would provide maximum participant variety; to examine if there are common cases among a variety of conditions instead of making generalizations (Yıldırım & Şimşek, 2013). In this regard the aim was to reach educators from many different departments and responsible for various courses. Five professors and ten assistant professors were selected from twelve different departments of the medical school. This selection resulted in a spread of three female and twelve male with experiences in CBME between two and three years.

Data Collection and Analysis
In the study, interview method was used for data collection. As the data collection tool, two semi-structured interview forms for the faculty members and the students developed by the researcher, were used. Each form has parallel questions related to the CBME curriculum and based on the framework of CIPP. The forms include two parts. In the first parts of each form, there are questions about their demographic information. In the second part of the form for the faculty members, there are six questions, and there are five ones in the other one. All of the questions are in line with CIPP's input, process, product evaluation.
The interviews were made face to face by the researcher at the end of the 2018-2019 academic year. Because all of the interviewees were volunteer to involve the study, they permitted recording the interviews. Each interview of the faculty members lasted among 45-60 min. and the students were interviewed individually in 20-30 min. or group of two-five in nearly an hour. Then, the researcher transcribed the whole responses. The data was analysed using a conventional content analysis method (Fraenkel & Wallen, 2000). Firstly, the data were coded, then grouped into emerging sub/themes. Finally, the themes obtained are discussed in the light of relevant researches in the literature.
For the credibility and transferability of the semi-structured interview forms, views about the forms were obtained from three different experts -two of them are in the field of educational sciences and one of them is in the field of medical education. The CBME at the medical school was described in detail, which helps the credibility of the study. In order to get variety in terms of data sources, opinions/suggestions of two main stakeholders of the evaluated curriculum, namely the students and the faculty members-were found out which increases the trustworthiness level. For this study, data collection and analysis processes were given in detail and direct quotations were used while analysing the obtained data. In order to ensure confirmability, 20 % of the data was firstly coded by two experts separately-one is the researcher and the other is a professor at the department of curriculum and instruction, who voluntarily helped the researcher as an expert. Then, in a meeting with the focus of inter-coder reliability, it was determined that the variation of codes, subthemes and themes determined by the researcher and the professor was minor and consensus was reached. After the first coding process was over, the rest of the data was coded by the researcher alone but making multiple checks. Additionally, all data was stored in order to maintain confirmability.

Role of Researcher
The interviewer was an instructor with PhD degree in the field of curriculum and instruction during the interviews. The researcher has also expertise in curriculum evaluation and qualitative analysis by participating in trainings, courses and various researches on graduate levels. She was also a member of curriculum evaluation unit of that medical school while the study was being conducted.

A. Input Evaluation
For input evaluation of the CBME curriculum, both the students' and the faculty members' views were explained below.

Faculty Members' Different Definitions of CBME
The definitions of CBME were varied due to the faculty members' views about CBME. Their views were grouped under two themes namely content and outcomes. In other words, the interviewed faculty members generally focused on the content and the outcomes in their definitions.
For some of the faculty members, CBME is "to determine what knowledge students will learn at the end of the course" (f:8). One of them clarified his definition uttering "Medical education is such a huge area that it includes a vast of education. Students drown in knowledge, they do not know what they should know. ... CBME is an education in which students are presented only what they should know at the end of their education." (F15). These definitions show the focus/emphasis on the course/programme. For some of the faculty members, CBME is "an outcome-based approach" (f: 7). One of them clarified "CBME is a system where firstly the outcomes are determined, then the outcomes direct the course plan, exam questions… " (F10). One another explained "CBME is a system that enables students to get ready for the lesson by learning what they are expected to do if they check out the predetermined outcomes." (F13) On the other hand, as Talbot (2004) claimed while defining CBME, it can be criticized as "a negative oversimplification of physician competence". However, one of the interviewed faculty defines CBME as simplifying physician competence, not regarding the situation as negative or over simplification. He indicates "We aim to train medical doctors and we want to present the students what they need to have as a medical doctor. The faculty members may desire to give extended information about his/her expertise. With the help of CBME, we limit the content to what they need to know, to the basic medical knowledge level… " (F14).

How to Determine the Competencies in the Frame of CBME
The competency determination process has shown some differences besides some common steps. Table 2 shows the process and different versions: Examining the content of the course ----------F3 As seen in Table 2, the practices in the competency determination process have eight different variations. On the other hand, the practices could be grouped into two themes determining scope of the competencies and evaluation of their appropriateness. But the sub-steps are changeable. It can be said that the faculty members used different sources while determining scope of the competencies. But the number of sources is low and they are restricted with some kind of written texts like NCC and textbooks. On the one hand, in the evaluation and revision of the competencies, only two faculty members (F2, F6) conducted some practices to evaluate and revise competencies analysing student success and its relevance with the content.

Students' Views and Suggestions about the Competencies Shaping the Medical Education
When students were asked about their views about the competencies shaping their medical education, they generally expressed their negative views/drawbacks of the competencies and made some suggestions. For some of the interviewed students, "the competencies are not clear enough" (f:4) and most of them explained their views repeating the words "not clear" (S12, S27). For some of the interviewed students, "the competencies are too broad" (f:7). One of them explained "Competencies are written so broadly that we get responsible for the whole of the content of the course for the exam" (S22). One of them claimed "Some faculty members write the title of the content as if they were the competencies (S23). One of them thought "Sometimes the competencies are too general that I think they are written for the sake of duty (S28)". Finally, some of the students claimed "competencies are clues for exam questions" (f:9). One of them expressed his happiness saying "I am really happy to learn the competencies" before the exams (S40). The students also suggested that the competencies should be more clear (f:6) and limited/specific (f:8).

B. Process Evaluation
For process evaluation of the CBME, both the students' views were illustrated in Table 3 and they were explained in accordance with the faculty members' views.

Students and the Faculty Members' Views about the Dimensions of CBME Curriculum
Students' views about the dimensions of CBME curriculum could be grouped into three main and five sub-themes as seen in Table 3. For each theme, they indicated some positive and negative views about the CBME. When asked to the interviewed faculty members about the content dimension, they frequently explained they select and organize the content based on the most popular textbooks in the related areas (f:8). They did not mention any problems about the content and added their satisfaction (f:13) but the students stated overwhelmingly negative views about the content and they claimed the content is heavy, not clear and including difficult subjects. The students also give some clues about their expectations in terms of the content dimension of CBME curriculum. For them, the content should be clear/understandable (f:28), not including irrelevant information (f:34), and in proper order (f:8). Some of the views stated by students are as follows: "Sometimes the content is not clear, which confuses us." (S9) "Some content presented us is not adequate for us to learn the subject. We hear most of the subjects for the first time. Some needs more clarification." (S10) "As natural -expected, the content is too heavy and difficult." (S12) In terms of teaching/learning experiences dimension of CBME curriculum, the students explained their views under two themes namely the teaching/learning process and the materials. They mostly explained their negative views instead of positive ones. To discover the teaching/learning process when asked to the interviewed faculty members, it was found out that they applied mostly deductive method (f:15), question-answer technique (f:9), and rarely case studies (f:3). It was also reached out the faculty members were generally satisfied with the teaching-learning process of the CBME. On the other hand, as seen in Table 3, the students overwhelmingly indicated negative views about the teaching/learning process (f:74) rather than positive ones (f:11). As understood from the statements of the students cited below, they also made some suggestions about the teaching/learning process. They think in the courses students should be active, different methods/techniques should be applied, the courses should be interacting and practical not only theoretical.
"Some instructors force us to learn deeply, but in some courses we cannot understand the issue because of heavy content and improper teaching methods." (S13) "Subjects are explained immediately and superficially, not in the way that students can understand" (S38) "For students to be engaged actively in courses, the instructors should be more enthusiastic and know how to make presentations efficiently/lively "(S44) When the interviewed faculty members talked about the teaching-learning process, nearly all of them explained they used PPT presentations (f:13) and included some related videos (f:6). In the same way, the students mostly criticized that the materials used in the courses are restricted to PPT prepared by the instructor and they added that the materials are not adequate. That finding shows that students expect to have different kinds of materials.
When it comes to the assessment dimension of CBME curriculum, the faculty members explained they prepared so many questions to upload to the assessment system (f:12) then they choose the most appropriate one for the related competency (f:6). The questions are generally multiple-choice (f:10) and/or true-false questions (f:10). On the one hand, students generally explained their satisfaction about the assessment system describing it especially as good and not bad. However, the students explicated their dissatisfaction about the exams as being difficult, only theoretical and detailed. Some of the students' views are as in the followings: "During the academic year, we have 12 exams. The assessment system is fair in terms of determining what we learn, which questions we can answer truly but we are getting more and more stressful because of frequent exams." (S14) "The exams are so frequent, but that frequency is good." (S33)

C. Product Evaluation
For product evaluation of the CBME, both the students' and the faculty members' views were explained below.

The Students' and the Faculty Members' Views about the Contributions of CBME to Students' Vocational/Academic Developments
The students and the faculty members thought the CBME contributed to students' vocational and academic developments. Their views can be grouped into three as CBME totally (f students : 19f faculty : 9), partly (f students : 11-f faculty : 4), and never (f students : 10-f faculty : 2) contributes to their vocational developments. Both the students and faculty members generally think that the CBME totally ensures their vocational development. Some of the students' views are as in the followings: "I think that our both academic and vocational success will be high in the future as the students are directed to the important things and there is no unnecessary information in our minds." (S16) "In this intense tempo, there are places where I go by memorizing without learning. So I think there will be a lot of things that I can't be successful with when I am doctor in the future." (S43).
We don't have enough practical lessons, which means less experience in the future (S14).
In the same way as vocational development, the students' and the faculty members' views about contributions of CBME to students' academic developments can be grouped into three as CBME totally (f students : 24-f faculty : 10), partly (f students : 11-f faculty : 4), and never (f students : 7-f faculty :1) contributes to their academic developments. Both the students and faculty members generally think that the CBME totally ensures their academic development although some students rarely indicated doubts about her academic development claiming that "the CBME is useful only for passing the course." (S28). But they most frequrentşy indicated hopes for their academic future as in the followings: "It restricts what we need to learn. Thus, we do not have unnecessary information and our academic success increases." (S29)

RESULTS AND DISCUSSION
In this study, the competency based medical education curriculum was evaluated within the scope of input, process, and product evaluation of Stufflebeam's CIPP model. In the frame of input evaluation, views about the CBME itself and the competencies were determined. The way that the faculty members defined the CBME is important because definitions give clues about their points of view and even their implications of CBME. In the literature, there are various definitions of CBME, emphasizing keywords like "outcomes defined, curriculum of competencies, demonstrable, assessment, learner-centred and societal needs" . On the other hand, the interviewed faculty members made various definitions generally focusing on the content and the outcomes of the CBME. Some of the definitions made by them can be interconnected with the definitions in the literature which asserted CBME as an example of an outcomes-based approach to the curricular design (Harden et al. 1999a;Harden et al. 1999b;Glasgow et al. 2006). It is also noteworthy that the faculty members did not make any connection with the competency and students or/and societal needs. However, the competencies should also meet those ever-changing needs (Boucher et al., 2017).
In CBME, competencies required for practice form the central component of all curricula (Boucher et al., 2017). The desired competencies of medical education are firstly identified, and then the educational experiences and assessment strategies are determined in CBME. For that reason, the competency determination process gets greater importance to reach the success at the end of the education process. The competencies must be derived from an assessment of societal needs for healthcare (Boucher et al., 2017). But for the evaluated CBME in that study, it was concluded that most of the faculty members are taken granted for the NCC and they made some minor changes to them. They think those changes are enough for adjusting the NCC to update their CBME only by adding some more specific competencies based on their own experiences and students' observations. For the Turkish medical education system, it is a must to be in parallel with NCC; on the other hand, in the frame of CBME, it is expected to determine the competencies focusing on both local healthcare needs, students' academic backgrounds/profiles and institutional priorities along with the universal health education standards.
The students generally explained their negative views/drawbacks of the competencies shaping their medical education and they made some suggestions. When the related literature is examined, it can be found out that students' expectations are the features that the competencies of CBME ought to have. It can be concluded that the competencies need to be revised in terms of both the students' expectations and the related literature.
In the frame of process evaluation, the views of the faculty members and the students about the implications of the CBME were analysed. When students' and faculty members' views about CBME practices were found out, it was determined that the faculty members frequently explained their positive views and pleasure with the CBME practices, on the other hand the students explained their negative views about the CBME especially in terms of the teaching/learning process. As understood from the Table 3 and their suggestions, the students expect the teaching-learning process to be more interactive, student-cantered, practical, clear and so efficient. These expectations are similar to the findings of other studies. For instance, in his study carried out with 204 undergraduate students from 11 different faculties of medicine all over Turkey, Tontus (2010) found out that the students view teaching negatively, think that they are not encouraged to participate during teaching sessions and the teaching is too teacher centred. In another study, Mirzazadeh et al. (2016) concluded that the medical students were mostly dissatisfied with the medical training they received, which means it did not meet their expectations.
Unfortunately, a comparative review of research in higher education literature reveals that teaching is organized in similar ways throughout the world, and causes common problems in terms of student learning. Indeed, how faculty teach-that is, the instructional methods, learning environment and assessment tools used-is quite similar worldwide (Forrest, 2004). The findings of this study supposed Forrest's (2004) assumption. On the other hand, it should be noted that students are not satisfied with that general tendency among the faculty members and they expect to be involved in the learning process actively. It is noteworthy that one of the fourteen Principles for Improving Higher Learning is that "Student performance is greatest when students are more actively than passively engaged in their academic work" (Angelo, 1993). In parallel with the principles, that faculty members apply different approaches to teaching is a necessity of CBME as indicated by Book (2014) as the students expect so.
These findings show that the faculty members both in and out of Turkey need pedagogical support. One of the reasons for that can be the general tendency of not only medical schools but higher education institutions in general. They only recently started to become aware that teaching, like research and the practice of any profession, demands training (Costa, 2010), which has resulted in some kinds of trainings focusing on the pedagogical needs of the faculty members. These training courses are mainly applied as workshops, seminars and the short-term courses and related literature reveals that both the attended faculty members and their students declared their satisfaction for the training courses (Steinert et al., 2005).
When it comes to assessment of students' learning, the faculty members explained that they generally use multiple-choice and/or true-false questions. On the other hand, they stated that with CBME, they aim to train students to apply what they learn. As related literature indicates, such question types are not proper for that aim. Instead, the faculty members should apply alternative assessment tools to determine whether the students apply what they learn.
In the frame of product evaluation, the views of the faculty members and the students about the effects of CBME were determined in terms of academic/vocational developments of students. The faculties and the students agreed on that the CBME helped students improve themselves both academically and vocationally. As the related literature supports, the CBME contributes to students' academic development by involving them in the learning process and directing them in selfdevelopment process (Candy, 1991;Toohey, 1999), and to their vocational development because it focuses on the vocational studies (Smith & Dollase, 1999).

CONCLUSION AND SUGGESTIONS
In comparison to traditional approaches to medical education that are expert-driven, and internally produced, CBME provides a method of organizing medical education that is learner-centred and oriented around population and health care system needs (Iobst et al., 2010). It describes the expected end-product i.e. a physician at the end of training; measures whether the outcome expected is achieved; and helps to identify learners experiencing difficulties in a short time, offering opportunities for enabling achievement (Saucier et al., 2012). In order for CBME to provide the pre-mentioned opportunities in medical education context, it is needed to develop and then evaluate CBME curriculum in the frame of theoretical backgrounds and principals. Otherwise, because of the problems/deficiencies in the development and implementation process, the efforts cannot end in vain.
In order to reach the expected results, the medical schools implementing CBME and those in a desire to implement it should explain aims, principles of CBME to their faculty members and students. They should provide short term courses, workshops, seminars, booklets, and informative videos on how to determine/write the competencies and the content; on effective teaching/learning methods. They should also make the necessary regulations in order to implement alternative evaluation approach. The current study has limitations in terms of the context and the participant. The future studies can be conducted in different contexts with greater number of participants. The current study is a qualitative study, various quantitative or comparative studies can be conducted including the alumni, too.