WHAT CHANGES IN IMPLEMENTING COMPETENCE-BASED MEDICAL EDUCATION IN INDONESIA: A QUALITATIVE STUDY

Background: Competence-based Medical Education (CBME) is the latest curriculum model adopted by many countries since 1970s. Reforming medical curriculum to adopt CBME implicates major changes in all aspects and research on this is still lacking. This study aims at identifying changes in implementing CBME from the aspects of organization and structural changes, curricular design, implementation and evaluation and cultural changes. Methods: Retrospective qualitative method is applied using purposive sampling. Selected documents are used as the data and analysed using thematic analysis. Results: Four themes are identified for the first aspect, i.e. the role of central authority, multidepartment committees, committed change agents, and decision-making procedures. Whereas for the second aspect, five themes are pinpointed, consisting of curriculum design at macro level, meso and micro level, faculty development program, learning resources, implementation, and monitoring and qualitative evaluation. For the third aspect, four themes emerge, namely enabling factor, inhibiting factor, the paradox of the new curriculum and quality assurance. Conclusion: Major changes taking place in the design and implementation of CBME have been identified from three aspects inductively. Awareness of the kinds of changes and using them for curriculum planning could improve the success in shifting towards CBME.


INTRODUCTION
Competence-based medical education (CBME) have been widely adopted in many countries. For the last five decades, CBME has been at the top of discussions in many conferences. As early as 1978, the WHO published a Competency-based Curriculum Development in Medical Education. 1 This is sparked by the competency-based movement in higher education in the 1960s where the premise is that explicit minimal competency-based education would promote educational accountability 2 as stakeholders concerned have the same level of expectations of the graduates' ability. Medical educators across major developed countries, namely United States, United Kingdom and Canada have been endorsing the adoption of competence-based medical education. 3 A growing number of key publications significantly increased. 4 Report on the implementation of CBME also mounted at the pilot level as well as at the full scale level across the whole curriculum. 5 There have been efforts to clarify the definition, constructs and characteristics of competencies and CBME. 6 Albanese et al argued that clear definitions of competencies are needed, because competency framework is now used for certification and maintenance of certification. They proposed the following definition: 'competencies are knowledge, skills, attitudes and personal qualities essential to the practice of medicine'. They also formulate characteristics of competencies, such as 'focus on the performance at the end of a particular stage, mastery of competencies is reflected in the application in the context of patient care, use of performance standard that is criterion-referenced'. 7 Tuning Educational Structures in Europe defines competences as 'a dynamic combination of knowledge, understanding, skills and abilities.' Competences are obtained or developed during the process of learning. A distinction can be made between generic competences (i.e. transferable competences across study areas) and subject-specific competences (i.e. competences specific to a subject area). 8 Epstein and Hundert propose the definition of professional competence as 'the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values and reflection in daily practices for the benefit of the individual and community being served'. 9 Physicians' competencies are multidimensional, dynamic, contextual and developmental. It comprises of multiple domains of abilities. For each domain of competence, there is a corresponding spectrum of ability from novice to master, contextual and ever-changing. Physician has a unique constellation of abilities at any time and in any context. 6 Competency development requires specific conditions and contexts. Learners need to observe models and exemplars that illustrate processes and outcomes to be learnt. They need to be given opportunities to perform focused practice and rehearse specific skills, whilst receiving immediate and informative feedback in a defined setting. They should be encouraged to perform these skills in diverse settings. They also must reflect on their performance. 10 Central to the development of competency is the process of longitudinal direct observation with scaffolding. Teachers provide successive levels of support that enable learners to attain successively higher levels of comprehension and skill acquisition during this time.
Despite the variety of definitions and understanding of competencies, in medical and health professional education, competencies frameworks have been used as the reference for curriculum design and planning at the school level. A number of countries have published their competencies framework, such as Standards of Indonesian Medical Doctors' Competencies, 11,12 Outcomes for Graduates, 12 CanMEDS 2015 Physician Competency Framework, 6 and ACGME six core competences in the US. 13 All these competencies framework comprises of several domain or area of competencies, such as communication skill, system-based practices, professionalism, etc. Each domain of competency is broken down into a number of competency components. Each competency component is further detailed into more specific learning outcomes. The identification of domain of competencies and their correlated competency components takes into account the role and functions of physicians in the health care system.
However, there are critics regarding the concepts of competency model being applied for medical profession, be at undergraduate or postgraduate level. Talbot argues that competency model might not be most appropriate for educating medical professions. Such competency models are constructs taken from the industry in the USA and adopted in UK for vocational trainings, later on by many countries. This model is grounded in the behavioral and cognitive paradigms. In this model, tasks are broken down into smaller units and the assessee is graded according to the successful completion of each unit of competency. 14 This way of looking at tasks is too simplistic and reductionism. Squires develops a medical practice which shows the complexity of the medicine. 15 Medical practitioners need to move beyond competencies. Malone and Supri argues that competence is complex and cannot be reduced into isolated elements, competence-based approaches are not appropriate for highly skilled profession such as medicine. They explain that competencebased education is more applicable for lower level occupations that involve routine tasks and simple testable skills. 3 Notwithstanding all the controversies surrounding the competence-based education, this educational approach is still the most popular. Evidences of forty years' application of competence-based curricula across the globe reveal that this educational approach is philosophically questionable, methodologically complex and highly controversial. 3 Major curriculum reforms have been attempted for the undergraduate and postgraduate medical education based on the competency's framework. An example is in Canada. The Royal College of Physicians and Surgeons of Canada has embarked on a major shift in medical education. Competency-based medical education is perceived as an integrated, longitudinal, trainee-focused approach "to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. It deemphasizes time-based training and promises greater accountability, flexibility, and learner centeredness." 8 Nousiainen at al highlighted three key aspects of implementation that will be encountered when a medical school decides to implement CBME, namely organizing the structural changes that will be necessary to deliver new curricula and methods of assessment; modifying the processes of teaching and evaluation; and helping to change the culture of education so that the CBME paradigm gains acceptance. Nine considerations are identified as supportive for changes for the two key aspects. 16 For the third key aspect, Ferguson et al identified cultural barriers to adopt CBME existed in the culture of medical education institutions. Therefore, change in educational culture must be embraced by all components of the medical education hierarchy. Research is essential to provide convincing evidence of the benefit of CBME. 17 Four overarching challenges have to be considered when embarking on the adoption of CBME at all levels, namely: "(1) aligning regulatory stakeholders to support competency based education and training; (2) integrating educational and clinical redesign efforts to align curricular objectives with experiential training; (3) establishing defined outcomes that reflect the needs of patients and populations in which individuals, programs, and institutions can be measured; (4) ensuring accountability among all stakeholders for the achievement of defined outcomes". 18 However, it is debatable whether a competence-based approach to the medical curriculum will fulfil these intended aims. 3 In Indonesia, the Medical Practice Law No.29/2004 adopted the competence-based approach in the medical profession education. This Law mandated the Indonesian Medical Council to issue Standards of Medical Doctors' Competencies in 2006, which comprises of seven areas of competencies as follows: (1) effective communication, (2) clinical skills, (3) scientific basis of medical knowledge, (4) management of health problems, (5) management of information, (6) self-awareness and self-development, (7) ethics, morals, medico-legal, professionalism, and patient safety. 19 The Indonesian Medical Council also passed the Standards of Medical Profession Education as a guidance for medical schools on how to develop CBME.
Since then, between 2006 until 2011 Indonesia experienced nationwide medical curricular reforms. Each medical school gradually strived to adopt CBME based on the Standards of Medical Doctors' Competencies. However, there were variations in the process of changes from subject-based to CBME among sixty medical schools with diverse conditions, likewise the implementation of CBME itself. Each medical school has an autonomy to develop their own curriculum, resulting in the varied models of CBME. The aim of this study is to investigate what changes have occurred in implementing CBMEboth from the organizational and curricular aspects.

METHODS Context
This study was conducted at Faculty of Medicine Universitas Gadjah Mada (FMUGM), because it was the first to implement CBME in Indonesia at full scale starting in the Academic Year 2007/2008. Taking into account the complexities of changes and to ensure that the reform process was smooth and on the right direction, FMUGM invited Maastricht and Groningen Medical Schools to coach during the preparation, implementation and evaluation of CBME from 2006-2014. A two million Euro grant from the Netherland Government under the Netherland Post-Secondary Training (NPT) Scheme was obtained to fund this competence-based curricular reform.

Study design
Qualitative paradigm is used in this retrospective study with the main focus to understand the organizational and curricular changes in implementing CBME. Qualitative researchers study phenomenon in their natural settings, attempting to make sense or interpret the phenomena in terms of the meanings people bring to them. 20 The phenomenon being studied here is changes in the implementation of CBME.

Sources of evidences
The main evidence used in this study is documents because they can provide a means of tracking change and development. Documents can be used as stand-alone method. 21 Documents are secondary data that are situational and exists independent of the study, and are not created for the purpose of the study. Secondary data provides objective buffer between the researcher and the researched. However, some cautions need to be taken into consideration, such as articulating the research questions clearly and selecting documents that could address these questions. The documents were screened for authenticity and credibility. After the documents were carefully selected, the data were examined and interpreted in order to elicit meaning, gain understanding and develop empirical knowledge. 22 As documents are secondary data, ethical clearance is not needed.

Data collection and analysis
Documents related to the design, preparation, implementation and monitoring of CBME implementation during the NPT Project at FMUGM were selected, comprising of: (i) project proposal, (ii) inception report, (iii) 6 project management reports, (iv) 8 management visits reports, (v) 2 monitoring report by external consultants, (vi) annual reports from 2007-2011, (vii) final report 2012 (viii) one external evaluation report from renown international consultants. The selection of documents was guided by the aims of the study, i.e. understanding the changes in implementing CBME, from the organizational and curricular aspects. 16 Thematic analysis is applied in this research as it is most appropriate for systematically identifying, organizing and developing insights into patterns of meaning (themes) across the data set. This offers researchers to make sense of collective meaning. 23 The purpose of thematic analysis is to identify relevant themes to answer particular research issues.
All documents were given symbols and read thoroughly. Paragraphs that were relevant to the research questions from each document were copied and pasted into a table. After all the excerpts from each document were collected in the table, the principal researcher read several times and colour coded the paragraphs or sentences that bore underlying meaning. Initial coding was first created for the colour coded paragraphs by the principal researcher. After several iterations, the initial coding was refined to provide answers or meaning to the research issues. The second author did the second coding from the excerpts independently. The principal researcher identified themes from the second coding. Final themes and subthemes were created after each emergent theme was validated. The final themes were grouped according to three key aspects of changes in CBME implementation. The third author reviewed the final manuscript.

RESULT AND DISCUSSION
A total of 212 pages from the selected documents were analysed. Seventy-two initial coding were constructed and 13 themes were identified as depicted in Table  1. They are group into organizational and structural changes that relates to the organizational structure, management and governance needed to deliver the curriculum; curricular process pertaining to the design, implementation and evaluation of the curriculum; and cultural changes.

No
Themes Grouped in Three Key Aspects of CBME Implementation

Multidepartment committees
One implication of having a central coordination in CBME is to establish multidepartment committees/ units/teams. This obviously implicate a change in the organization structure. When embarking on preparing the CBME, it was soon realized that the first thing needed to be changed was the organization structure. This has been highlighted by Nousiainen et al. 16 A number of multidepartment committees were established, as described in the following excerpts.

Decision making procedures
As for the members of Faculty Senate and Faculty Board, a series of leadership workshops run by experts from Maastricht and Groningen medical schools were conducted to introduce the tenets of CBME and make them committed to adopt this education approach. To avoid the confusion of who is doing what as the whole institution was in the process of changes, the ISO management system was installed and hundreds of standards operating procedures (SOP) were developed through a series of workshops. This is to ensure decision making procedures were complied. The following excerpt explains the leadership workshop. This endeavor confirms Ferguson et al to align regulatory stakeholders to support CBME and to ensure the accountability among all stakeholders for the achievement of defined outcomes. 18 In this case, it is the internal stakeholders that were intervened through internal regulation, such as academic policies.
At the block level, the decision-making process in the selection of content is done through consensus building, such as nominal group technique to avoid gaps and overlaps of content across the blocks. The decision-making process at the block level needs an authorization from the higher level, such as faculty senate. This is explained in the following excerpt.

Curricular Design, Implementation and Evaluation Curriculum design at macro, meso and micro level
Harris et al highlight that CBME involve curriculum design accommodate a various number of activities, each of which should contribute to the achievement of explicit, agreed outcomes. Once these outcomes are clearly set forth, curricular components are designed to foster the acquisition of ''enabling skills'' and of knowledge. This should be structured in a logical sequence. 25 In a CBME system, curricula and assessments are driven by predefined outcomes or competencies. Learners are expected to demonstrate achievement of these outcomes before moving to the next stage of training. CBME does not mandate any particular teaching strategy or philosophy, and many methods (e.g., problem-based learning, case-based teaching) may continue within the CBME approach.
Parallel to the reorganization and structural changes, the curriculum design was conducted. The issuance of the Standards of Medical Doctors Competence by the Indonesian Medical Council in 2006 came at the right time of the beginning of the project. It was soon agreed at the inception phase that competence-based medical education would be adopted. FMUGM already implemented the problem-based learning in 1992 partially, and gradually in 2002 it reached the full scale. When the agreement to adopt CBME was taken, the narrative that was used was the process was still the same -i.e. problem-based learning as the backbone, but we only changed the outcome of the medical graduates to comply with the Standards. Due to this, a number of adjustments in curriculum structure and contents, with the emphasis on clinical education, as well as in assessment system needed to be done. In line with Harris et al as explained above, CBME is flexible enough to accommodate any existing learning strategy. This is explained in the following excerpt. Designing curriculum at macro level was executed by the curriculum committee who worked closely with the experts from Maastricht and Groningen medical schools. A number of workshops to develop a curriculum map for the whole years from academic to clinical phase and also study visit to Netherlands were conducted. In those workshops that were attended by representatives from all departments (24 departments), the philosophy of CBME and how to translate the Standards of Competences into curriculum map were discussed. It was quite a rigorous and iterative process when the participants distributed hundreds of competency components and learning outcomes, including seven hundred diseases, two hundred clinical skills, hundreds of problems -based on sign and symptoms into each block to make up a blueprint block or meso level. Each blue print attempted to balance between national standards and local contents. To achieve a full integration of curriculum -both horizontal and vertical integration as well as longitudinal lines, a new curriculum map was agreed. The above process is explained in the following excerpts.  [14][15][16][17][18][19][20].
Once, the curriculum map and block blueprint for each block were approved, each block coordination team develop a block book for tutor and for students. At the micro level topic tree was developed and trigger problems were identified. Then, weekly instructional design was developed. Blueprint assessment for each block was also plotted to assess the achievement of block learning outcomes.
At macro level, assessment strategy was also designed systematically to ensure that the assessments of achieving competencies components and learning outcomes from various domains were included.

Faculty Development Program
Whilst the process of designing and developing were taking place, the committee responsible for faculty development designed a faculty development programme based on the roles and function of academic staffs. During this phase, training and workshops on 'CBME: the challenges and pitfalls' were most crucial to make everybody understand the rationales behind the whole process of changes. Tutor and instructor training as well as block development trainings and clinical rotation guidelines were also prioritized. The following quotation shows the faculty development efforts. "The staff appreciated the educational training that they had received and still receive every 6 months. More staff training would be appreciated (both more staff members and repeated training). We recommend that the staff training that was already initiated be continued and possibly expanded, including the opportunity for staff to meet colleagues and exchange experiences" (EE06 page 13).
This is in line with Dath and Iobst who argue the important of faculty development in the transition to CBME, be at the system level as well as at the individual teacher level. 28

Learning resources
In conjunction with other developments, a committee on learning resources identified skills lab equipment that had to be procured under this grant, not only for the skills laboratory at FMUGM, but also for outreach teaching hospitals. Assisted by experts from Maastricht Medical School, the layout of skills lab was redesigned. The instructional materials, such as literature references, skills training guidelines, and block books were also developed. The following quotation reflects the opinion from the external evaluator.
"We are impressed by the quality and "looks" of the block books, the tutor guides, the instructional materials used in the Skills lab, and instructional materials used in the clinical part of the program. We have never seen a more systematic attempt to make the curriculum explicit for students and staff then here at UGM. We conclude this with much respect for all those who made this happen: staff at UGM and technical advisors from Maastricht and elsewhere" (EE06 page 8).
"The Skills lab is a world-class facility in its own right. It is quite amazing how much has been made possible with limited means. Students often mention the Skills lab as the best part of their curriculum. The early training of the doctor's skills gives them the self-confidence to deal with the challenges of the clinical rotations. The young full-time staff makes the impression of being extremely committed and competent" (EE06 page 8).
"Nowadays UGM is equipped with a modern, well equipped skills lab, very well organized with well trained staff. The skills lab and its staff meet the criteria necessary for a fully integrated skills training in the medical curriculum" (FR12 page 14 lines [24][25][26][27][28][29] The affiliated teaching hospitals were also equipped with skills training equipment. The idea was that the students could always practice their basic clinical skills, although they were posted in teaching hospitals away from the main campus. The placement of medical students in teaching hospital was well planned based on calculation of the student-patientstaff ratio as explained in the following quotation.

Implementation
The implementation of CBME started with the batch 2007 Semester 1, whilst other semesters were still using the old subject-based curriculum. The implementation in semester 1 could be seen as a 'try out phase'. While the semester 1 was running, all the committees were preparing for all components needed to run the semester 2. At the end of each block, the evaluation team did the block evaluation and the results were immediately fed back into the next block. This way, a gradual expansion was taking place as the students moved up the semesters as shown in the following excerpt.

Monitoring and Qualitative Evaluation
Monitoring and evaluation were conducted by the external evaluators. The project hired renowned international experts to do the external evaluation qualitatively. The results demonstrated that the new curriculum was more enterprising. The students became more curious, they had better communication skills and had more self-esteem. The external evaluators also observed that the students had higher level of clinical competences and enjoyable learning environment. They commented that tutorial discussion did work, because they presumed that in the Asian culture where power distant is high, tutorial discussion might not work as good as in the western countries with a more egalitarian culture. 29 The following excerpt illustrate this. The following quotations reflect students' opinions.
"I love to discuss the subject-matter with my friends in the tutorial. It helps me understand the issues much better. It's much easier to ask questions in the group than during lectures" "The tutorials lead to deeper learning and better integration." "The best thing of the UGM-system is the integration between the basic and the clinical sciences. It helps you put the knowledge to use." "It's two-way traffic here. Students and staff are very close. My friends in Malaysia who are in medical school there tell me that they have never spoken to a member of the staff." "The shy students learn to feel at ease here and become self-confident. I learned to speak up in the group and to actively contribute." "I like that the training is so practical. Also, with the Skills lab (EE06 p 4) From the students' perspectives, the students perceived to have deeper understanding, more opportunities to discuss the issues, better integration of basic and clinical skills. They also had more opportunities to directly apply their knowledge.
They also felt to have closer relations with the staffs. From the tutors' perspective, they perceived that the new education approach drove them to update their knowledge.
The following excerpts illustrate the external evaluator impression of the CBME.
"The assessors are very impressed by the curriculum as it is in its current state. It is a modern curriculum that not only is designed carefully, but also appears to be viable in the daily reality of teaching and learning in a professional environment (EE06 page 16).
"Our own observations of small-group tutorials corroborated these opinions of students and staff. We were struck by the quality of the discussions, by the inviting and informal atmosphere in which the discussions took place, and by the fact that the group members all actively participated in the discussion" (EE06 page 5)

Cultural Changes
With regards to cultural changes, there were enabling factors. The presence of Maastricht and Groningen Medical Schools were seen as the mentor and the coach. They adopted the approach that was more demand-driven, flexible, and local ownership instead of imposing their thoughts and experiences. This had made the academic staffs at FMUGM more confident, more cooperative and more receptive to the new approach of CBME. The role of departments was strengthened. This also had provoked more involvement of the departments during the curriculum design and preparation. During the running of the project, the deanery had always been supportive of the adoption of CBME and had exercised strong leadership to direct the changes under the guidance of Maastricht and Groningen Medical Schools. Negotiation and consensus were part of the process and these were allowed to occur making academic staffs felt herd and appreciated. The following quotations illustrate this phenomenon. However, there were inhibiting factors that could hinder the progress of CBME implementation. Some of the senior academic staffs still had subjectbased mindset. It was understandable as they were going through this curriculum models for dozens of years. However, they were deliberately intensely involved and were given responsibilities as chairs of committees. Other inhibiting factors were long hours of working and multiple commitments from the academic staffs. Although they showed enthusiasm, but the bureaucratic regulations pushed them to have multiple commitments and responsibilities.  "We also realized that: There is a natural and quite understandable resistance within the departments to the proposed student-oriented changes in the curriculum. As far as we can see this resistance may well be caused more by an overload of tasks for individual staff members than by a resistance to change as such" (MOM07 lines [30][31][32][33][34][35][36] From students' perspectives, the following are the inhibiting factors.

Inhibiting Factor
"Heavy study load, lack of basic knowledge to get proper discussion during tutorials, Not well instructed tutors, limited access to internet through hotspots, postponement of lectures and even tutorials, assignments which were not clear to them" (VR 08 page 2 lines [11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27] "Like tutors, instructors experience difficulties being available at times agreed upon. Therefore, students are confronted too many times with last-minute replacements of these instructors, replacements who are not always well prepared. It would be better to either pay instructors directly for educational services rendered to the Skills lab ("fee for service"), or increase the number of full-time instructors. This would ensure continuity of the skills curriculum and optimal quality of the instruction" (EE06 page [9][10] Literatures also suggest that enabling and inhibiting factors exist when moving towards CBME, such as Caccia et al, 8 Hawkins et al, 26 and Griffiths et al. 30 The paradox of a new curriculum The paradox of new curriculum is when new curriculum solves the old problems, but at the same time also create new problems. In this regards, the administrative and bureaucratic regulations from the Government were not quite in line with the principles of CBME. "A new curriculum solves some old and pertinent problems but it also creates some new ones. In the remainder of this report we will list some of these new problems and suggest avenues for improvement. Of course, we will throughout also report on the many things we found to be particularly positive developments" (RI06 page 2)

Quality assurance
To sustain the changes, quality assurance unit was established and equipped with full time administrative staffs. A dedicated team was appointed to execute the quality assurance functions. The following excerpt reflects how a quality assurance process function.
"Finally, a feedback-loop has to be developed to improve block programs, books and teaching materials of curriculum year 1 on the basis of feedback from students, tutors and instructors has been collected." (MM07 page 3 lines 1-7) Schelle et al. 31 agreed the importance of installing a quality assurance measures when implementing CBME.

CONCLUSION
This study explores the changes that took place during a curricular reform from subject-based curriculum to CBME in a faculty of medicine from three aspects. The changes are complex and involve the whole system. In the first aspect, i.e. organizational and structural changes, four themes were identified consisting of the role of central authority, multidepartment committees, committed change agents and decision-making procedures. With regards to the second aspect, namely curricular design, implementation and evaluation, five themes emerged -curriculum design at macro, meso and micro level, faculty development programme, learning resources, implementation and monitoring and qualitative evaluation. As for the third aspectcultural changes, there are four themes identified, namely enabling factors, inhibiting factors, paradox of the new curriculum, and quality assurance.

RECOMMENDATION
The kinds of changes that took place during the design and implementation of CBME have been elaborated based on empirical evidences. These changes can be used as references, inspirations, or planning by any medical schools who are about to embark on implementing CBME which could increase the possibility of success. These changes can also be referred to by universities or national governments to provide supports for the medical schools adopting CBME.