Standardizing assessment practices of undergraduate medical competencies across medical schools: challenges, opportunities and lessons learned from a consortium of medical schools in Uganda

Introduction Health professions education is gradually moving away from the more traditional approaches to new innovative ways of training aimed at producing professionals with the necessary competencies to address the community health needs. In response to these emerging trends, Medical Education for Equitable Services to All Ugandans (MESAU), a consortium of Ugandan medical schools developed key competencies desirable of graduates and successfully implemented Competency Based Education (CBE) for undergraduate medical students. Objectives To examine the current situation and establish whether assessment methods of the competencies are standardized across MESAU schools as well as establish the challenges, opportunities and lessons learned from the MESAU consortium. Methods It was a cross-sectional descriptive study involving faculty of the medical schools in Uganda. Data was collected using focus group discussions and document reviews. Findings were presented in form of themes. Results Although the MESAU schools have implemented the developed competencies within their curricular, the assessment methods are still not standardized with each institution having its own assessment procedures. Lack of knowledge and skills regarding assessment of the competencies was evident amongst the faculty. The fear for change amongst lecturers was also noted as a major challenge. However, the institutional collaboration created while developing competencies was identified as key strength. Conclusion Findings demonstrated that despite having common competencies, there is no standardized assessment blue print applicable to all MESAU schools. Continued collaboration and faculty development in assessment is strongly recommended.


Introduction
There has been a global trend in health professions education to move away from the more traditional teacher-centred learning to more innovative student-centred learning approaches such as problem based learning and competency based education [1,2].
Such changes in teaching and learning also require changes in student assessment in order to achieve an effective constructive alignment [3,4]. It has been reported that effective student assessment drives learning and that assessment plays a crucial role in the learning process [5,6]. Literature is replete with documentation about student assessment in health professions education.
Davis and Harden [7] describe assessment as the single most important determinant of what students actually learn (as opposed to what they are taught) and is considered to be uniquely powerful as a tool for manipulating the whole education process. Epstein and Huder [8] further supplement that assessment becomes the most important basis of the medical education system to public accountability. In order to achieve maximum benefits from assessment activities, it should not be detached from the planning of the curriculum and should always be an active component in curriculum planning activities [9,10].
Van der Vleuten [11] describes assessment as an evaluation of student performance against set learning outcomes and Bridge et al [12] further supplement that the basic tenets of any assessment are 1) validity (whether the chosen method measures what it claims to measure), 2) reliability (the degree to which the measurement is accurate and reproducible), 3) acceptability (the extent to which stakeholders; students, faculty and patients, endorse the measure and associated interpretation of the scores), 4) feasibility (affordability and efficiency of the assessment method for testing purposes) and 5) educational impact (the capacity of the assessment method to drive students' study efforts based on the curriculum requirements and how assessment leads to overall improvement on the curriculum.
The choice of any assessment method depends on the purpose of the assessment and the student competencies being assessed at any one particular time [13]. Thus different student competencies need to use different assessment methods in order to make objective judgments about student performance. It is thus evident from previous literature that suitable assessment methods need to be in place in order to assist lecturers make objective judgments about students.
Innovations in medical education have led to the emergency of what is called transformative learning where teaching and learning no longer focus on knowledge only, but also emphasize other professional competencies such as clinical/practical skills, communication skills, leadership, professionalism and ethical practice [7]. Such type of learning is likely to produce medical professionals who are well grounded in not only knowledge, but also in other graduate attributes that make a complete professional and which are necessary to meet today`s health needs of the population. In such learning, student competencies have to be clearly defined and assessment designed to evaluate those competencies [10].
In order to address the then challenges in health professions education in Uganda and produce professionals with the desirable competencies to address the community health needs, five Ugandan medical schools teamed up to form a consortium called Medical

Results
The findings from this study are summarized under the following themes

Assessment of the competencies: the current situation
Analysis of data showed that the MESAU schools were indeed implementing competency based education. The competencies of Medical knowledge, Clinical skills, Professionalism and Communication skills were well documented and faculty were aware of these competencies as developed by the MESAU consortium.
Regarding the assessment of these competencies, findings from the data showed significant variations across the MESAU schools. Table   2 illustrates the assessment methods of each competency within each medical school. From the table, all the medical schools seem to concentrate on assessing knowledge and skills. Majority of the schools lack properly documented methods for assessing soft skills like professionalism and communication. Additionally, there is clear evidence that assessment methods are generally not the same across the medical schools despite the fact that they are assessing the same competencies. Each medical school seems to be using its own assessment methods. During the interviews and focus group discussions, this clearly came out as illustrated in the response below: ``Although we got some training and workshops on CBE together with other medical schools in the consortium, I do not remember discussing and adopting common assessment methods for these competencies. We just devised our own within our individual school``.

Challenges
Another major theme that emerged from the data pointed at challenges faced as far as assessment of competencies was Page number not for citation purposes 4 concerned. One key challenge that was prominent in the data gathered was a lack of a standard and uniform assessment plan and blue print that seemed to be overlooked during the development of the competencies. ``While we gathered together as a consortium of medical schools to develop common competencies, we unfortunately overlooked the idea of simultaneously adopting common assessment methods for each competency``.
``Since we want to produce graduates with similar attributes for our population needs, we must have curricular with similar attributes across all the medical schools which we did effectively during stakeholders meetings. However, we must also have similar assessment methods for those attributes which we unfortunately gave little attention``.
From the above responses, it can be observed that while common competencies desirable of graduates were adopted for the MESAU schools, assessment methods that should be uniform and standard across the schools seemed to have received less attention during the process of developing competencies.
Another key challenge identified was limited knowledge of lecturers regarding the assessment of competencies. There seemed to be a knowledge gap amongst the lecturers as to how to assess the new competencies and they seemed to be deficient of the suitable assessment methods for each competency. The following response reflects this general observation: ``To many of us, this competency based education is a new thing.
We do not know how to effectively assess students in many students. Even the competencies where we have some experience like medical knowledge, there are still issues. To make matters worse, each medical school seems to be assessing the competencies using their own methods``. ``Am used to the way we have been assessing students without these competencies. I appreciate the introduction of the competencies, but I hope if we do have common assessment practices, we shall remain with authority as lecturers``.
From the response above, one can observe that there is fear that lecturers will lose their authority of assessing students the way it has always been done if common assessment practices are adopted across the medical schools.

Opportunities
However, there were also some opportunities cited. Having developed and adopted the medical competencies across the MESAU schools, this provides an opportunity to collaborate across the medical schools to develop common assessment methods for each of the competencies in future. The momentum gathered during the development of the competencies can be extended to developing assessment methods. This opportunity was reflected in the response below: ``Since we can now collaborate as medical schools, this provides us a chance to have similar assessment methods for the competencies such that we can use the same benchmarks when assessing students``.
Another opportunity that emerged from the data was the possibility of having joint planning meetings since the medical schools can now communicate to each other within the consortium.
``The idea of being in a consortium as medical schools gives us an opportunity to organize assessment planning meetings. This will Page number not for citation purposes 5 give us a chance to have standard assessment methods that are similar across the medical schools``.
Having a unified voice for the Ugandan medical schools to advocate for change in policies governing higher institutions of learning was another opportunity. ``We could use our voice and strength as a consortium to advocate for change whereby we can suggest the possibility of having common assessment boards for medical schools``, said one participant.
Lastly, there was an opportunity within the consortium of sharing best assessment practices as some of the schools seemed to be a head of others.

Lessons learned and way forward
The participants in this study also highlighted key lessons and a way forward. One key lesson was the benefit of collaboration and team work across the medical schools. Participants observed that the consortium approach had enhanced collaboration across the medical schools which was not the case before and this collaboration could be used as a foundation to advocate for standardized assessment practices across the schools. Subsequently, it was suggested that having joint faculty development programmes in assessment is essential as many participants felt they had less knowledge and skills in assessing the new competencies.
``The major solution is faculty development. Many of us are ignorant about assessment of competencies and we need to have joint training programmes across all the medical school…`` Another key lesson learned was the practice of looking at training in its entirety not just focusing on few aspects. Participants felt that assessment methods should have been discussed alongside the competencies with all stakeholders, but this was not the case.
Therefore, it is essential to scrutinize the curriculum as a whole including content, teaching and learning methods, assessment, implementation issues and evaluation plans.
Participants suggested going through the same procedure of identifying stakeholders such that assessment methods are developed collaboratively with involvement of all stakeholders like it was done for the competencies.
``Since we came up with competencies as a team of stakeholders, let us also identify those stakeholders and develop assessment methods as a team``.
Continuous engagement of medical schools across the consortium and sharing of best practices was also suggested as a way forward.
This continuous engagement was also suggested as a way of continuously sensitizing faculty about the benefits of having standardized assessment practices across a consortium of medical schools that target having graduates with similar competencies.
``We should continuously share best practices through seminars and training workshops as a way of improving and as a change management strategy because such continuous engagement can create buy-in from those lecturers that are afraid of change``, one participant said.

Discussion
The purpose of this study was to examine the current situation and Inadequate knowledge and skills of faculty on how to assess the various competencies was identified as a major challenge. Training faculty in CBE including assessment has been emphasized in previous literature [7]. In their study, Davis and Harden [7] reported that having well trained faculty with knowledge of CBE and how to objectively assess the various competencies is key to successfully implement CBE. In this study, lecturers had limited expertise in assessing the newly developed competencies partly because there had been no faculty development programmes prior to commencing CBE. It appears like many of the lecturers still depend on the traditional assessment methods to assess the competencies. Unfortunately, many of these assessment methods may not be valid and reliable in assessing the new competencies.
For example, using an oral exam to assess students` der Vleuten and Schuwirth [13] caution that depending on traditional assessment methods to assess the newly emerging competencies is a disservice to the students. Therefore, the need to train faculty in assessment is paramount.
Similarly, the fear of change was a major factor noted in this study.
Many lecturers seemed to be comfortable with the way they had been assessing students. Some of them felt that their authority would be taken away if common assessment methods were adopted. The fear to change from the traditional of teaching and assessing students has been reported previously [3]. Findings from this study are in resonance with the previous studies that faculty fear change. The reason as to why this could be so is the fear of the unknown. Many lecturers in medical schools are not education experts and many of these concepts and principles are new to them.
To many they do not make sense. This study also has some implications for future research. With the autonomy of higher institutions of learning in many countries where each institution develops its own curricular and designs its own assessment methods, the issue of standardizing assessment practices across all medical schools in a country needs further investigation. This is a key direction for future research in this area.
Another direction for future research would be to investigate whether medical schools teaming up together to form consortia can have greater impact on advocating for change in policies that affect both training and health care provision.
In this study, non-probabilistic sampling was used to select participants, which is a limitation and thus findings may not be fully generalized to other settings. Additionally, the fact that not all lecturers in the medical schools were involved; some important information could have been missed from them which could probably have added richness to the findings, another limitation of the study. However, the study provides useful findings upon which more studies can build.

Conclusion
The MESAU consortium successfully developed common competencies which have been successfully implemented within the undergraduate medical training across Ugandan medical schools.
Despite this success however, there are still challenges of standardizing assessment methods of the competencies across the medical schools. The collaboration amongst the medical schools within the consortium however provides a useful platform upon which the institutions can come together to standardize assessment of the developed competencies. Such collaboration can also provide key lessons to other medical schools in Africa and beyond.