Developing an Assessment Tool to Measure Educational and Research Performance of PhD Students in the School of health Management and Medical Informatics

© 2015 The Authors. This is an Open Access article distributed under the terms of the Creative Commons Attribution (CC BY), which permits unrestricted use, distribution, and reproduction in any medium, as long as the original authors and source are cited. No permission is required from the authors or the publishers. Article History: Received:15 Feb 2015 Accepted: 18 June 2015 ePublished: 4 July 2015


Introduction
The national health system of any country requires skilled and capable manpower and integrated and effective efforts for these systems to function correctly. 1It is important to note that having sheer scientific skills is not enough for each profession.Academic careers, in particular, require another set of capabilities. 2ccordingly, the higher education system plays an important role in training competent students, particularly in postgraduate education.It is necessary that the curriculum promotes a set of qualifications for the students' future careers. 3n the other hand, development in the field of education (especially in areas related to health) without an effective evaluation system makes it difficult to achieve goals.Managers and policymakers need comprehensive, accurate and timely data for policymaking and planning in such a system.This data can be obtained through control and assessment of systems in the form of formative assessment (during the implementation course or program) and summative assessment (after the implementation or in specific and strategic periods). 4][7][8][9] It is clear that all aforementioned activities in the evaluation cycle must be implemented suitably and with high quality at all levels of the educational system, from schools and departments to faculty members and students.Since the leading systems worldwide in the field of educational accreditation pay special attention to these items and find the simultaneous promotion of items important, we must as well. 10t is necessary to promote the competence and improvement of postgraduate students, PhD students in particular, as they are not only studying at the highest scientific level, but will likely later acquire positions as faculty members or hold other education or research positions.In the near future, a proper education assessment system is an undeniable necessity for these students.This importance was shown in scientific literature that found that a 360-degree evaluation of students (i.e., self-assessment, peer-assessment, teacherassessment, etc.) was necessary. 11 review of previous studies indicated that certain criteria, such as the ability to apply knowledge in practical fields, good communication skills, emphasis on research and evidence-based activities, interest in teaching, teaching and presentation skills in scientific courses, planning and decision-making skills, critical thinking ability and creation and strengthening of professional skills were used in the assessment of PhD students' current skills.3][14][15] It is then necessary that the aforementioned criteria be collected in a formal tool that includes characteristics such as a respect for all of the basic features and capabilities in students' future careers, scores and values of each item, objectivity, avoidance of forgery and deception, brevity, the ability to show the situation in terms of numbers and figures and the ability toperform statistical analysis on collected data. 16ecause various studies found that performance assessment tools helped teachers and educational administrators to examine students' situations, improve objectivity in the evaluation process, integrate the academic evaluation system and enhance the students' knowledge and abilities, the present study aimed to develop educational and research tools for evaluating the performance of PhD students in schools of health management and medical informatics.The researchers hope that the implementation of this tool in students' respective fields leads to an important step toward improving the educational evaluation system, consequently improving the motivation and abilities of students at this critical and influential educational level.

Materials and Methods
This qualitative study was conducted in Tabriz School of Health Management and Medical Information.First, the researchers did a review study using various combinations of keywords and tools in databases, including Science Direct, PubMed, Ovid Medline and Springer, as well as the Persian equivalent of the keywords in the SID, Magiran and IRANDOC.Researchers looked for the main items related to the academic performance of PhD students.Papers referring to each of the mentioned items and to the factors or process of developing similar tools were identified and studied.Then, in order to improve the richness of the study, we did one-on-one or small group interviews with experts using open questions, recorded their desired essential criteria and items regarding the evaluation of PhD students in the medical education department and used these recordings in the later stages of research.All data and items obtained from the previous stages were then put in a questionnaire.To confirm the content and face validity of the tool, the questionnaire was reviewed by 20 experts. 17Inclusion criteria for experts to take part in the expert's panel included sufficient experience in the field of medical education, experience in training PhD students and willingness to participate in the study.In this part of the study, all questions were examined from the viewpoint of experts and the five aspects of relevance, transparency, simplicity, necessity and feasibility were assessed on a scale of 1-4.Based on the statistics, first the content validity ratio (CVR) was studied and, if the question was approved in this indicator, the other content validity indexes (CVI) were examined.In all cases, the 75 percent score was considered valid. 18,19Obviously, if any criteria or standard did not meet conditions and did not acquire the minimum score, they could not be included in the final tool.Then, to complete and finalize the designed tool, the researchers and experts classified the factors and the final standards accurately.By placing a value (based on a score of 100 for the whole tool) on each factor based on the effort needed to achieve the desired factor and its impact on both academic achievement and students' preparation for their future profession, a scoring system was designed for the developed tool.A monthly checklist based on the comprehensive tool was designed for students' to report their activities and achievements.In addition, a set of checklists was compiled to measure performance of students and to be used in the annual as well as final reviews using the comprehensive tool.Interestingly, the desired minimums in any area of the tool were considered an initial condition for participating in comprehensive exams, starting PhD thesis research, attaining the prerequisites for a PhD dissertation and graduation.Ethical principles considered in this study included the complete freedom of all participants to accept or refuse to cooperate in the study, obtaining informed consent from participants, respecting the privacy of participants and ensuring the participants that the use of their data and information was exclusively in line with the study goals.

Results
Twenty experts who had enough experience in the educational affairs of medical sciences universities participated in this study, and their characteristics are shown in Table 1 (some experts were placed in more than two of the following categories).In this study, the researchers identified 22 and 19 items related to assessment of research education performance using a literature review and interviews with experts, all of which were set in a Delphi questionnaire and sent to 20 experts for content validity.The whole questionnaire

A r c h i v e o f S I D www.SID.ir
was completed (with a response rate of 100 percent), and after necessary analysis the CVR and CVI indicators were calculated to 92 and 95 percent, respectively.Only one of the studied items did not get the minimum (a sample Delphi questionnaire is presented in Table 2).Next, the researchers attempted to classify the items obtained so that they set the similar indicators in specific categories and selected an appropriate title for each.Finally, 12 main categories in research performance and 11in education were obtained after carrying out the analysis.Subsequently, the spectrum related to any aspect was identified and determined and 96 items in the two main domains (53 for research and 43 for education) were obtained.
In the final phase of designing the tool, 12 experts at a special meeting selected items on the basis of a 100-point scale and the scores for research and education were considered 70 and 30 points, respectively.They distributed the scores between the domains and aspects based on their importance.The experts also determined the minimum score for each domain or aspect and assigned the mentioned minimum as a prerequisite and condition for comprehensive exam, dissertation and graduation.The process of implementing the evaluation system is designed in such a way that after students have completed the monthly evaluation sheets and delivered the relevant documents along with other complementary information, they must be examined in different period-annually, before the comprehensive exam and at the dissertation defense.If the documents and scores were confirmed by the department, students would be allowed to enter the next phase; in addition, to be on the safe side, the department can ask the students to present all documents in this tool before the comprehensive exam and dissertation in order to examine the related documents.If students obtain desirable scores, they will be encouraged by the department, but in the case of little or no progress in the dimensions determined by the tool, the causes are identified, some strategies and solutions are presented to students and then progress is monitored and evaluated.

Discussion
The aim of this study was to develop a tool through which the performance of PhD students in fields related to faculty of management and medical information could be used all over the country and even beyond the national boundaries (if it is localized and matched with the field).Through the use of this tool, students could be evaluated and reviewed constantly and through application of promoting indexes and norms within the standards and assessment items, their comprehensive progress and growth related to their field of study as well as their future jobs could be realized.In this study, a rich and comprehensive tool including all related and effective items in two fields, education and research, was designed through the use of a collection of the appropriate methods to carry out the study; furthermore, a score was considered for each item in proportion with its influence on the education and research performance of the PhD students.Another significant point is that a minimum acceptable score was considered for every student in vital stages and points of their education to ensure comprehensive growth of the students and help the students get ready to deal with different jobs when they start working.Another strength of this model is the use of the perspectives of PhD students in the studied fields in addition to the faculty members' views related to the field, which could undoubtedly lead to increasing acceptance of this tool for its consistency with the realities of the field.In a study conducted in 2002,multiple and scientific criteria were designed to develop a web-based model for assessing the performance of third-year medical students with 11 items on it, including competency base, communication ability, clinical oral exams, a structural objective clinical test related to evaluation ability, appropriate diagnosis and treatment and even the standard international test were recognized and devised within the model. 11n another study conducted in Tehran University of Medical Sciences in 2011, a tool that included three main domains of clinical skills, accountability and respect, including ethic and professional responsibility and communication skills, was designed in order to compare assessment methods of clinical skills in nursing students in the internal surgery ward in selected hospitals. 6n another study that was done in 2007 to develop an evaluation tool for nursing students, items such as theoretical knowledge, critical thinking, self-learning ability, continuous progress in job and effective communication with coadjutors were identified as the main criteria. 15t is notable that in contrast to the present study, all these studies were done specifically in the field of medical sciences focused on the clinical competencies in the studied fields, while in healthcare management science and related fields, clinical science is on a second level of priority.The most important point in this regard is the ability to apply existing resources in the healthcare field to achieve health system goals.Another reason for the difference in results may be due to the education level, because most studies have been conducted in the field of basic education whereas the present study was conducted on PhD students at the highest scientific level, that is, on PhD students who are preparing themselves to work as faculty members.We conjecture that if studies were done on PhD students in fields such as nursing and medicine, their compliance with the results of the present study would be extremely high.The limitations of this study were: the absence of a similar foreign tool for modeling in the field and using it in the present model; non-compliance of available tools, standards and indicators in clinical fields assessment due to differences in the nature of academic fields; the unavailability of many articles and books due to costs and high workload of the faculty members and students as a barrier to attend the interviews or fill out the questionnaire.

Conclusion
Considering that this tool is comprehensive, was developed Table 2. Delphi questionnaire for selecting valid items to be presented in the final tool based on community needs, includes all the essential components needed and uses a method appropriate with objectives of the study, the researchers recommend that the tool be used by domestic and even foreign faculties of management and medical information.It is also suggested that similar studies for developing useful tools be used for the evaluation of clinical students in accordance with the requirements of their specialized areas since this approach can extend outside the medical education field.

3 . 3 .
: is necessary (-): is not necessary Table The final tool for evaluating the educational-research performance of PhD students : is necessary (-): is not necessaryTable The final tool for evaluating the educational-research performance of PhD studentsA r c h i v e o f S I D www.SID.ir

Table 1 .
Characteristics of the study experts