The Effectivity of Multi Source Feedback (MSF) to Assess Professional Behaviour (Pb) of Nursing Students: an Evaluation Study

Background: Nurses are requested to give an excellent services to patients not only excellent in skills but also excellent in behaviour which serve comfort to patients. The behaviour is called professional behaviour or soft skill. Purpose: The aim of this research is investigating the effectiveness of MSF in assessing nursing students' professional behaviour (PB).Methods: The design of the research is an evaluation study which compares the result of PB between before and after joining practice in clinical setting. The participants were thirty three participants consisting of 33 nursing students and 3 mentors. Data were collected using a quantitative approach. The participants are requested to do self and peer assessment using SPRAT modification form while mentors are asked to assess nursing students using the same form. The assessment was conducted twice, before and after joining internship. The data were analysed to compare the score of PB before and after joining the internship among group of participant using Paired sample T-test or dependent sample T test. Result: The results showed that there were differences of score between before and after joining internship with α = 0.000; 0.001; and 0.000 for self, peer, and mentor assessment respectively. There was no difference of score before joining internship among assessors with α = 0.509, and there was a difference of score after joining internship among assessors with α = 0.005. It means that MSF is an effective method to assess PB of nursing students. Conclusion: Applying MSF method to assess PB of nursing student is suggested for better assessment.

A study stated that most nurses do not perform good PB, such as: not responsive, unclear instruction, grumpy, unfriendly (Boediman, 2005). Another study describes that nurses do not responsive since the limited number of nurses, double job, and working out of competences. Survey said that patient want kind nurses, interesting performance, responsive nurses, effective communication, and professional (Anjaryani, 2009). This is a challenge for nursing education institution since every graduate must be guaranteed having good PB.
Assessment is a process which makes sure that students achieve certain competences (Epstein, 2007). Multi Source Feedback (MSF) is one of assessment methods which involve sources as assessors. Focus on MSF is giving feedback in numeric and narrative form (Epstein, 2007). It is also said that MSF is more chosen than other methods because MSF invites feedback from multi sources which ensure the objectivity. Sources are self, peer, and patients (Epstein, 2007). Another reason is MSF is proven to improve PB of medical students, such as self confident and communication (Davis & Archer, 2005).
During the implementation of MSF, instrument used is Shieffield Peer Review Assessment (SPRAT) which is developed to assess the achievement of medical VWXGHQWV ¶ FRPSHWHQFLHV LQ FOLQLFDO VHWWLQJ $UFKHU 635$7 FRQWDLQV RI questions covering cognitive, affective and psychomotor aspects. Acher (2008) stated that SPRAT can be accepted as an instrument of assessment but it needs to be clear about the aims of the using.
Previous study stated that MSF with SPRAT as its instrument can be applied to assess PB of nursing students both in clinical and community setting however it needs modification (Asmara, 2013;Asmara, 2014). Modification is conducted since the assessment focus on PB or affective aspect while SPRAT which is developed by Archer (2008) is covering three aspects of competences namely cognitive, affective, and psychomotor aspects. There are 16 questions covering the PB or affective aspect, that is statements no 4, 5, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, and 24. Furthermore, Asmara (2013) stated that MSF can be implemented as an assessment method in clinical setting since it is more objectives by involving many assessor but it QHHGV PRGLILFDWLRQ E\ DGGLQJ VWDWHPHQWV QDPHO\ FULWLFDO WKLQNLQJ VWXGHQWV ¶ performance, and politeness. Another study stated that MSF can be applied in community setting as well. Caders are involved as a modification for colleagues, and there are statements must be added, such as critical thinking, impressive on achievement, and caring (Asmara, 2014). Then, the statements of the SPRAT modification are 21 (Asmara, 2013;Asmara 2014).
Nurse Media Journal of Nursing, 5 (2), 2015, Before implementing as an instrument, validity and reliability tests must be applied (Dharma, 2011). Asmara (2015) conducted the validity and reliability test of SPRAT modification. There were 23 statements which discipline and honest were added during the content validity. Beside content validity, construct validity was conducted. It showed that 23 statements were valid.
In order to make sure that MSF as an assessment method and SPRAT modification as its instrument are feasible and effective to be applied, a study must be conducted. It will be investigated the differences of score of PB between before and after joining internship. It also investigates the differences of score of PB among assessors. Differences of score show feasibility of method and instrument.
Every nursing student must achieve the competences of PB since this competence is needed in the working world. In order to make sure that students achieve the competencies, an assessment must be conducted. MSF as an assessment method and SPRAT modification as its instrument can be applied to assess the achievement. For that reason, it is important to evaluate the effectiveness of MSF and SPRAT modification as an assessment method and its instrument to assess PB of nursing students.
The study becomes important since it will produce an objective assessment method which involves multi sources to give feedback. It also uses valid and reliable instrument so achievement of PB competences can be standardized. Assessment involves students, peer, and mentors as assessors using SPRAT modification form.

OBJECTIVE
The objective of this study was to investigate the effectiveness of MSF in assessing QXUVLQJ VWXGHQWV ¶ SURIHVVLRQDO EHKDYLRXU 3%

METHODS
The study was an evaluation study which was conducted to evaluate the implementation of MSF as an assessment method and SPRAT modification as its instrument. It is conducted by comparing the score of PB between before and after joining internship, and comparing the scores among assessors.

Ethical approval
Faculty of Medicine Diponegoro University Ethics Committee issued the ethical approval. Before involving the research process, participants were asked to fill the form of informed consent. They knew their right to withdraw from the process, and fortunately none withdrew from the study.

Setting
The research was conducted in maternal ward. Students as respondents were asked to conduct self and peer assessment on PB while mentor were requested to give feedback RQ VWXGHQWV ¶ 3% 7KH DVVHVVPHQW ZDV FRQGXFWHG WZLFH EHIRUH DQG DIWHU MRLQLQg internship.

Subject
The participants involved in the research were 33 students and 3 mentors. Students were divided into 3 groups where each group has one mentor. A random sample was used to choose the respondents from population. The response rate was 100% meaning that 33 students and 3 mentors were involved until the process finished.

Instrument
Research instrument was SPRAT modification which contains 23 statements. Each statement was scored from 1 to 6. Previous study stated that SPRAT modification was valid and reliable instrument (Asmara, 2015). Data were collected in two phases, before and after joining internship in the maternity nursing stage. Length of internship is 4 weeks. Lally (2009) said that behavior changing can be done from 18 ± 365 days. Each group of assessors (self, peer, and mentor) was asked to give score in each phase.

Data analysis
Comparison study was chosen as a method in this study since the aim of the research was to investigate the effectiveness of MSF by comparing the scores between before and after joining the internship (Fraenkel & Wallen, 2010).
There were two kind of data analysis, first was univariate analysis and second was bivariate analysis (Dharma, 2011;Field, 2009). During the univariate analysis, data were calculated to find the frequencies and mean of the scores of PB before and after joining internship. While bivariate analysis was conducted to compare the score between before and after joining internship, and to compare the scores among assessors.
After data were calculated using univariate analysis, normality test was conducted to know the normality data. Since the data were normal distributed, Paired sample T-test or dependent sample T test were used to compare the data between before and after joining internship, and to compare the data among assessors.

RESULT
The result was served based on the data analysis, and was described on the tables. Table  1 described the frequency of scores before joining internship. The lowest score was peer assessment namely 105.82.   .000 .000 .000  Table 4 showed that there was no difference of scores before joining internship among assessors, and there was a difference of scores after joining internship among assessors ZLWK .

DISCUSSION
Result showed that there were differences of mean of scores between before and after joining internship. It described increase in scores where scores before joining internship were higher than it after joining internship. However mentor assessment showed the lowest scores in difference between before and after joining internship while the difference of scores before and after joining internship in self and peer assessment was not as high as mentor assessment.
Explanation of the result was that students have responsibility to evaluate self performance in the self assessment. Students can utilize chance to have internship in clinical setting to improve PB so that the scores of PB after joining internship was higher. Van Mook, et.al, (2009b) stated that self assessment will help student to conduct self reflection which will help them to compose the better learning plan. It is important Nurse Media Journal of Nursing, 5 (2), 2015, to involve other resources or assessment methods as triangulation of self assessment which make an assessment more objective (Epstein, 2007).
Another assessment method which was used in MSF method was peer assessment. This method invited peers to give feedback for students. Van Mook, et.al (2009b) stated that peer assessment can be applied when peer and students are in one level and interact each other. Peer assessment is important to be applied since students get positive IHHGEDFN IURP WKHLU IULHQGV DQG FDQ EH XVHG WR LPSURYH VWXGHQWV ¶ 3% The third assessment method in this research was assessment from mentor or clinical instructor. Van Mook, et.al, (2009b) stated that it was important to distinguish roles between instructor and assessor. Halo and horn effect rose up when mentor cannot distinguish it. Halo effect was occurred when mentor gave a good score because a student showed a good performance in another competence while horn effect was reverse. It was happen when mentor gave a less score since the student showed a bad performance in another competence (Van Mook, et.al, 2009a;Wood, et.al. 2006). Instead of distinction of the roles, rubric was helpful to make mentor assess objectively. It was because rubric gave explanation for each statement or question. Rubric is certain criteria that must be fulfilled by students to achieve the best result of assessment (Andrade & Du, 2005).
There were no differences of scores before joining internship among assessors and there were differences of scores after joining internship among assessors with . DQG UHVSHFWLYHO\ ([SODQDWLRQ ZKLFK OLHG RQ WKLV ZDV VWXGHQWV ¶ SHUIRUPDQFH %HIRUH joining the internship, students performed the PB as they had while after joining the internship, they performed better PB since they got feedback that be used to improve their PB. Berk (2009) stated that feedback which was given using MSF and SPRAT modification as its instrument gave benefit for student to improve the PB as it was a formative assessment.
The study introduced two kind of feedback, namely formative and summative feedback. Formative feedback was an objective and constructive statement which was served to LPSURYH VWXGHQWV ¶ SHUIRUPDQFH ZKLOH VXPPDWLYH IHHGEDFN ZKLFK ZDV KHOG LQ WKH HQG RI learning process to decide whether students passed and competent or not (Davis, et.al., 2009). In this case, formative feedback was given before joining the internship while summative feedback was given after joining the internship or on the week 4 of internship.
Considering the importance of feedback to improve VWXGHQWV ¶ 3% 06) ZDV DSSOLHG E\ serving feedback from various sources (multi sources). Epstein (2007) stated that MSF can involve nurses, physicians, patients, and students as sources. Furthermore Lynch, et.al (2004) in Hodges, et.al (2011) said that it is better to involve multi sources or DVVHVVRU LQ JLYLQJ IHHGEDFN RQ HYDOXDWLRQ RI VWXGHQWV ¶ 3% 7KH VDPH LGHD FDPH IURP Malling, et.al (2009) who stated that MSF was an effective method to assess competences of management and leadership.
Nurse Media Journal of Nursing, 5 (2), 2015, The study showed that MSF was a good method to evaluate PB of nursing students since it was showed the differences of scores between before and after joining internship, and differences of scores among assessors. Asmara (2013;2014) stated that MSF was an objective method since it involved multi sources in giving feedback. Furthermore, SPRAT modification which was used in this study was a valid and reliable instrument so it can be used as an instrument of assessment (Asmara, 2015).

CONCLUSION
Above all, MSF and SPRAT modification as its instrument were an effective method and instrument to assess PB of nursing students. In order to develop PB of nursing students it needs policy in the institution that MSF will be used as a method and SPRAT modification as its instrument to assess PB of nursing students. Furthermore, it needs to develop the evaluation items of PB based on nursing competences.