Comparison of item analysis results of Korean Medical Licensing Examination according to classical test theory and item response theory

The results of the 64th and 65th Korean Medical Licensing Examination were analyzed according to the classical test theory and item response theory in order to know the possibility of applying item response theory to item analys and to suggest its applicability to computerized adaptive test. The correlation coefficiency of difficulty index, discriminating index and ability parameter between two kinds of analysis were got using computer programs such as Analyst 4.0, Bilog and Xcalibre. Correlation coefficiencies of difficulty index were equal to or more than 0.75; those of discriminating index were between 0.023 and 0.753; those of ability parameter were equal to or more than 0.90. Those results suggested that the item analysis according to item response theory showed the comparable results with that according to classical test theory except discriminating index. Since the ability parameter is most widely used in the criteria-reference test, the high correlation between ability parameter and total score can provide the validity of computerized adaptive test utilizing item response theory.

a: Difference between unique value of the frist factor and that of second factor.b: Difference between unique value of the second factor and that of third factor.b: Difference between unique value of the third factor and that of the fourth factor.

Table 1 .
Unique value by factor analysis of the 65th Korean Medical Licensing Examination

Table 2 .
Mean and standard deviation (SD) of difficulty and discriminating index according to classical test theory

Table 4 .
Correlation coefficiency of difficulty index values of the 64th and 65th National Medical Licensing Examination analyzed by 4 kinds of programs

Table 7 .
Correlation coefficiency of total score and ability parameter values of the 64th and 65th National Medical Licensing Examination analyzed by 2 kinds of programs for item response theory