고려대학교

초록/요약

When we analyze performance assessment result data assessed by single rater on a large scale, the school effect and rater effect should be considered. The purpose of the present study is to estimate the amount of variation in individual scores attributable to the differences between medical schools and between standardized patients (SPs) (raters), and to examine how the characteristics of schools and SPs might explain such variation. Students were grouped by the school attended and by the SP who evaluated the students’ performance and acted as a patient in each task (case). Each student was cross-classified with a school and a SP. The cross-classified model is a useful method for analyzing the combined effect (school and rater effect) in this type of hierarchical data. The results by this model showed that students’ performance outcomes were significantly related to the differences between schools and SPs. Each task had four content domains: History Taking, Physical Examination, Clinical Courtesy, and Patient-Physician Interaction. Among them, the school effect on student scores with regard to Physical Examination items and the SP effect with regard to History Taking and Patient-Physician Interaction items were significant. Some predictors, for example, mean scores of schools and SPs, explained the variation of student scores significantly. For reliable performance evaluation of the medical students, SPs should be carefully trained in the Patient-Physician Interaction domain.