What Should Scoring Scale Descriptors Look Like?
What Should Scoring Scale Descriptors Look Like?
- 주제(키워드) 제2 외국어 평가 , 영어작문평가 , 영어직접평가 , 영어 채점 기준표 , 채점표 기술내용 , 채점자 신뢰도 , 점수 신뢰도제2 외국어 평가 , 영어작문평가 , 영어직접평가 , 영어 채점 기준표 , 채점표 기술내용 , 채점자 신뢰도 , 점수 신뢰도 , L2 language testing , ESL/EFL writing assessment , ESL/EFL direct assessment , ESL/EFL scoring rubric/scale , scoring scale descriptors , rater reliability , score reliability
- 발행기관 한국응용언어학회
- 발행년도 2012
- 총서유형 Journal
- UCI G704-000491.2012.28.1.010
- KCI ID ART001646519
- 본문언어 영어
초록/요약
The present study is aimed at investigating whether the specification of scoring scale descriptors affects the reliability of Korean EFL raters, and to what degree this occurs. A comparison of two scoring scales with differing levels of specification was undertaken. 32 Korean EFL secondary school teachers rated four writing samples using the two scoring scales and responded to a questionnaire asking about their perceptions of the two scales. The data were analyzed using a multi-faceted Rasch analysis. It was found that more specific scales improved inter-rater reliability. More specific scales also contributed to preventing biased rating against criteria. However, they failed to narrow the gap between raters in terms of degree of severity and to increase individual raters’ consistency in scoring different writing samples. An analysis of the raters’ responses to the survey questionnaire revealed that the more specific scoring scales were preferred as it was more useful in deciding on a score. Conversely, they were criticized for being too complex and time-consuming to apply.
more