Document Type


Publication Date

Spring 2021


Eberly College of Arts and Sciences




This study investigated how four assessors implemented an oral assessment when a little direction was provided to the assessors. We focused on the interrater reliability of oral grading and the types of follow-up questions asked. We observed three interaction types during oral assessment: no follow-up, follow-up related to the problem, and follow-up not directly related to the problem. As expected, assessors differed on several aspects of the assessment, including grading standards and follow-up experiences for students. Our findings revealed interrater reliability of oral grading with three-point scale was excellent, but not so much when six-point scale was implemented. Moreover, we observed four key roles of follow-up during an oral assessment in mathematics: to seek details of the thought process, to fill the gap in a student’s knowledge, to provide the student access to the problems, and to foster deeper learning.

Rubrics.pdf (306 kB)
Rubrics used by all assessors