Semester

Summer

Date of Graduation

2005

Document Type

Dissertation

Degree Type

EdD

College

College of Education and Human Services

Department

Learning Sciences and Human Development

Committee Chair

David L. McCrory.

Abstract

The purpose of this study was to determine what effect the personal characteristics of reviewers might have on their reviews. The participants in this study were nine instructional leaders who used a rubric to evaluate P-12 teachers' web-based instructional units, but still scored them differently.;The characteristics examined in this study included the reviewers' styles of teaching, personal computer use, concerns about instructional technology integration, professional positions, perceived roles, and personal criteria for what makes a good web-based instructional unit.;The study employed both quantitative and qualitative research methodologies. The quantitative instruments used were the Stages of Concern Questionnaire (SoCQ), the Principles of Adult learning Scale (PALS), and the Survey of Computer Use (SCU). The qualitative methods used were interviews of the instructional leaders and coding and compiling of comments made by the instructional leaders when they were evaluating the units.;The results of this study indicate that the reviewers who had teaching styles that were more "learner-centered" were inclined to pass more units than their "teacher-centered" counterparts. The reviewers who had the lowest concerns overall about technology passed less units than those with higher technology concerns. Reviewers who were not in P-12 teaching positions passed more units, while the reviewer whose job it was to teach technology in the public school system passed the least. In addition, the group identified as "advanced" appeared to be judged more stringently, and the group identified as "beginners" appeared to be evaluated more generously. Other factors were less clear; however, when interviewed, the reviewers mentioned various factors that may have influenced their scoring, including their perceptions of their roles and their personal feelings about the participants.;The results of this study imply that reviewers, even when using a rubric, will still incorporate their own opinions and expertise into their reviews. When choosing and assigning reviewers, characteristics that might have an effect should be kept in mind and controlled as much as possible by having at least three reviewers for each product being evaluated and by balancing groups of reviewers by assigning reviewers with similar characteristics to different groups.

Download

Share

COinS