Author ORCID Identifier

https://orcid.org/0000-0001-7323-7250

Semester

Summer

Date of Graduation

2026

Document Type

Dissertation

Degree Type

PhD

College

Eberly College of Arts and Sciences

Department

Psychology

Committee Chair

Claire St. Peter

Committee Co-Chair

Kathryn Kestner

Committee Member

Kathleen Morrison

Committee Member

Ryan Best

Committee Member

Stephanie Jones

Abstract

Accurate observational data are essential for drawing correct conclusions about behavioral phenomena and ensuring the replicability of research findings. Accuracy is the extent to which an observer’s recorded responses correspond with true values (i.e., the actual occurrence of behavioral events). Correspondence is quantified using one of several percentage-based algorithms. However, these accuracy estimates may be influenced by characteristics of the data, such as response rate and response distribution, as well as the algorithm used to calculate them. The purpose of this dissertation was to evaluate how response rate, response distribution, and the algorithm used to evaluate accuracy influences estimates. Across both experiments, participants collected continuous data from videos of mock teacher-student interactions using computerized data-collection software, and their records were compared with true values. Experiment 1 assessed the effects of response rate on accuracy estimates across a continuous spectrum of rates, four algorithms, and a single response topography. A mixed effects regression with a polynomial term found that response rate differentially influenced each algorithm. Experiment 2 evaluated the effects of response distribution on accuracy estimates across eight algorithms, including occurrence and nonoccurrence variants, and identified the types of data-collection errors detected by each algorithm. An ANOVA identified a significant interaction between distribution and algorithm and significant main effects for distribution and algorithm. Collectively, the findings demonstrate that accuracy estimates are not solely a reflection of an observer’s skills, but are influenced by response rate, response distribution, and the algorithm used to calculate them. Researchers and practitioners should interpret accuracy estimates in the context of both the characteristics of the data and the algorithms used to calculate them to meaningfully evaluate the believability of observational data.

Download

Share

COinS