Sensitivity of test items to teaching quality
Alexander Naumann, DIPF, Germany ; Svenja Rieser, University of Wuppertal (BUWI), Germany ; Stephanie Musow, Jan Hochweber, University of Teacher Education St. Gallen (PHSG), Switzerland ; Johannes Hartig, DIPF, Germany
Learning and Instruction Volume 60, Number 1, ISSN 0959-4752 Publisher: Elsevier Ltd
Instructional sensitivity is the psychometric capacity of tests or single items of capturing effects of classroom instruction. Yet, current item sensitivity measures’ relationship to (a) actual instruction and (b) overall test sensitivity is rather unclear. The present study aims at closing these gaps by investigating test and item sensitivity to teaching quality, reanalyzing data from a quasi-experimental intervention study in primary school science education (1026 students, 53 classes, Mage = 8.79 years, SDage = 0.49, 50% female). We examine (a) the correlation of item sensitivity measures and the potential for cognitive activation in class and (b) consequences for test score interpretation when assembling tests from items varying in their degree of sensitivity to cognitive activation. Our study (a) provides validity evidence that item sensitivity measures may be related to actual classroom instruction and (b) points out that inferences on teaching drawn from test scores may vary due to test composition.
Naumann, A., Rieser, S., Musow, S., Hochweber, J. & Hartig, J. (2019). Sensitivity of test items to teaching quality. Learning and Instruction, 60(1), 41-53. Elsevier Ltd.