Validity and Reliability of Peer Assessments with a Missing Data Estimation Technique
Kwangsu Cho, Christian Schunn, University of Pittsburgh, United States
EdMedia + Innovate Learning, in Honolulu, Hawaii, USA ISBN 978-1-880094-48-8 Publisher: Association for the Advancement of Computing in Education (AACE), Waynesville, NC
The validity and reliability of peer assessment are under debate because previous research did not consider both of the critical indices. Therefore, this paper examines the validity and reliability with the data collected through SWoRD (Scaffolded Writing and Rewriting in the Discipline; Cho & Schunn, 2003), a reciprocal peer writing review system, from a graduate class. Additionally, this paper also introduces a method of estimating missing data which often are problematic in computing those indices. It was resulted that the overall validity of peer assessment is moderate, but validities in subcomponents are low. However, peer assessments are very reliable overall as well as in its subcomponents. Finally, this paper ends by introducing a technique to estimate the number of evaluators to achieve a certain level of reliability.
Cho, K. & Schunn, C. (2003). Validity and Reliability of Peer Assessments with a Missing Data Estimation Technique. In D. Lassner & C. McNaught (Eds.), Proceedings of ED-MEDIA 2003--World Conference on Educational Multimedia, Hypermedia & Telecommunications (pp. 1511-1514). Honolulu, Hawaii, USA: Association for the Advancement of Computing in Education (AACE).
© 2003 Association for the Advancement of Computing in Education (AACE)