You are here:

Using Peer Reviews in MOOCs

, Competence Center E-Commerce (CCEC), Germany ; , Fachhochschule Südwestfalen, Germany ; , Nexedi GmbH, Germany

EdMedia + Innovate Learning, in Washington, DC ISBN 978-1-939797-29-2 Publisher: Association for the Advancement of Computing in Education (AACE), Waynesville, NC


This paper examines the applicability of a peer review approach for the assessment of practical project work in MOOCs. For this, we first analyze the challenges of using peer assessments in MOOCs, then outline the results of a participant observation on four Coursera courses, and finally provide a case study that involved the design and implementation of a peer review system as part of an online course on Enterprise Resource Planning (ERP). The findings indicate that with appropriate design and implementation, a peer review approach can serve as an effective assessment solution also for practical project work.


Qian, X., Weber, P. & Wölfel, K. (2017). Using Peer Reviews in MOOCs. In J. Johnston (Ed.), Proceedings of EdMedia 2017 (pp. 74-83). Washington, DC: Association for the Advancement of Computing in Education (AACE). Retrieved January 17, 2019 from .

View References & Citations Map


  1. Admiraal, W., Huisman, B., & Pilli,O. (2015). Assessment in Massive Open Online Courses. The Electronic Journal of E-learning. Retrieved April, 2016, from
  2. Balfour, S.P. (2013). Assessing Writing in MOOCs: Automated Essay Scoring and Calibrated Peer Review™. Research& Practice in Assessment. Retrieved February, 2016, from
  3. Bloom, D. (1971). Mastery learning. New York: Holt, Rinehart& Winston.
  4. Boud, D. (1999). Peer learning and assessment. Assessment and Evaluation in Higher Education, 4, 413-426.
  5. Caplan, S. (2013, June 6). MOOCs-massive open online courses: Jumping on the bandwidth. Retrieved December 28, 2015, from
  6. Chaudhary, S., & Dey, N. (2013). Assessment in open and distance learning system (ODL): a challenge. Open Praxis, 5(3), 207-216.
  7. Cho, K., Schunn, C., & Wilson, R. (2006). Validity and reliability of scaffolded peer assessment of writing from instructor and student perspective. Journal of Educational Psychology, 4, 891-901.
  8. Duhring, J. (2013, May 10). Massive MOOC Grading Problem– Stanford HCI Group Tackles Peer Assessment. Retrieved November 30, 2015, from
  9. Friedman, T. (2012, May 16). Come the revolution. Retrieved November 26, 2015, from
  10. Gielen, S., Dochy, F., Onghena, P., & Struyven, K.S. (2011). Goals of peer assessment and their associated quality concepts. Studies in Higher Education, 6, 719-735.
  11. Graesser, A., & Mc Namera, D. (2012). Automated analysis of essays and open-ended verbal responses. In H. Cooper, P. Camic, D. Long, A. Panter, D. Rindskopf, & K. Sher, APA Handbook of research methods in psychology, 1, 307-325. Washington, DC: American Psychological Association.
  12. Grover, S., Franz, P., Schneider, E., & Pea, R. (2013). The MOOC as distributed intelligence: Dimensions of a Framework& Evaluation of MOOCs. Stanford Graduate School of Education. Stanford: LYTICS. Is the 95% MOOC dropout rate the big issue? (2013). Retrieved January 19, 2016, from
  13. Lu, Y., Warren, J., Jermaine, C., Chaudhuri, S., & Rixner, S. (2015). Grading the graders: Motivating peer graders in a MOOC. Rice University, Department of Computer Science. International WorldWide Web Conference Committee. Retrieved February 4, 2015, from
  14. Luo, H., Dutton, J., Robinson, A., & Park, J. (2014). Peer Grading in a MOOC: Reliability, validity, and perceived effects. Online Learning Journal. Retrieved February 1, 2016, from
  15. Mack, N. (2005). Module 2: Participant Observation. In Qualitative Research Methods: A Data Collector’s Field Guide,14-27. Family Health International. Retrieved February 4, 2015, from
  16. Rees, J. (2013, March 5). Peer grading can't work. Retrieved October 27, 2015, from
  17. Robinson, R. (2001). Calibrated Peer Review™: An Application To Increase Student Reading& Writing Skills. The American Biology Teacher, 63(7), 474-480.
  18. Sandeen, C. (2013). Assessment 's place in the new MOOC world. Research& Practice in Assessment, 8, 5-12.
  19. Shah, N., Bradley, J., Wainwright, M., & Ramchandran, K. (2013, December). A Case for Ordinal Peer-evaluation in MOOCs. NIPS Workshop on Data Driven Education. Retrieved December 17, 2015, from
  20. Shermis, M., Burstein, J., Higgins, D., & Zechner, K. (2010). Automated essay scoring: writing assessment and instruction. International Encyclopedia of Education, 4, 20-26.
  21. Skinner, B. (1968). The technology of teaching. New York: Appleton-Century-Crofts.
  22. Spille, D. (2012). Assessment Matters: Self-assessment and peer assessment. Teaching Development.
  23. Topping, K. (2009). Peer assessment. Theory into Practice, 1, 20-27.
  24. Van Hattum-Janssen, N., & Fernandes, J. (2012). Peer feedback: quality and quantity in large groups. Thessaloniki, Greece: JEFI 40th annual conference. Retrieved November 11, 2015 from DASHDASH

These references have been extracted automatically and may have some errors. If you see a mistake in the references above, please contact

View References & Citations Map

Cited By

  1. The City and You: An Exploration of Course Design, Learning Objectives, and Learner Outcomes and Assessments

    Richard Florida, Martin Prosperity Institute, Rotman School of Management, University of Toronto, Canada; Hedieh Najafi, Online Learning Strategies, University of Toronto, Canada; Karen King, Martin Prosperity Institute, Rotman School of Management, University of Toronto, Canada; Laurie Harrison & Will Heikoop, Online Learning Strategies, University of Toronto, Canada

    E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2018 (Oct 15, 2018) pp. 842–849

These links are based on references which have been extracted automatically and may have some errors. If you see a mistake, please contact