You are here:

Assessing Online Dialogue in Higher Education

, Bishop's University, Canada ; , Concordia University, Canada ; , Mount St. Vincent University, Canada ; , Bishop's University, Canada

E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, in Orlando, Florida, USA ISBN 978-1-880094-83-9 Publisher: Association for the Advancement of Computing in Education (AACE), San Diego, CA


The aim of this study is to develop valid researcher and instructor approaches to successfully assess online dialogue. Two content analysis approaches measuring critical thinking and interactivity were applied to representative online groups of 3-4 graduate education students engaged in 2 two-week long online activities. Two representative successful groups and 2 less successful groups were chosen based on the instructor’s marks. The interactivity measure did not distinguish between the more and less successful groups but the critical thinking measure did. A revised content analysis is being applied by two coders to two online activities in 3 classes of education undergraduate students (n=82). The instructor is assessing the students’ online dialogue using a holistic rubric we have developed, as are the coders. The results from the content analysis and the holistic marks will be compared, in a search for validity, and inter-rater reliability will be explored for all measures.


Bures, E., Abrami, P., Barclay, A. & Bures, E. (2010). Assessing Online Dialogue in Higher Education. In J. Sanchez & K. Zhang (Eds.), Proceedings of E-Learn 2010--World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (pp. 438-448). Orlando, Florida, USA: Association for the Advancement of Computing in Education (AACE). Retrieved February 23, 2019 from .

View References & Citations Map


  1. Abrami, P.C. & Bures, E.M. (1996). Computer-supported collaborative learning and distance education. The American Journal of Distance Education, 10(2), 37-42.
  2. Barros, B. & Verdejo, M.F. (2000). Analysing student interaction processes in order to improve collaboration. The DEGREE approach. International Journal of Artificial Intelligence in Education, 11, 221-241.
  3. Blake, C. & Rapanotti, L. (2001). Mapping interactions in a computer conferencing environment. In Proceedings from Euro 2001, Matriche, the Netherlands.
  4. Brown, A. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141-178.
  5. Bullen, M. (1998). Participation and critical thinking in online university distance education. Journal of Distance Education/Revue de l'enseignement à distance: 13(2).[iuicode:]
  6. Bruffee, K. (1993). Collaborative learning: Higher Education, interdependence, and the authority of knowledge. Baltimore, MD: Johns Hopkins University Press.
  7. Cameron, C. (1995). HipBone Games. Seattle, WA: Rheingold Associates. Available: De Vries, E., Lund, K, & Baker, M. (2002). Computer-mediated epistemic dialogue: Explanation and argumentation as vehicles for understanding scientific notions. The Journal of the Learning Sciences, 11(1), 63-103.
  8. De Wever, B., Schellens T., Valcke, M., & Van Keer H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers and Education, 46, 6-28.
  9. Dewey, J. (1933). How we think. Boston, Mass: Heath.
  10. Facione, P.A. (1990). Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. Research Findings and Recommendations. American Philosophical Association, Washington, DC: American Psychological Associations' Board of Educational Affairs.
  11. Facione, P.A. And Facione, N.C. (1994). Holistic critical thinking scoring rubric. California Academic Press.
  12. Fahy, P.J. (2001). Addressing some common problems in transcript analysis. The International Review of Research in Open and Distance Learning, 1(2). Available at: #Fahy
  13. Fahy, P.J., Crawford, G., Ally, M., Cookson, P., Keller, V., & Prosser, F. (2000). The development and testing of a tool for analysis of computer-mediated conferencing transcripts. The Alberta Journal of Educational Research, Vol. XLVI(1), 85-88.
  14. Feenberg, A. (1991). Social factors in computer mediated communication. In L. Harasim (Ed.), On-Line education: Perspectives on a new medium (pp. 67-97). New York: Praeger.
  15. Garrison, D.R. (1991). Critical thinking and adult education: A conceptual model for developing critical thinking in adult learners. International Journal of Lifelong-Education, 10(4), 287-303.
  16. Getzels, J. & Csikszentmihalyi, M. (1976). The creative vision: A longitudinal study of problem finding in art. New York: Wiley.
  17. Gunawardena, C., Lowe, C. & Anderson, T. (1997). Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing. Journal of Educational Computing Research, 17(4), 397-431.
  18. Hara, N., Bonk, C.J. And Angeli, C. (2000). Content analysis of an on-line discussion in an applied educational psychology course. Instructional Science, 28, 115-152.
  19. Harasim, L., Hiltz, S.R., Teles, L. & Turoff, M. (1995). Learning networks: A Field guide to teaching and learning online (pp. 117-136). Cambridge, MA: MIT Press.
  20. Henri, F. (1991). Computer conferencing and content analysis. In A. Kaye (Ed.), Collaborative learning through computer conferencing, Berlin: Springer-Verlag.
  21. Herrington, J. And Oliver, R. (1997). Multimedia, magic and the way students respond to a situated learning environment. Australian Journal of Educational Technology, 13(2), 127-143.
  22. Hewitt, J. (2004). An Exploration of community in a knowledge forum classroom: An Activity System Analysis. In S. Barab, R. Kling & J. Gray (Eds.) Designing Virtual Communities in the Service of Learning (pp. 210-238), Cambridge:
  23. King, A. (1994). Guiding knowledge construction in the classroom: Effects of teaching children how to question and explain. American Educational Research Journal, 21(2), 338-368.
  24. Lee, E., Chan, C. & Aalst, J. (2006). Students assessing their own collaborative knowledge building. Journal of Computer-supported collaborative learning, 1, 57-87.
  25. Mason, R. (1991). Methodologies for evaluating applications of computer conferencing, In A.R. Kaye (Ed.) Collaborative learning through Computer Conferencing, Heidelberg: Springer-Verlag.
  26. McAlister, S., Ravenscroft, A. & Scalone, E. (2004). Combining interaction and context design to support collaborative argumentation using synchronous CMC. Journal of Computer Assisted Learning Special Issue: Context, collaboration, computers and learning, 20(3), 194-204.
  27. Newman, R., Webb, B. & Cochran, C. (1995). A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Interpersonal Computing and Technology, 3(2), 56-77.
  28. O'Donnell, A.M. & King, A. (1999). Cognitive perspectives on peer learning. Mahwah, NJ: Erlbaum.
  29. Ohlsson, S. (1995). Learning to do and learning to understand: A lesson and a challenge for cognitive modeling. In P. Reimann& H. Spada (Eds.), Learning in humans and machines (pp. 37-62). Oxford, England: Elsevier.
  30. Reeves, T. (2000). Alternative Assessment Approaches for Online Learning Environments in Higher Education. Journal of Educational Computing Research, 23(1), 101-111.
  31. Reeves, T.C., Herrington, J., & Oliver, R. (2005). Design research: A socially responsible approach to instructional technology research in higher education. Journal of Computing in Higher Education, 16(2), 97-116.
  32. Rourke, L., Anderson, T., Garrison, D.R. & Archer, W. (2001). Methodological issues in analyzing text-based computer conferencing transcripts. International Journal of Artificial Intelligence in Education, 12, 8-22.
  33. Scardamalia, M. & Bereiter, C. (1994). Computer support for knowledge building communities. Learning Sciences, 3(3), 265-283.
  34. Schrire, S. (2004). Interaction and cognition in asynchronous computer conferencing. Instructional Science, 32, 475-502.
  35. Sproull, L. & Kiesler, S. (1991). Connections: new ways of working in the net-worked organization. Cambridge, MA: MIT Press.
  36. Vygotsky, L. (1978). Mind in society. Cambridge, MA: The MIT Press.
  37. Webb, N. (1989). Peer interaction and learning in small groups. International Journal of Educational Research, 13(1), 2129.
  38. Xin, C. & Feenberg, A. (2006). Pedagogy in Cyberspace: The Dynamics of Online Discussion. Journal of Distance Education 21(2), 1-25.

These references have been extracted automatically and may have some errors. If you see a mistake in the references above, please contact

View References & Citations Map

Cited By

  1. Discussion Forum Efficacy in an Online Course: Dialogue vs Interaction

    Emmanuelle Bernardin, Audencia Nantes, France

    E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2013 (Oct 21, 2013) pp. 264–273

These links are based on references which have been extracted automatically and may have some errors. If you see a mistake, please contact