You are here:

The reality of assessing ‘authentic’ electronic portfolios: Can electronic portfolios serve as a form of standardized assessment to measure literacy and self-regulated learning at the elementary level? / L’évaluation d’e-portfolio «authentiques»
ARTICLE

, Bishop's University ; , ,

CJLT Volume 39, Number 4, ISSN 1499-6677 e-ISSN 1499-6677 Publisher: Canadian Network for Innovation in Education

Abstract

This study explores electronic portfolios and their potential to assess student literacy and self-regulated learning in elementary-aged children. Assessment tools were developed and include a holistic rubric that assigns a mark from 1 to 5 to self-regulated learning (SRL) and a mark to literacy, and an analytical rubric measuring multiple sub-scales of SRL and literacy. Participants in grades 4, 5 and 6 across two years created electronic portfolios, with n=369 volunteers. Some classes were excluded from statistical analyses in the first year due to low implementation and some individuals were excluded in both years, leaving n=251 included in analyses. All portfolios were coded by two coders, and the inter-rater reliability explored. During the first year Cohen’s kappa ranged from 0.70 to 0.79 for literacy and SRL overall, but some sub-scales were unacceptably weak. The second year showed improvement in Cohen’s kappa overall and especially for the sub-scales, reflecting improved implementation of the portfolios and use of the assessment tools. Validity was explored by comparing the relationship of portfolio scores to other measures, including the government scores on the open-response literacy questions for the Canadian Achievement Tests (version 4), the scores we assigned to the CAT-4s using our assessment tools, and scores on the Student Learning Strategies Questionnaire (SLSQ) measuring SRL. The portfolio literacy scores correlated (p

Citation

Bures, E., Barclay, A., Abrami, P. & Meyer, E. (2013). The reality of assessing ‘authentic’ electronic portfolios: Can electronic portfolios serve as a form of standardized assessment to measure literacy and self-regulated learning at the elementary level? / L’évaluation d’e-portfolio «authentiques». Canadian Journal of Learning and Technology / La revue canadienne de l’apprentissage et de la technologie, 39(4),. Canadian Network for Innovation in Education. Retrieved March 22, 2019 from .

View References & Citations Map

References

  1. Abrami, P.C. & Barrett, H. (2005). Directions for research and development on electronic portfolios. Canadian Journal of Learning and Technology, 31(3), 1-15. Available: http://www.cjlt.ca/content/vol31.3/abrami.html
  2. Abrami, P.C., Bures, E.M., Idan, E., Meyer, E., & Venkatesh, V. (2013). Electronic Portfolio Encouraging Active and Reflective Learning (ePEARL). In R. Azevedo& V. Aleven (Eds.), International handbook of metacognition and learning technologies (pp. 503-515).
  3. Azevedo, R., Moos, D.C., Johnson, A.M., & Chauncey, A. (2010). Measuring cognitive and meta-cognitive regulatory processes during hypermedia: Learning and challenges. Educational Psychologist, 45(4), 210-223.
  4. Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28, 117-148.
  5. Barrett, H. (2007). Research electronic portfolios and learner engagement: The REFLECT Initiative. Journal of Adolescent& Adult Literacy, 50(6), 436-449.
  6. Broadfoot, P., & Black, P. (2004). Redefining assessment? The first ten years of assessment in education. Assessment in education: Principles, policy and practice, 11(1), 7-26.
  7. Bures, E., Abrami, P. & Bentley, C. (2007). Assessing electronic portfolios--Now that we have them, what can we do with them? In G. Richards (Ed.), Proceedings of World Conference on E-Learning inCorporate, Government, Healthcare, and Higher Education 2007 in Quebec (pp. 7030-7038). Chesapeake, VA: AACE.
  8. Chambers, S.M. & Wickersham, L.E. (2007). The electronic portfolios journey: A year later. Education, 127(3), 351-360.
  9. Foote, C.J. & Vermette, P.J. (2001). Teaching portfolio 101: Implementing the teaching portfolio in introductory courses. Journal of Instructional Psychology, 28(1), 31-38.
  10. Gardner, H. (1990). On leadership. New York: The Free Press. The reality of assessing
  11. Geertz, C. (1973). The Interpretation of cultures: Selected essays. New York: Basic Books.
  12. Herman, J.L., Gearhart, M., & Baker, E.L. (1993). Assessing writing portfolios: Issues in the validity and meaning of scores. Educational Assessment, 1(3), 201-224.
  13. Herman, J.L, Gearhart, M., & Aschbacher, P.R. (1996). Portfolios for classroom assessment: Design and implementation issues. In R. Calffee & P. Perfumo (Eds.), Writing portfolios in the classroom: Policy and practice, promise and peril (pp. 27-59). Hillsdale, NJ: Lawrence
  14. Hill, J., Song, L., & West, R. (2009). Social learning theory and web-based learning environments: A review of research and discussion of implications. The American Journal of Distance Education, 23, 88–103.
  15. Koretz, D. (1998). Large-scale portfolio assessments in the US: Evidence pertaining to the quality of measurement. Assessment in Education, 5(3), 309-334.
  16. Knighton, T., Brochu, P., & Gluszynski, T. (2010). Measuring up: Canadian results of the OECD PISA study-The performance of Canada's youth in reading, mathematics and science; 2009 first results for Canadians aged 15 (Catalogue No. 81-590-X). Ottawa: Human Resources and Skills Development Canada, Council of Ministers of Education, Canada and Statistics Canada.
  17. Lankshear, C. & Knobel, M. (2006). New literacies: Everyday practices and classroom learning. New York: McGraw-Hill.
  18. Lave, J. & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York: Cambridge University Press.
  19. Love, T., & Cooper, T. (2004). Designing online information systems for portfolio-based assessment: Design criteria and heuristics. Journal of Information Technology Education, 3, 65-80.
  20. Meyer, E., Abrami, P.C., Wade, A., Aslan, O. & Deault, L. (2010). Improving literacy and metacognition with electronic portfolios: Teaching and learning with ePEARL. Computers& Education, 55(1), 84-91. Doi:10.1016/J.compedu.2009.12.005The reality of assessing
  21. Newmann, F., Brandt, R., & Wiggins, G. (1998). An exchange of views on "Semantics, psychometrics, and assessment reform: A close look at 'Authentic' assessments." Educational Researcher, 27(6), 19-21.
  22. OECD. (2010). PISA 2009 Results: What Students Know and Can Do– Student Performance in Reading, Mathematics and Science (Volume I).
  23. Perry, N.E. (1998). Young children's self-regulated learning and contexts that support it. Journal of Educational Psychology, 90, 715–729.
  24. Pintrich, P.R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 95(4), 667– 686.
  25. Reeves, T. (2000). Alternative assessment approaches for online learning environments in higher education. Journal of Educational Computing Research, 23(1), 101-111.
  26. Reeves, T., Herrington, J., Oliver, R. & Woo, Y. (2004). Designing authentic activities in webbased courses. Journal of Computing in Higher Education, 16(1), 3-29.
  27. Schraw, G. (2010). Measuring self-regulation in computer-based learning environments. Educational Psychologist, 34(4), 258-266.
  28. Schunk, D.H., & Zimmerman, B.J. (2006). Competence and control beliefs: Distinguishing the means and the ends. In P.A. Alexander& P.H. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 349-367). Mahwah, NJ: Lawrence Erlbaum Associates.
  29. Sharples, M., Taylor, J. & Vavoula, G. (2007). A theory of learning for the mobile age. In R. Andrews& C. Haythornthwaite (Eds.), The Sage handbook of elearning research (pp. 221247).
  30. Stiggins, R.J. (2004). New assessment belief for a new school mission. Phi Delta Kappan, 86(1), 22-27.
  31. Upitis, R., Abrami, P.C., Brook, J., Troop, M., & Varela, W. (2012). Learning to play a musical instrument with a digital portfolio tool. Journal of Instructional Pedagogies, 9, 1-15.
  32. Wade, A., & Abrami, P.C. (2005). An electronic portfolio for learning. Canadian Journal of Learning and Technology, 3(3).
  33. Wiggins, G. (1990). The case for authentic assessment. Practical Assessment, Research& Evaluation, 2(2). Retrieved from http://PAREonline.net/getvn.asp?v=2 & N=2
  34. Yancey, K. (2004). Teaching literature as reflective practice. Urbana, IL: National Council of Teachers of English.
  35. Zellers, M. & Mudrey, R. (2007). Electronic portfolios and metacognition: A phenomenological examination of the implementation of electronic portfolios from the instructors’ perspective. International Journal of Instructional Media, 34(4), 419-430.
  36. Zimmerman, B. (2000). Attaining self-regulation: A social-cognitive perspective. In M. Boekaerts & P.R. Pintrich (Eds.), Handbook of self-regulation (pp. 13-39). New York:

These references have been extracted automatically and may have some errors. If you see a mistake in the references above, please contact info@learntechlib.org.