You are here:

A Test Instrument Calibration Study: The Rasch Model Approach to Investigate Web-mediated Instructional Outcomes
PROCEEDINGS

, Universiti Tun Hussein Onn Malaysia, Malaysia ; , RMIT University, Australia

Global Learn, in Penang, Malaysia ISBN 978-1-880094-79-2 Publisher: Association for the Advancement of Computing in Education (AACE)

Full Text

Abstract

Measuring cognitive performance is a complex process, which is made worse, by the number of mitigating variables. Consequently researchers must carefully calibrate their assessment instruments to ensure they test the intended learning outcomes. The purpose of this paper is to explain a methodology to determine test accuracy in an investigation of Web-mediated instructional outcomes. There is little evidence in the literature on human performance measurement when using educational ICT tools online. Previous research reveals there are interactive effects of paper-based instruction, within an individual’s cognitive style construct. Whether these effects also relate to a Web-mediated context remains unclear. We describe a doctoral research study underway, which uses a quasi-experimental research design, involving: an exploratory study, the validation and reliability testing of the assessment instrumentation to prepare for the main experiment. The data were analysed using the Quest Interactive Test Analysis System that is based on the Rasch probabilistic measurement technique.

Citation

Mohamad, M. & McKay, E. (2010). A Test Instrument Calibration Study: The Rasch Model Approach to Investigate Web-mediated Instructional Outcomes. In Z. Abas, I. Jung & J. Luca (Eds.), Proceedings of Global Learn Asia Pacific 2010--Global Conference on Learning and Technology (pp. 4253-4262). Penang, Malaysia: Association for the Advancement of Computing in Education (AACE). Retrieved August 10, 2024 from .

© 2010 Association for the Advancement of Computing in Education (AACE)

References

View References & Citations Map
  1. Adams, R.J., & Khoo, S.T. (1996). QUEST:The interactive test analysis system (Vol. 2.1). Melbourne: Australian Council for Educational Research.
  2. Akdemir, O., & Koszalka, T.A. (2008). Investigating the relationships among instructional strategies and learning styles in online environments. Computers& Education, 50, 1451-1461.
  3. Atkinson, C., & Mayer, R.E. (2004). Five ways to reduce PowerPoint overload. 15. Available from http://www.indezine.com/stuff/atkinsonmaye.pdf
  4. Bagley, C.A. (1990). Structured Versus Discovery Instructional Formats for Improving Concept Acquisition by DomainExperienced and Domain-Novice Adult Learners., University of Minnesota, Minnesota.
  5. Bond, T., & Fox, C.M. (2007). Applying the Rasch Model: Fundamental Measurement in the Human Sciences (2 ed.). London: Lawrence Erlbaum.
  6. Chen, S.Y., & Liu, X. (2008). An integrated approach for modeling learning patterns of students in Web-based instruction: A cognitive style perspective. ACM Transactions on Computer-Human Interaction, 15(1), 28.
  7. Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.
  8. Cronbach, L.J., & Snow, R.E. (1977). Aptitudes and instructional methods: a handbook for research on interactions New York Irvington Publishers
  9. Embretson, S.E. (1996). Item Response Theory Models and Spurious Interaction Effects in Factorial ANOVA Designs. Applied Psychological Measurement, 20(3), 201-212.
  10. Gagne, R.M. (1985). The Conditions of Learning and Theory of Instruction (4 ed.). New York Holt, Rinehart and Winston.
  11. Graff, M. (2003). Cognitive Style and Attitudes Towards Using Online Learning and Assessment Methods. Electronic Journal of e-Learning, 1(1), 21-28.
  12. Izard, J.F. (2004). Best practice in assessment for learning. Paper presented at the Third Conference of the Association of Commonwealth Examinations and Accreditation Bodies on Redefining the roles of educational assessment, March 812, Nadi, Fiji: South Pacific Voard for Educational Assessment.
  13. Kalyuga, S. (2007). Expertise Reversal Effects and Its Implications for Learner-Toilored Instruction. Educational Psychology Review, 19, 509-539.
  14. Kyllonen, P.C., & Lajoie, S.P. (2003). Reassessing Aptitude: Introduction to a Special Issue in Honor of Richard E. Snow. Educational Psychologist, 38(2), 79-83.
  15. Leung, H.K.N. (2003). Evaluating the Effectiveness of e-Learning Computer Science Education, 13(2), 123-136
  16. Massa, L.J., & Mayer, R.E. (2006). Testing the ATI hypothesis: Should multimedia instruction accommodate verbalizervisualizer cognitive style? Learning and Individual Differences, 16, 321-335.
  17. Masters, G.N. (1984). Constructing an Item Bank Using Partial Credit Scoring. Journal of Educational Measurement, 21(1), 1932.
  18. Mayer, R.E. (2003). The Promise of Multimedia Learning: Using the Same Instructional Design Method Across Different Media. Learning and Instruction, 13(2), 125-139.
  19. Mayer, R.E. (2009). Multimedia Learning (2 ed.). Cambridge, U.K: Cambridge University Press. McKay, E. (2000a). Instructional Strategies Integrating Cognitive Style Construct: A Meta-Knowledge Processing ModelContextual components that facilitate spatial/logical task performance. Doctoral Dissertation (Comp. Sci. And IS). Available from http://tux.lib.deakin.edu.au/adt-VDU/public/adt-VDU20061011.122556/ Deakin University, Australia.
  20. McKay, E. (2007). Planning effective HCI to enhance access to educational applications. Universal Access in the Information Society, 6(1), 77-85.
  21. Myers, I.B., & McCaulley, M.H. (1985). Manual: A guide to the Development and Use of the Myers-Briggs Type Indicator (2 ed.). California: Consulting Psychologists.
  22. Rasch, G. (1960). Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen: Danish Institute for Educational Research.
  23. Riding, R.J., & Cheema, I. (1991). Cognitive styles-an overview and integration. Educational Psychology: An International Journal of Experimental Educational Psychology, 11(3-4), 193-215.
  24. Riding, R.J., & Grimley, M. (1999). Cognitive Style, Gender and Learning from Multi-Media Materials in 11-year-old Children. British Journal of Educational Technology, 30(1), 43-56.
  25. Riding, R.J., & Rayner, S.G. (1998). Cognitive Styles and Learning Strategies Understanding Style Differences in Learning and Behaviour. London: David Fulton
  26. Riding, R.J., & Sadler-Smith, E. (1992). Type of Instructional Material, Cognitive Style and Learning Performance Educational Studies, 18(3), 323-340.
  27. Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation.. Learning and Instruction, 13(2), 141-156.
  28. Siemon, D., Izard, J., Breed, M., & Virgona, J. (2006). The Derivation of a Learning Assessment Framework for Multiplicative Thinking. Paper presented at the Conference of the International Group for the Psychology of Mathematics education, Prague.
  29. Sloan, D., Nelson, B., & Sloan, M. (2007). How Should Inclusivity Influence Teaching of ICT Design? ACM SIGCSE Bulletin 39(3), 307-308.
  30. Watson, J., & Callingham, R. (2003). Statistical Literacy: A Complex Hierarchical Construct. Statistics Education Research Journal, 2(2), 3-46.
  31. Wright, B.D., & Masters, G.N. (1982). Rating Scale Analysis. Chicago: Mesa Press.

These references have been extracted automatically and may have some errors. Signed in users can suggest corrections to these mistakes.

Suggest Corrections to References

Pages You've Visited

Abstracts