You are here:

Use of sound with digital text: Full of sound and fury signifying something

, University at Buffalo, United States

CITE Journal Volume 12, Number 3, ISSN 1528-5804 Publisher: Society for Information Technology & Teacher Education, Waynesville, NC USA


The purpose of this interpretive case study was to explore—through a close analysis of one class project—students’ use of audio signs and the teacher's scaffolding of the use of audio signs. Two research questions guided this study: (a) In what ways did the fifth-grade students use audio signs, specifically transitions sounds, when constructing multimodal texts with different sign systems (e.g., visual, linguistic or audio signs)?; (b) In what ways did the classroom teacher shape the specific social cultural environment for audio sign use? The findings of this study argue for professional development opportunities for teachers where they not only learn how to use various software programs but also learn the content knowledge necessary for communicating with multiple signs such as audio.


Shanahan, L. (2012). Use of sound with digital text: Full of sound and fury signifying something. Contemporary Issues in Technology and Teacher Education, 12(3), 264-285. Waynesville, NC USA: Society for Information Technology & Teacher Education. Retrieved March 25, 2019 from .

View References & Citations Map


  1. Altman, R. (1992). In sound theory/sound practice. New York, NY: Routledge.
  2. Apple Classroom of Tomorrow. (1995). Changing the conversations about teaching, learning, & Technology: A report of 10 years of ACOT research. Cupertino, CA: Apple computer. Retrieved from
  3. Bailey, N. (2006). Designing social futures: Adolescent literacies in and for new times. (Unpublished doctoral dissertation). University at Buffalo, State University of New York.
  4. Bearne, E., & Wolstencroft, H. (2007). Visual approaches to teaching writing: Multimodal literacy 5-11. Thousand Oaks, CA: Sage.
  5. Bezemer, J., & Kress, G. (2008). Writing in multimodal texts: A social semiotic account of designs for learning. Written Communication, 25, 166-195.
  6. Bigand, E. (1993). Contributions of music to research on human auditory cognition. In S. McAdams, & E. Bigand (Eds.), Thinking in sound (pp. 231-277). New York, NY: Oxford
  7. Blackstock, J., & Miller, L. (1992). The impact of new information technology on your children’s symbol weaving efforts. Computers in Education, 18, 209-221.
  8. Bridgett, R. (2010). From the shadows of film sound: Cinematic production and creative process in videogame audio: collected publications 2000-2010 [Self-published e-book].
  9. Chion, M. (1990). Audio-vision: Sound onScreen. New York, NY: Columbia University Press.
  10. Cope, B., & Kalantzis, M. (2000). Designs for social futures. In B. Cope& M. Kalantzis (Eds.), Multiliteraces: Literacy learning and the design of social futures (pp. 203234).
  11. Gall, M., & Breeze, N. (2005). Music composition lessons: The multimodal affordances of technology. Educational Review, 57(4), 415-434.
  12. Gee, J.P. (2003). What videogames have to teach us about learning and literacy. New York, NY: Palgrave MacMillian.
  13. Halliday, M.A.K. (1978) Language as social semiotic: The social interpretation of language and meaning. Sydney, Australia: Edward Arnold.
  14. Harmon, R. (1988). Film producing: Low-budget films that sell. Hollywood, CA: Samuel French Trade.
  15. Healy, A. (Ed.) (2008). Multiliteracies and diversity in education: New pedagogies for expanding landscapes. Melbourne, Australia: Oxford University Press.
  16. Hodge, R., & Kress, G. (1988). Social semiotics. Cambridge, England: Polity.
  17. Hull, G.A., & Nelson, M.E. (2005). Locating the semiotic power of multimodality. Written Communication, 22, 224-261.
  18. Jewitt, C. (2005). Multimodality, “Reading” and “Writing” for the 21st Century. Discourse, 26(3), 315-331.
  19. Jewitt, C., & Kress, G. (2003). Multimodal literacy. New York, NY: P. Lang.
  20. Kalanztis, M., & Cope, B. (2005). Learning by design. Melbourne, Australia: Victorian Schools Innovation Commission.
  21. Knobel, M., & Lankshear, C. (2007). A new literacies sampler. New York, NY: Peter Lang.
  22. Kohfeld, D.L. (1971). Simple reaction time as a function of stimulus intensity in decibels of light and sound. Journal of Experimental Psychology, 88, 251-257.
  23. Kress, G. (1998). Visual and verbal modes of representation in electronically mediated communication: the potentials of new forms of text. In I. Snyder (Ed.), Page to screen: Taking literacy into the electronic era (pp. 5-79). London, England: Routledge.
  24. Kress, G., & Van Leeuwen, T. (1996). Reading images. London, England: Routledge.
  25. Kress, G., & Van Leeuwenn, T. (2001). Multimodal discourse: The modes and media of contemporary communication. London, England: Hodder Education.
  26. Lankshear, C. & Knobel, M. (2007). Sampling “the new” in New Literacies. In M. Knobel& C. Lankshear (Eds.), A new literacies sampler (pp. 1-24). New York, NY: Peter Lang.
  27. Leander, K. (2009). Composing with old and new media: toward parallel pedagogy. In V. Carrington& M. Robinson (Eds.), Digital literacies: Social learning and classroom practices (pp. 147-164). London, England: Sage.
  28. Mahiri, J. (2006). Digital DJ-ing: Rhythms of learning in an urban school. Language Arts, 84(1), 55-62.
  29. McVee, M.B., Bailey, N.M., & Shanahan, L.E. (2008). Using digital media to interpret poetry: Spiderman meets Walt Whitman. Research in the Teaching of English, 43(2), 112-143.
  30. McVee, M.B., Bailey, N.M., & Shanahan, L.E. (2012). The (artful) deception of technology integration and the move toward a new literacies mindset. In S.M. Miller& M.B. McVee (Eds.), Multimodal composing in classrooms: Learning and teaching is digital world (pp. 13-31). New York, NY: Routledge.
  31. Merriam, S.B. (1998). Qualitative research and case study applications in education. San Francisco, CA: Jossey-Bass.
  32. Miller, S.M. (2008). Teacher learning for new times: Repurposing new multimodal literacies and digital video composing for schools. In J. Flood, S.B. Heath, & D. Lapp (Eds.) Handbook of research in teaching literacy through the communicative and visual arts. (Vol. 2; pp. 441-453). New York, NY: Lawrence Erlbaum.
  33. Miller, S.M., & Borowicz. (2006). Why multimodal literacies? Designing digital bridges to 21st century teaching and learning. Buffalo, NY: GSE Publications.
  34. Miller, S.M., & McVee, M (Eds.) (2012). Multimodal composing in classrooms: Learning and teaching in the digital world. New York, NY: Routledge.
  35. Murray, L. (2010). Authenticity and realism in documentary sound. The Soundtrack, 3(2), 131-137.
  36. Myers, J., & Beach, R. (2001). Hypermedia authoring as critical literacy. Journal of Adolescent and Adult Literacy, 44(6), 538-546.
  37. New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60-92.
  38. Pea, R. (1993). Practices of distributed intelligence and designs for education. In G. Salomon (Ed.), Distributed cognition: Psychologoical and educational considerations. (pp. 47-87). Cambridge, England: CUP.
  39. Perkins, M. (1983). Sensing the world. Indianapolis, IN: Hackett.
  40. Ranker, J. (2008). Making meaning on the screen: Digital video production about the Dominican Republic. Journal of Adolescent and Adult Literacy, 51(5), 410-422.
  41. Ranker, J. (2009). Learning nonfiction in an ESL class: The interaction of situated practice and teacher scaffolding in a genre study. Reading Teacher, 62(7), 580-589.
  42. Shanahan, L.E. (2006). Reading and writing multimodal texts through information and communication technologies. Unpublished doctoral dissertation, University at Buffalo/The State University of New York.
  43. Sider, L. (2003). If you wish to see, listen: The role of sound design. Journal of Media Practice. 4(1), 5-15.
  44. Snyder, T.D., Tan, A.G., & Hoffman, L. (2006). Digest of education statistics 2005 (NCES 2006-030). Washington, DC: U.S. Government Printing Office.
  45. Underwood, M. (2008). I wanted an electronic silence…: Musicality in sound design and the influences of new music on the process of sound design for film. The Soundtrack, 1(3), 193-210.
  46. Unsworth, L. (2001). Teaching multiliteracies across the curricula. Buckingham, England: Open University Press.
  47. Van Leeuwen, T. (1999). Speech, music, sound. London, England: Macmillian.
  48. Winn, W.D. (1993). Perception principles. In M. Fleming, & W.H. Levie (Eds.), Instructional message design: Principles from the behavioral and cognitive sciences (2nd ed.; pp. 55-126). Englewood Cliffs, NJ: Educational Technology.
  49. Yantac, A.E., & Ozcan, O. (2006). The effects of the sound-image relationship within sound education for interactive design. Digital Creativity, 17(2), 91-99.
  50. Yost, W.A. (1993). Overview: Psychoacoustics. In W.A. Yost, A.N. Popper, & R.R. Fay (Eds), Human psychophysics (pp. 1-12). New York, NY: Springer.
  51. Zammit, K. (2010). The new learning environments framework: scaffolding the development of multiliterate students. Pedagogies, 5(4), 325-337.

These references have been extracted automatically and may have some errors. If you see a mistake in the references above, please contact

View References & Citations Map

Cited By

  1. What can your ears do? A systematic literature review regarding the role of non-speech audio

    Azra Tayebi, UEF, Finland; Carolina Islas Sedano, UEF / Ubium, Finland

    EdMedia + Innovate Learning 2014 (Jun 23, 2014) pp. 2653–2663

These links are based on references which have been extracted automatically and may have some errors. If you see a mistake, please contact