This interpretive case study investigated how a fifth-grade teacher’s social practices with visual and linguistic signs positioned her students (10- and 11-year-olds) to take up particular modes as they constructed digital compositions. The context of the study was a suburban public school in the northeastern United States. Analysis was threefold. The discourse surrounding multimodal composition was analyzed via inductive analysis. Students’ use of semiotic resources in the HyperStudio composition was analyzed with Unsworth’s image-language intermodal framework. Then, teacher-student conversations related to visual and linguistic signs were triangulated with students’ compositions. Findings show that a classroom teacher’s limited content knowledge as related to metafunctions and metalanguage of visual and linguistic sign systems affected the information taught to the students and, ultimately, their use of visual and linguistic signs. Students demonstrated tacit knowledge of image-language relations beyond what was taught but lacked the explicit knowledge to more strategically use visual and linguistic signs. Implications include the importance of creating opportunities for teachers to develop more substantive content knowledge of the metalanguages and metafunctions of various sign systems.
AjayiL. (2009). English as second language learners’ exploration of multimodal texts in a junior high school. Journal of Adolescent and Adult Literacy, 52(7), 585-595.
BezemerJ.KressG. (2008). Writing in multimodal texts: A social semiotic account of designs for learning. Written Communications, 25(2), 166-195.
4.
BigumC.DurrantC.GreenB.HonanE.LankshearC.MorganW.. . . WildM. (1997). Digital rhetorics: Literacies and technologies in education. Vol. 1: Current practices and future directions. Retrieved from http://www.chrisbigum.com/downloads/volume1.pdf
5.
DalyA.UnsworthL. (2011). Analysis and comprehension of multimodal texts. Australian Journal of Language and Literacy, 34(1), 61-68.
HallidayM. A. K. (1985). An introduction to functional grammar. London: Arnold.
8.
HallidayM. A. K.MatthiessenC. (2004). An introduction to functional grammar (3rd ed.) London: Arnold.
9.
HealyA. (Ed.). (2008). Multiliteracies and diversity in education: New pedagogies for expanding landscapes. Melbourne: Oxford University Press.
10.
HonanE. (2008). Barriers to teachers using digital literacies. Literacy, 42(1), 36-42.
11.
HullG. A.NelsonM. E. (2005). Locating the semiotic power of multimodality. Written Communication, 22, 224-261.
12.
IedemaR. (2003). Multimodality, resemiotization: Extending the analysis of discourse s multi-semiotic practice. Visual Communication, 2(1), 30-57.
13.
JewittC. (Ed.). (2009) The Routledge handbook of multimodal analysis. New York: Routledge.
14.
JewittC.KressG. (2003). Multimodal literacy. New York: Lang.
15.
KalantzisM.CopeB. (2005). Learning by design. Melbourne: Victorian Schools Innovation Commission.
16.
KamilM.IntratorS.KimH. (2000). The effects of other technologies on literacy and learning. In KamilM.MosenthalP.PearsonP.BarrR. (Eds.), Handbook of reading research (Vol. 3, pp. 771-778). Mahwah, NJ: Erlbaum.
17.
KingJ. R.O’BrienD. G. (2002). Adolescents’ multiliteracies and their teachers’ needs to know: Toward a digital detente. In AlvermannD. E. (Ed.), Adolescents and literacies in a digital world (pp. 40-50). New York: Lang.
18.
KressG. (1998). Visual and verbal modes of representation in electronically mediated communication: The potentials of new forms of text. In SnyderI. (Ed.), Page to screen taking literacy into the electronic era (pp. 5-79). London: Routledge.
19.
KressG. (2003). Literacy in the new media age. London: Routledge.
20.
KressG.van LeeuwenT. (1996). Reading images: The grammar of visual design. London: Routledge.
21.
KressG.van LeeuwenT. (2001). Multimodal discourse: The modes and media of contemporary communication. London: Arnold.
22.
KressG.van LeeuwenT. (2006). Reading images: The grammar of visual design (2nd ed.). London: Routledge.
23.
LankshearC.BigumC. (1999). Literacies and new technologies in school settings. Curriculum Studies, 1, 445-465.
24.
LankshearC.KnobelM. (2003). New literacies: Changing knowledge and classroom learning. Philadelphia: Open University Press.
25.
LankshearC.KnobelM.GreenB. (2000). Teachers and technoliteracy: Managing literacy, technology, and learning in schools. St. Leonards, Australia: Allen & Unwin.
26.
LeanderK. (2009). Composing with old and new media: Toward parallel pedagogy. In CarringtonV.RobinsonM. (Eds.), Digital literacies: Social learning and classroom practices (pp.147-164). London: Sage.
27.
LimV. F. (2004). Developing an integrative multi-semiotic model. In O’HalloranK. (Ed.), Multimodal discourse analysis: System functional perspectives (pp. 220-246). New York: Continuum.
28.
LiuY.O’HalloranK. L. (2009). Intersemiotic texture: Analyzing cohesive devices between language and images. Social Semiotics, 19(4), 367-388.
29.
MartinecR.SalwayA. (2005). A system for image-text relations in new (and old) media. Visual Communication, 4(3), 337-371.
30.
McVeeM. B.BaileyN. B.ShanahanL. E. (2008). Using digital media to interpret poetry: Spiderman meets Walt Whitman. Research in Teaching of English, 43(2), 112-143.
31.
MerriamS. B. (2009). Qualitative research and case study applications in education. San Francisco: Jossey-Bass.
32.
MilesM. B.HubermanA. M. (1994). Qualitative data analysis. Thousand Oaks, CA: Sage.
33.
MillerS.M. (2008). Teacher learning for new times: Repurposing new multimodal literacies and digital video composing for schools. In FloodJ.HeathB.LappD. (Eds.), Handbook of research on teaching literacy through the communicative and visual arts (Vol. 2, pp. 441-460). New York: International Reading Association / Simon & Schuster Macmillan.
34.
MillerS. M.BorowiczS. (2006). Why multimodal literacies? Designing digital bridges to 21st century teaching and learning. Buffalo, NY: GSE.
35.
MillerS. M.McVeeM. (Eds.). (2012). Multimodal composing in classrooms: Learning and teaching in the digital world. New York: Routledge.
36.
NelsonM.HullG.Roche-SmithJ. (2008). Challenges of multimedia self-presentation: Taking and mistaking the show on the road. Written Communication, 25, 415-440.
37.
New London Group. (1996). A pedagogy of multiliteracies: Designing social futures. Harvard Educational Review, 66(1), 60-92.
38.
O’HalloranK. L. (2011). Multimodal discourse analysis. In HylandK.PaltridgeB. (Eds.), Companion to discourse (pp. 120-137). London: Continuum.
39.
O’TooleM. (1994). The language of displayed art. London: Leicester University Press.
40.
RoyceT. D. (1998). Synergy on the page: Exploring intersemiotic complementarity in page-based multimodal text. Japan Association Systemic Functional Linguistic Occasional Papers, 1(1), 25-49.
41.
SeidmanI. (1998). Interviewing as qualitative research (2nd ed.). New York: Teachers College.
ShanahanL. E. (2013). Multimodal representations: A fifth-grade teacher influences students’ design and production. Pedagogies: An International Journal, 8(2), 85-102.
44.
SmagorinskyP. (1995). Constructing meaning in the disciplines: Reconceptualizing writing across the curriculum as composing across the curriculum. American Journal of Education, 103(2), 160-184.
45.
SmagorinskyP. (2008). The method section as conceptual epicenter in constructing social science research reports. Written Communication, 25(3), 389-411.
46.
SpradleyJ. P. (1980). Participant observation. New York: Harcourt Brace Jovanovich College.
47.
StraussA. L.CorbinJ. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage.
48.
UnsworthL. (2001). Teaching multiliteracies across the curricula. Buckingham, England: Open University Press.
49.
UnsworthL. (2006). Towards a metalanguage for multiliteracies education: Describing the meaning-making resources of language-image interactions. English Teaching: Practice and Critique, 5(1), 55-76.
50.
VasudevanL.SchultzK.BatemanJ. (2010). Rethinking composing in a digital age: Authoring literacy identities through multimodal storytelling. Written Communication, 27, 442-468.
51.
VygotskyL. S. (1981). The genesis of higher mental functions. In WertschJ. V. (Ed.), The concept of activity in Soviet psychology (pp. 144-188). Armonk, NY: Sharpe.
52.
VygotskyL. S. (1997). The collected works of L. S. Vygotsky: Vol. 4. The history of the development of higher mental functions (HallM. J., Trans.; ReiberR. W., Ed.). New York: Plenum.
53.
WertschJ. V. (1991). Voices of the mind. Cambridge, MA: Harvard University Press
54.
WitteS. P. (1992). Context, text, intertext: Toward a constructivist semiotic of writing. Written Communication, 9, 237-308.
55.
ZammitK. (2010). The new learning environments framework: Scaffolding the development of multiliterate students. Pedagogies: An International Journal, 5(4), 325-337.
