In response to the growing prevalence of online second language learning and the burgeoning field of international Chinese language education, this study examines the impact of multimodal inputs (MMI) on vocabulary acquisition within online environments among learners of Chinese as a second language (CSL). A teaching intervention was conducted with 90 Mongolian CSL learners, who were grouped into audiovisual, audio, and visual groups. The findings indicate that the audiovisual condition significantly improved vocabulary retention compared to the single-modality conditions in a delayed post-test. Nevertheless, the efficacy of the MMI treatment was observed to vary with learners’ proficiency levels, with beginner-level CSL learners deriving greater benefit from MMI than intermediate-level learners. Furthermore, participants expressed both favorable and critical perspectives regarding the application of MMI in vocabulary instruction. These results highlight the potential of MMI interventions to enhance vocabulary learning in online second-language education, while also underscoring the necessity of considering learners’ target language proficiency and their attitudes when developing MMI-based instructional approaches.
AbrahamL.B. (2008). Computer-mediated glosses in second language reading comprehension and vocabulary learning: A meta-analysis. Computer Assisted Language Learning, 21, 199–226.
2.
AchaJ. (2009). The effectiveness of multimedia programmes in children’s vocabulary learning. British Journal of Educational Technology, 40, 23–31.
3.
AkbulutY. (2007). Effects of multimedia annotations on incidental vocabulary learning and reading comprehension of advanced learners of English as a foreign language. Instructional Science, 35, 499–517.
4.
Al SeghayerK. (2001). The effect of multimedia annotation modes on l2 vocabulary acquisition: A comparative study. Language Learning and Technology, 5, 202–232.
5.
BoersF.WarrenP.HeL.DeconinckJ. (2017). Does adding pictures to glosses enhance vocabulary uptake from reading?System, 66, 113–129.
6.
BrownR.WaringR.DonkaewbuaS. (2008). Incidental vocabulary acquisition from reading, reading-while-listening, and listening to stories. Reading in a Foreign Language, 20, 136–163.
7.
Cárdenas-ClarosM.S.SydorenkoT.HuntleyE.Montero PerezM. (2023). Teachers’ voices on multimodal input for second or foreign language learning. Language Teaching Research. Advance online publication. https://doi.org/10.1177/13621688231216044
8.
ChapelleC.A. (2009). The relationship between second language acquisition theory and computer-assisted language learning. The Modern Language Journal, 93, 741–753.
9.
ChengL. (2008). Washback, impact and consequences. In HornbergerN.H. (Eds.), Encyclopedia of language and education (pp. 359–372). Springer.
10.
ChunD.M.PlassJ.L. (1996a). Effects of multimedia annotations on vocabulary acquisition. Modern Language Journal, 80, 183–198.
11.
ChunD.M.PlassJ.L. (1996b). Facilitating reading comprehension with multimedia. System, 24, 503–519.
12.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.
13.
DiaoY.SwellerJ. (2007). Redundancy in foreign language reading comprehension instruction: Concurrent written and spoken presentations. Learning and Instruction, 17, 78–88.
14.
EllisN.C.BeatonA. (1993). Psycholinguistic determinants of foreign language vocabulary learning. Language Learning, 43, 559–617.
15.
GohC.C.VandergriftL. (2021). Teaching and learning second language listening: Metacognition in action. Routledge.
16.
GuQ.ZangC. (2011). Shuru motai dui dier yuyan lijie he fudai cihui xide de yingxiang [The impact of input modality on second language comprehension and incidental vocabulary acquisition]. Jiefangjun Waiguoyu Xueyuan Xuebao, 3, 89–95.
17.
HampelR.SticklerU. (2012). The use of videoconferencing to support multimodal interaction in an online language classroom. ReCALL, 24, 116–137.
18.
HongW.WuA.WuQ. (2018). Renwu de motai peizhi dui Hanyu eryu wenben lijie, cihui he jufa xuexi de yingxiang [The impact of task modality configuration on Chinese as a second language text comprehension, vocabulary, and syntax learning]. Hanyu Jiaoxue Yu Yanjiu, 32, 401–416.
19.
HuW.LuoY. (2024). Chinese English language learners’ vocabulary retention: Investigating the effectiveness of neuro/metacognitive and socio-cultural strategies. BMC Psychology, 12, 113.
20.
JiangN.NekrasovaT.M. (2007). The processing of formulaic sequences by second language speakers. Modern Language Journal, 91, 433–445.
21.
JinH. (2017). Hanzi jiagong yanjiu dui Hanzi yuedu jiaoxue de qishi [Insights from Chinese character processing research for Chinese character reading instruction]. Guoji Hanyu Jiaoxue Yanjiu, 2, 45–60.
22.
KhezrlouS.EllisR.SadeghiK. (2017). Effects of computer-assisted glosses on EFL learners’ vocabulary acquisition and reading comprehension in three learning conditions. System, 65, 104–116.
23.
KodaK. (2005). Insights into second language reading: A cross-linguistic approach. Cambridge University Press.
24.
LiW.YuJ.ZhangZ.LiuX. (2022). Dual coding or cognitive load? Exploring the effect of multimodal input on English as a foreign language learners’ vocabulary learning. Frontiers in Psychology, 13, 834706.
25.
LightbownP.HalterR.WhiteJ.HorstM. (2002). Comprehension-based learning: The limits of ‘do it yourself’. The Canadian Modern Language Review, 58, 427–464.
MaoW.B.YangZ.L.WangL.S. (2010). Modality effect in false recognition: Evidence from Chinese characters. International Journal of Psychology, 45, 4–11.
28.
MayerR.E. (2002). Multimedia learning. Psychology of Learning and Motivation, 41, 85–139.
29.
MayerR.E. (2014). Cognitive theory of multimedia learning. In MayerR.E. (Ed.), The Cambridge handbook of multimedia learning (2nd ed., pp. 43–71). Cambridge University Press.
30.
McBride-ChangC.ShuH.ZhouA.WatC.P.WagnerR.K. (2003). Morphological awareness uniquely predicts young children’s Chinese character recognition. Journal of Educational Psychology, 95, 743–751.
31.
Ministry of Education of the People’s Republic of China. (2010). Hanyu guoji jiaoyu yong yinjie-Hanzi-cihui dengji huafen [Graded Chinese syllables, characters, and words for the application of teaching Chinese to the speakers of other languages]. Shangwu yinshuguan.
32.
MorenoR.MayerR.E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology, 94, 156–163.
33.
NationI.S.P. (2001). Learning vocabulary in another language. Cambridge University Press.
34.
NiuR.Helms-ParkR. (2014). Interaction, modality, and word engagement as factors in lexical learning in a Chinese context. Language Teaching Research, 18, 345–372.
35.
PaivioA. (1991). Dual coding theory: Retrospect and current status. Canadian Journal of Psychology, 45, 255–287.
36.
PaivioA. (2014). Mind and its evolution: A dual coding theoretical approach. Psychology Press.
37.
PalmerA.BachmanL.F. (1996). Language testing in practice: Designing and developing useful language tests. Oxford University Press.
38.
PetersE. (2019). The effect of imagery and on-screen text on foreign language vocabulary learning from audiovisual input. TESOL Quarterly, 53, 1008–1032.
39.
PlassJ.L.ChunD.M.MayerR.E.LeutnerD. (1998). Supporting visual and verbal learning preferences in a second-language multimedia learning environment. Journal of Educational Psychology, 9, 25–36.
40.
PuimègeE.PetersE. (2019). Learning L2 vocabulary from audiovisual input: An exploratory study into incidental learning of single words and formulaic sequences. Language Learning Journal, 47, 424–438.
41.
PuimègeE.PetersE. (2020). Learning formulaic sequences through viewing L2 television and factors that affect learning. Studies in Second Language Acquisition, 42, 525–549.
42.
RamezanaliN.FaezF. (2019). Vocabulary learning and retention through multimedia glossing. Language Learning and Technology, 23, 105–124.
43.
RamezanaliN.UchiharaT.FaezF. (2021). Efficacy of multimodal glossing on second language vocabulary learning: A meta-analysis. TESOL Quarterly, 55, 105–133.
44.
RottS. (1999). The effect of exposure frequency on intermediate language learners’ incidental vocabulary acquisition and retention through reading. Studies in Second Language Acquisition, 21, 589–619.
45.
SadeghiK.KhezrlouS.ModirkhamenehS. (2017). Calling Iranian learners of L2 English: Effect of gloss type on lexical retention and reading performance under different learning conditions. Journal of Research in Reading, 40, S66–S86.
46.
ShadievR.WuT.HuangY. (2020). Using image-to-text recognition technology to facilitate vocabulary acquisition in authentic contexts. ReCALL, 32, 195–212.
47.
ShuH.ChenX.AndersonR.C.WuN.XuanY. (2003). Properties of school Chinese: Implications for learning to read. Child Development, 74, 27–47.
48.
SmithC.D.ScarfD. (2017). Spacing repetitions over long timescales: A review and a reconsolidation explanation. Frontiers in Psychology, 8, Article 962.
49.
SwellerJ. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257–285.
50.
SwellerJ. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4(4), 295–312.
51.
SwellerJ. (2010). Cognitive load theory: Recent theoretical advances. In PlassJ.L.MorenoR.BrünkenR. (Eds.), Cognitive load theory (pp. 29–47). Cambridge University Press.
52.
SwellerJ.ChandlerP. (1991). Evidence for cognitive load theory. Cognition and Instruction, 8(4), 351–362.
53.
TaylorG. (2005). Perceived processing strategies of students watching captioned video. Foreign Language Annals, 38, 422–427.
54.
TragantE.MuñozC.SpadaN. (2016). Maximizing young learners’ input: An intervention program. The Canadian Modern Language Review, 72, 234–257.
55.
TragantE.VallbonaA. (2018). Reading while listening to learn: Young EFL learners’ perceptions. ELT Journal, 72, 395–404.
56.
VahediV.S.GhonsoolyB.PishghadamR. (2016). Vocabulary glossing: A meta-analysis of the relative effectiveness of different gloss types on L2 vocabulary acquisition. Teaching English With Technology, 16, 3–25.
57.
WangD.LiD. (2023). Integrating Digital Multimodal Composition (DMC) into Chinese language teaching. In HongJ.F.SiC.M. (Eds.), Teaching Chinese language in the international school context (pp. 87–102). Springer. https://doi.org/10.1007/978-981-99-6376-8_7
58.
WebbS.ChangA.C.-S. (2012). Vocabulary learning through assisted and unassisted repeated reading. The Canadian Modern Language Review, 68, 267–290.
59.
WescheM.ParibakhtT.S. (1996). Assessing second language vocabulary knowledge: Depth versus breadth. Canadian Modern Language Review, 53, 13–40.
60.
YehY.WangC.W. (2003). Effects of multimedia vocabulary annotations and learning styles on vocabulary learning. Calico Journal, 21, 131–144.
61.
YunJ. (2011). The effects of hypertext glosses on L2 vocabulary acquisition: A meta-analysis. Computer Assisted Language Learning, 24, 39–58.
62.
ZhangS. (2024). A study on the teaching practice of multimodal data analysis in international Chinese language education for international students in China. Applied Mathematics and Nonlinear Sciences, 9, 1–15.
63.
ZhouJ. (2022). The contribution of morphological awareness and vocabulary knowledge to Chinese as a second language reading comprehension: A path analysis. Journal of Psycholinguistic Research, 51, 55–74.
