The limited effects of visual and audio modalities on second language listening comprehension

Abstract

This study examined the effects of watching gestures and lip movement on second language (English) listening comprehension. The participants were 30 high and 22 low proficient learners of English. There were six listening tasks combining two variables: modality and difficulty. The modality consisted of three types of assessments: Body (the upper half of body was visible), Face (close up view), and Audio. The difficulty consisted of two levels: Easy and Hard. Learners watched Body, Face, or listened to Audio, and worked on six comprehension questions twice for each task. Moreover, the participants were asked about their modality of preference. The notable results were (1) the modality of listening had no influence on their listening performance, (2) for the high proficiency group, the score difference between the easy and the hard texts was larger at the second attempt than at the first attempt, (3) the participants generally preferred to watch the whole body, followed by watching the face, and then listening only, (4) the high proficiency group showed more variations of preference than the low proficiency group, and (5) the participants’ modality preference had no influence on their listening performance.