Semantic Modality in Information Design: Improving Learning Efficiency in a Multimedia Environment

Abstract

Instructional theories emphasize the structural modality of the information design, that is, whether the information is visual (e.g., text vs. graphics vs. animations) or auditory. However, we argue that semantic modality, the internal cognitive representation, must also be considered in order to reduce working memory load imposed by cognitive operations. A within-subjects design comparing performance on a dual mode task demonstrated the importance of considering semantic modality. The hypothesized factor resulting in the change in performance was the type of mental processing required while physical stimuli were held constant. Significant differences in performance between control and experimental trials were found for experimental conditions in which a visual task was manipulated to interfere with an auditory task during internal coding. No significant differences were found when the experimental condition was a visual task manipulated to allow separate internal coding. The results have implications for cognitive theories of multimedia learning with respect to issues of optimal methods of information design.