Learning From Technical Documents: The Role of Intermodal Referring Expressions

Abstract

Objective: We investigated the impact of two types of intermodal referring expressions on efficiency of instructions for use. Background: User manuals for software or technical devices such as a video recording system frequently combine verbal instructions and illustrations. Much research has shown that the presence of an illustration has a beneficial effect on learning. The present study focuses on a factor that modulates this beneficial effect. The combination of text and an illustration can be effective only if the user integrates the information coming from these two media. This integration depends largely on the intermodal referential expressions, the function of which is to mark explicitly the relations between the text and the illustration. Method: In an experiment (N = 104), we compared the effectiveness of two intermodal referring expressions often used in procedural texts: indexes (numbers introduced in the illustrations and in the instructions to establish cross-references) and icons (visual representations of the components of the device, which are inserted in the verbal instructions). Results: The icons condition led to the most efficient use of the device. Conclusion: This experiment shows that learning from multimedia documents depends on the possibility of effectively connecting the verbal instructions to the illustration. Application: Taking into account the ergonomic properties of the cross-media referring expressions should allow text designers to improve the effectiveness of technical documents.

Get full access to this article

View all access options for this article.

References

André, E. , & Rist, T. (1994). Referring to world objects with text and pictures. In Proceedings of the 15th International Conference on Computational Linguistics—COLING 94 (pp. 530–534). Kyoto, Japan: International Committee on Computational Linguistics.

Chambers, S. K. , Alexander, J. E. , Howard, C. K. , Andre, T. , O'Boyle, M.W. , Eastman, V., et al. (1992). Symbol and word effectiveness for conveying photocopier functions in experienced photocopier users. Perceptual and Motor Skills, 74, 1203–1215.

Chandler, P. , & Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8, 293–332.

Chandler, P. , & Sweller, J. (1992). The split-attention effect as a factor in the design of instruction. British Journal of Educational Psychology, 62, 233–246.

Ellis, J. A. , Whitehill B. V. , & Irick, C. (1996). The effects of explanations and pictures on learning, retention, and transfer of a procedural assembly task. Contemporary Educational Psychology, 21, 129–148.

Ganier, F. , Gombert, J.-E. , & Fayol, M. (2000). Effets du format de présentation des instructions sur l'apprentissage de procédures à l'aide de documents techniques [The effects of instructional formats on procedural learning when using technical documents]. Le Travail Humain, 63, 121–152.

Glenberg, A. M. , & Langston, W. E. (1992). Comprehension of illustrated text: Pictures help to build mental models. Journal of Memory and Language, 31, 129–151.

Glenberg, A. M. , & Robertson, D. A. (1999). Indexical understanding of instructions. Discourse Processes, 28, 1–26.

Gyselinck, V. , & Tardieu, H. (1999). The role of illustrations in text comprehension: What, when, for whom, and why? In H. van Oostendorp & S. R. Goldman (Eds.) Construction of mental representation during reading (pp. 195–218). Hillsdale, NJ: Erlbaum.

10.

Hegarty, M. , & Just, M. A. (1993). Constructing mental models of machine from text and diagrams. Journal of Memory and Language, 32, 717–742.

11.

Howard, C. , O'Boyle, M. W. , Eastman, V. , Andre, T. , & Motoyama, T. (1991). The relative effectiveness of symbols and words to convey photocopier functions. Applied Ergonomics, 22, 218–224.

12.

Kalyuga, S. , Ayres, P. , Chandler, P. , & Sweller, J. (2003). The expertise reversal effect. Educational Psychologist, 38, 23–31.

13.

Kalyuga, S. , Chandler, P. , & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13, 351–371.

14.

Kalyuga, S. , Chandler, P. , & Sweller, J. (2000). Incorporating learner experience into the design of multimedia instruction. Journal of Educational Psychology, 92, 126–136.

15.

Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York: Cambridge University Press.

16.

Mayer, R. E. (1989). Systematic thinking fostered by illustrations in scientific text. Journal of Educational Psychology, 81, 240–246.

17.

Mayer, R. E. (1997). Multimedia learning: Are we asking the right questions? Educational Psychologist, 32, 1–19.

18.

Mayer, R. E. (2003). The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction, 13, 125–139

19.

Mayer, R. E. , & Anderson, R. B. (1991). Animations need narrations: An experimental test of a dual-coding hypothesis. Journal of Educational Psychology, 83, 484–490.

20.

Mayer, R. E. , & Anderson, R. B. (1992). The instructive animation: Helping students build connections between words and pictures in multimedia learning. Journal of Educational Psychology, 84, 444–452.

21.

Najjar, L. J. (1998). Principles of educational multimedia user interface design. Human Factors, 40, 311–323.

22.

Pineda, L. , & Garza, G. (2000). A model for multimodal reference resolution. Computational Linguistics, 26, 139–193.

23.

Sachs, J. S. (1967). Recognition memory for syntactic and semantic aspects of connected discourse. Perception and Psychophysics, 2, 437–442.

24.

Sweller, J. , Chandler, P. , Tierney, P. , & Cooper, M. (1990). Cognitive load factor in the structuring of technical material. Journal of Experimental Psychology: General, 119, 176–192.