The advent of two technologies - inexpensive video recorders and digital video annotation software - is revolutionizing the study of human interaction. With palm-held camcorders, researchers can now easily collect video data in the field, where humans interact most naturally. More important, using free digital video annotation software, one can annotate and analyze video data on a laptop. In this methods review, we discuss our own experience with such tools in the study of gesture and language. We note points of general interest to researchers of other disciplines, and conclude with a discussion of new issues raised by the presence of readily available multimedia data.
Bigbee, A., Loehr, D. and Harper, L. (2001) `Emerging Requirements for Multi-Modal Annotation and Analysis Tools' , Proceedings, 7th European Conference on Speech Communication and Technology (Eurospeech 2001), Aalborg, Denmark, September.
3.
Boersma, P. (2001) `Praat, A System for Doing Phonetics by Computer' , Glot International5(9/10): 341-345 .
4.
Condon, W. and Ogston, W. (1966) `Soundfilm Analysis of Normal and Pathological Behavior Patterns' , Journal of Nervous and Mental Disorders143: 338-347 .
5.
Dybkjær, L., Berman, S., Kipp, M., Olsen, M., Pirrelli, V., Reithinger, M. and Soria, C. (2001) `Survey of Existing Tools, Standards, and User Needs for Annotation of Natural Interaction and Multimodal Data', ISLE Deliverable D11.1, URL (consulted Nov. 2002): http://www.ilc.pi.cnr.it/EAGLES96/isle/
6.
(Originally published as Gesture and Environment. New York: King's Crown Press .)
7.
Kendon, A. (1972) `Some Relationships between Body Motion and Speech: An Analysis of an Example', in Aron Siegman and Benjamin Pope (eds) Studies in Dyadic Communication. New York: Pergamon Press .
8.
Kendon, A. (1980) `Gesticulation and Speech: Two Aspects of the Process of Utterance', in Mary Ritchie Key (ed.) The Relationship of Verbal and Nonverbal Communication, pp. 207-227. The Hague: Mouton .
9.
Kipp, M. (2001) `Anvil - A Generic Annotation Tool for Multimodal Dialogue' , Proceedings, 7th European Conference on Speech Communication and Technology (Eurospeech 2001), Aalborg, Denmark, September.
Knudsen, M., Martin, J-C., Dybkjær, L., Ayuso, M., Bernsen, N., Carletta, J., Heid, Ul, Kita, S., Llisterri, J., Pelachaud, C., Poggi, I., Reithinger, N., Van Elswijk, G. and Wittenburg, P. (2002) `Survey of Multimodal Annotation Schemes and Best Practice', ISLE Deliverable D9.1, URL (consulted Nov. 2002): http://www.ilc.pi.cnr.it/EAGLES96/isle/
McQuown, N. A. (ed.) (1971) The Natural History of an Interview. Microfilm Collection of Manuscripts on Cultural Anthropology, 15th series, Joseph Regenstein Library, University of Chicago .
14.
McNeill, D. (1992) Hand and Mind: What Gestures Reveal about Thought. Chicago: University of Chicago Press .
15.
Muybridge, E. (1887) Animal Locomotion: An Electro-Photographic Investigation of Consecutive Phases of Animal Movements. Philadelphia: J.B. Lippincott .
16.
Neidle, C. (2000) `SignStream™: A Database Tool for Research on Visual-Gestural Language', American Sign Language Linguistic Research Project, Report Number 10, Boston University , URL (consulted Nov. 2002): http://www.bu.edu/asllrp/reports.html#RPT10
17.
Oviatt, S.L., DeAngeli, A. and Kuhn, K. (1997) `Integration and Synchronization of Input Modes during Multimodal Human-Computer Interaction' , Proceedings, Conference on Human Factors in Computing Systems (CHI '97), Atlanta, March, ACM Press.
18.
Quek, F., Bryll, R., Kirbas, C., Arslan, H. and McNeill, D. (2000) `A Multimedia Database System for Temporally Situated Perceptual Psycholinguistic Analysis' , Proceedings, 3rd International Conference on Methods and Techniques in Behavioral Research (Measuring Behavior), Nijmegen, The Netherlands, August.
