We describe two experiments on signaling and detecting uncertainty in audiovisual speech by adults and children. In the first study, utterances from adult speakers and child speakers (aged 7—8) were elicited and annotated with a set of six audiovisual features. It was found that when adult speakers were uncertain they were more likely to produce fillers, delays, high intonation, eyebrow movements, and “funny faces.” The basic picture for the child speakers was somewhat similar, in that the presence of certain audiovisual cues correlated with uncertainty, but the differences were relatively small and less often significant. In the second study both adult and child judges watched responses from adult and child speakers selected from the first study to find out whether they were able to correctly estimate a speakers' level of uncertainty. It was found that both child and adult judges gave more accurate scores for answers from adult speakers than from child speakers and that child judges overall provided less accurate scores than adult judges.
Ackerman, B. (1983). Form and function in childrens' understanding of ironic utterances. Journal of Experimental Child Psychology , 35, 487-508.
2.
Barkhuysen, P., Krahmer, E., & Swerts, M. (2005a). Predicting end of utterance in multimodal and unimodal conditions, submitted.
3.
Barkhuysen, P., Krahmer, E., & Swerts, M. (2005b). Problem detection in human-machine interactions based on facial expressions of users. Speech Communication , 45(3), 343 - 359.
4.
Bartsch, K., & Wellman, H ( 1995). Children talk about the mind, New York: Oxford University Press.
5.
Beach, C., Katz, W., & Skowrosnki, A. (1996). Children's processing of prosodic cues for phrasal interpretation. Journal of the Acoustical Society of America, 99, 1148-1160.
6.
Bieri, D., Reeve, R., Champion, G., Addicoat, L. & Ziegler, J. (1990). The Faces Pain Scale for the self-assessment of the severity of pain experienced by children: Development, initial validation, preliminary investigation for ratio scale properties. Pain , 41, 139-150.
7.
Bisanz, G., Vesonder, G., & Voss, J. (1978). Knowledge of one's own responding and relation of such knowledge to learning: A developmental study. Journal of Experimental Child Psychology, 25, 116-128.
8.
Brennan, S.E., & Williams, M. (1995). The feeling of another's knowing: prosody and filled pauses as cues to listeners about the metacognitive states of speakers . Journal of Memory and Language, 34, 383-398.
9.
Brown, A., & Lawton, S. (1977). The feeling of knowing experience in educable retarded children. Developmental Psychology, 13, 364-370.
10.
Butterfield, E., Nelson, T., & Peck, V. (1988). Developmental aspects of the feeling of knowing . Developmental Psychology, 24, 654-663.
11.
Cassell, J. (2004). Towards a model of technology and literacy development: Story listening systems. Journal of Applied Developmental Psychology, 25(1), 75-105.
12.
Clark, H., & Krych, M. (2004). Speaking while monitoring addressees for understanding . Journal of Memory and Language, 50, 62-81.
13.
Cruttenden, A. (1974). An experiment involving comprehension of intonation in children from 7 to 10. Journal of Child Language, 1, 221-231.
14.
Cruttenden, A. (1985). Intonation comprehension in 10-year-olds. Journal of Child Language. 12, 643-661.
15.
Cruttenden, A. (1986). Intonation. Cambridge: Cambridge University Press.
Doherty-Sneddon, G., & Kent, G. (1996). Visual signals and the communication abilities of children, Journal of Child Psychology and Psychiatry. 37, 949-959.
18.
Ekman, P. (1979). About brows: Emotional and conversational signals . In M. von Cranach et al. (Eds.), Human Ethology (pp.169-202). Cambridge University Press, Cambridge.
Esposito, A., Marinaro, M., & Palombo, G. (2004). Children speech pauses as markers of different discourse structures and utterance information content. In From Sound to Sense, June 11-13, MIT, Cambridge, MA, pp.139-144.
21.
Flavell, J. (1999). Cognitive development: Childrens' knowledge about the mind. Annual Review of Psychology, 50, 21-40.
22.
Geluykens, R. (1987). Intonation and speech act type. An experimental approach to rising intonation in declaratives. Journal of Pragmatics, 11, 483-494.
23.
Graham, T. (1999). The role of gesture in children's learning to count. Journal of Experimental Child Psychology, 74, 333-355.
24.
Granström, B., House, D., & Lundeberg, M. (1999). Prosodic cues in multimodal speech perception . Proceedings 14th International Conference of the Phonetic Sciences (ICPhS), San Francisco.
25.
Harris, M., & P. Pexman (2003). Children's perceptions of the social function of verbal irony. Discourse Processes, 36, 147-165.
26.
Hart, J.T. (1965). Memory and the feeling-of-knowing experience. Journal of Educational Psychology, 56, 208-216.
27.
Jameson, A., Nelson, T., Leonesio, R., & Narens, L. (1993). The feeling of another person's knowing. Journal of Memory and Language, 32, 320-335.
28.
Keating, P., Baroni, M., Mattys, S., Scarborough, R., Alwan, A., Auer, E., & Berstein, L. (2003). Optical phonetics and visual perception of lexical and phrasal stress in English. In Proceedings 16th International Conference of the Phonetic Sciences (ICPhS), Barcelona, Spain, pp.2071-2074.
29.
Keenan, T., & Quigely, K. (1999). Do young children use echoic information in their comprehension of sarcastic speech? British Journal of Developmental Psychology, 17, 83-96.
30.
Koenig, M., Clément, F., & Harris, P. (2004). Trust in testimony; Children's use of true and false statements. Psychological Science, 15(10), 694-698.
31.
Koriat, A. (1993). How do we know that we know? The accessibility model of the feeling of knowing. Psychological Review, 100, 609-639.
32.
Krahmer, E., & Swerts, M. (2004). More about brows. In Zs. Ruttkay & C. Pelachaud (Eds.), From brows to trust: Evaluating embodied conversational agents . Kluwer Academic Press, Dordrecht .
33.
Kreutzer, M., Leonard, S., & Flavell, J. (1975). An interview study of children's knowledge about memory. Monographs of the Society for Research in Child Development , 40,1-57.
34.
Ladd, D.R. (1996). Intonational phonology. Cambridge : Cambridge University Press.
35.
Leuckefeld, K., Hahne, A., & Alter, K. (2003). Neuronal correlates of prosodic information processing and syntactic disambiguation in school-aged children. Paper presented at CUNY Conference on Human Sentence Processing, MIT.
36.
Lockl, K., & Schneider, W. (2002). Developmental trends in children's feeling-of-knowing judgments. International Journal of Behavioral Development , 26, 327-333.
37.
Matsui, T., McCagg, P., Yamamoto, T., Murakami, Y., & Fernald, A. (2003). Japanese preschoolers' early understanding of (un)certainty: A cultural perspective on the role of language in development of theory of mind. In A. Brugos, L. Micciulla, & C. Smith, (Eds.), Proceedings of the 28th annual Boston University Conference on Language Development (BUCLD) (pp. 350-362). Boston: Cascadilla Press.
38.
MacWhinney, B., & Osser, H. (1977). Verbal planning functions in Children's speech . Child Development, 48, 978-985.
39.
McNeill, D. (1992). Hand and Mind: What gestures reveal about thought . Chicago: The University of Chicago Press.
40.
Montanari, S., Yildirim, S., Khurana, S., Landes, M., Lawyer, L., Andersen, E., & Narayanan, S. (2004). Analyzing the interplay between spoken language and gestural cues in conversational child-machine interactions in pre/early literare age groups, In Proceedings of InSTIL/ICALL-NLP and Speech Technologies in Advanced Language Learning Systems, Venice, 17-19 June.
41.
Moore, C., Bryant, D., & Furrow, D. (1989). Mental terms and the development of certainty . Child Development, 60, 167-171.
42.
Moore, C., Pure, K., & Furrow, D. (1990). Children's understanding of the modal expression of speaker certainty and uncertainty and its relation to the development of a representational theory of mind. Child Development, 61, 722-730.
43.
Munhall, K., Jones, J., Callan, D., Kuratate, T., & Vatikiotis-Bateson, E. (2004). Visual prosody and speech intelligibility. Psychological Science, 15, 133-137.
44.
Narayanan, S., & Potamianos, A. (2002). Creating conversational interfaces for children. IEEE Transactions on Speech and Audio Processing, 10, 65-78.
45.
Nelson, T. (1993). Metacognition: Core readings. Prentice Hall.
46.
Nelson, T., & Narens, L. (1990). Metamemory: A theoretical framework and new findings . The Psychology of Learning and Motivation, 26, 125 -141.
47.
Pena-Brooks, A., & Hegde, M. (2000). Assessment and treatment of articulation and phonological disorders in children. Texas: pro-ed .
48.
Pressley, M.Levin, J.R., & Ghatala, E.S. (1984). Memory strategy monitoring in adults and children . Journal of Verbal Learning and Verbal Behavior, 23, 270-288.
49.
Read, J., MacFarlane, S., & Casey, C. (2002). Endurability, engagement and expectations, In M. Bekker, et al. (Eds.), Interaction design and children. Shaker Publishing.
50.
Robinson, E., Mitchell, P., & Nye, R. (1995). Young children's treating of utterances as unreliable sources of knowledge. Journal of Child Language, 22, 663-685.
51.
Sabbagh, M., & Baldwin, D. (2001). Learning words from knowledgeable versus ignorant speakers: Links between preschoolers' theory of mind and semantic development . Child Development, 72(4), 1054-1070.
52.
Schmidt, K., & Cohn, J. (2002). Human facial expressions as adaptations: Evolutionary questions in facial expression. Yearbook of Physical Anthropology , 44, 3-24.
53.
Schneider, W. (2001). Memory development in children. In N. Smelser & P. B. Baltes (Eds.), International Encyclopedia of the Social and Behavioral Sciences, New York: Pergamon.
54.
Schneider, W., & Lockl, K. (2002). The development of metacognition and knowledge in children and adolescents. In T. Perfect & B. Schwartz (Eds.), Applied MetacognitionCambridge: Cambridge University Press, pp.224-257.
55.
Smith, V.L., & Clark, H.H. (1993). On the course of answering questions. Journal of Memory and Language, 32, 25-38.
56.
Srinivasan, R., & Massaro, D. (2003). Perceiving prosody from the face and voice: Distinguishing statements from echoic questions in English. Language and Speech, 46, 1 - 22.
57.
Swerts, M., & Krahmer E. (2005). Audiovisual prosody and feeling of knowing. Journal of Memory and Language, 53(1), 81-94.
58.
Vogel, I., & Raimy, E. (2002). The acquisition of compound vd. phrasal stress; the role of prosodic constituents. Journal of Child Language , 29, 225-250.
59.
Wellman, H. (1977). Tip of the tongue and feeling of knowing experiences: A developmental study of memory monitoring. Child Development , 48, 13-21.
60.
Wellman, H., Cross, D., & Watson, J. (2001). Meta-analysis of theory-of-mind development: The truth about false belief. Child Development, 72(3), 655-684.
