Television programs and films include emotional content which is intended to be captured and interpreted by the viewer. Auditory content such as sound effects, mood music, and vocal intonation cannot be perceived by those in the audience who cannot hear. The tone of voice used to communicate sarcasm is one such type of content. While captioning is designed to assist the deaf and hard of hearing in understanding this information, it has many weaknesses. This paper presents the results of a pilot study which applied user interface design principles to develop prototypes of enhanced captioning technology that can support the translation of sarcastic auditory emotional content into a visual modality.
BryantG.TreeFox J. (2005). Is there an ironic tone of voice? Language and Speech, 48(3), 257–277.
3.
ButtB.FarrellM.StoreyD.) (2004). Ruby Reborn [Television series episode]. In B. Butt, D. Storey, & V. Thompson (Executive Producers), Corner Gas. Toronto: CTV.
EkmanP. (1999). Basic emotions. In DalgleishT.PowerT.The Handbook of Cognition and Emotion, pp. 45–60. Sussex, U. K.: John Wiley & Sons, Ltd.www.paulekman.com/pdfs/basic_emotions.pdf.
8.
EkmanP.FriesenW. V. (1986). A new pan-cultural facial expression of emotion. Motivation and Emotion, 10(2), 159–168.
9.
EysenckH. J.EysenckM. W. (1985). Personality and individual differences: A natural science approach. New York: Plenum.
FelsD.SilvermanC. (2002). Issues of closed captioning and interactive television. In MiesenbergerK.ClausJ.ZaglerW.Computers Helping People With Special Needs: 8th Annual Conference Proceedings/ICCHP 2002 (pp. 292–294. New York: Springer.
12.
GarziaR.LondonR. (1995). Vision and reading, St. Louis: Mosby Publishing.
13.
HarkinsJ.KorresE.SingerB.VirvanB. (1995). Non-speech information in captioned video: A consumer opinion study with guidelines for the captioning industry (OP95–1). Washington, DC: Gallaudet Research Institute.
MatessinoM. (1999, Sept. 24). “Letter in response to “A Study of Jaws Incisive Overture To Close Off the Century”. www.filmscoremonthly.com.
20.
MehrabianA. (1996). Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 14(4), 261–292.
21.
PlutchikR. (1980). Emotion, a psychoevolutionary synthesis. New York: Harper & Row.
22.
RashidR.AitkenJ.FelsD. (2006). Expressing emotions using animated text captions. In MiesenbergerK.KlausJ.ZaglerW.KarshmerA. (eds), Computers helping people with special needs: 10th International Conference, ICCHP 2006 (pp. 24–31. Lecture Notes in Computer Science. New York: Springer.
23.
Sarcasm (2007). The American Heritage® College Dictionary of the English Language, (4th ed.). Boston: Houghton Mifflin Company.
