Fingerspell: Let Your Fingers Do the Talking

Abstract

In this article we discuss an application that translates hand gestures of the American Sign Language (ASL) alphabet and converts them to text. The FingerSpell application addresses the communication barrier of the deaf and the hearing-impaired by eliminating the need for a third party with knowledge of the American Sign Language, allowing a user to use hand signs that will be translated to letters and words. Through the use of a data glove embedded with multiple input sensors, the application parses each hand and finger gesture signed by the user and detects unique cases which trigger application responses. Application responses come in both visual and auditory forms. The letter corresponding to the unique gesture will appear on the screen and play aloud using computer speech synthesis. The user may also sign an end of word gesture which cues the application to read the previous string of letters together, as one word. This allows a deaf person to communicate with another person without the need for the other person to understand sign language. In addition, this application can serve as a teaching tool for those learning how to sign. FingerSpell is a stepping stone to opening up lines of communication for the Deaf and hearing-impaired communities. Future research and advancements can extend the FingerSpell application to cover the full sign language (word signs) and alphabets of other sign languages.