On the Merits of Using Multiple Indices in Evaluating Small Graphical Symbols

Abstract

This paper summarizes an international study that aimed to evaluate candidate telecommunication graphical symbols (icons and pictograms) developed in the west across different cultural groups by means of tests producing multiple indices or parameters of performance. Prospective users from eastern (Asian) and western countries were used as subjects. Several tests were performed utilizing videophone symbols based on studies done by the Human Factors Technical Committee (HFTC) of the European Telecommunication Standards Institute (ETSI). The ETSI-recommended symbols for 7 videophone functions or referents were tested using more than 300 subjects from Indonesia, Malaysia, Philippines, Thailand, Sri Lanka, Finland, Sweden and the USA. Two other sets of 7 symbols each from the ETSI study were also tested. The tests used were spontaneous identification, the cued response, and the preference tests. Confidence judgement (subjective certainty ratings) complemented the subjects' answers for the first tests. Semantic differential scaling tests (SDT) were also done as added evaluatory tool. Results from spontaneous identification tests revealed very poor identification of most of the symbols in contrast to the cued response test results. Barely recognizing what the symbols meant strongly suggested the need to either redesign the symbols or to ensure adequate opportunities for familiarizing and educating the prospective users with the new symbols. The subjective certainty scores helped in studying the level of confidence of the answers by the subjects. Furthermore, the studies revealed that symbols could be easily recognized (high hit rates) but also confused as representing another (wrong) function at the same time. The “missing values” were also important since they indicated situations when respondents either did not know the answer or thought that none among the symbols were comprehensible or representative of a desired function. The preference tests pertained to aesthetics of the symbols individually and as a set. In turn, the SDT scores revealed that symbols could have different connotative meanings in relation to the functions they were intended to represent. Overall, Asian subjects performed comparably well with the European and American subjects, preferring the same set of videophone symbols, but usually at the expense of more errors and confusions. Thus, the studies showed that using multiple indices helped reveal subtle but potentially important differences in the results between different cultural groups.