Techniques for Assessing Safety Symbol Comprehension: Web-Based vs. In-Person Questionnaire Administration

Abstract

This study compares methods for testing comprehension of a set of three safety symbols – in-person questionnaire administration and three variations of a web-based questionnaire using similar protocols. Both the medium and the particular questions asked were varied. The in-person method included follow-up and probing questions and used questions about the individual symbols only if needed to elicit complete answers. The web-based questionnaires omitted follow-up and probing questions, and always included questions about the individual symbols. The second variation of the web-based questionnaire added an example of a “good/poor answer” as shown in ANSI Z535.3 Annex B. The third included the example answer and an additional question regarding consequences. The in-person method measured 90% correct comprehension with 2% critical confusions. Comprehension measured by each of the three web-based methods did not significantly differ from the in-person method. The comparison of different web-based questionnaires indicates that for web-based testing, asking about individual symbols within a symbol set, providing an example of a good/poor answer, and/or including an additional question regarding consequences can improve the method's ability to capture comprehension data. These results suggest that under some circumstances, a web-based questionnaire could substitute for in-person comprehension testing.