Measurement of Importance in a Student Satisfaction Questionnaire: Comparison of the Direct and Indirect Methods for Establishing Attribute Importance

Abstract

Student dissatisfaction with certain aspects of college has been linked to attrition. Some developers of satisfaction surveys contend that in addition to satisfaction ratings, the questionnaire should contain an importance scale because dissatisfaction with the less important aspects of college is not as problematic as dissatisfaction with the more important aspects. Critics of importance ratings argue that they are unnecessary because respondents implicitly consider the importance of an attribute when formulating their satisfaction judgment. It may be possible to keep the length of the questionnaire shorter and still infer the importance of the attributes by examining the correlations between attributes and the measure of overall satisfaction. The attributes with the higher correlations with overall satisfaction are thought to be more important than attributes with low correlations with overall satisfaction. Questions remain about whether the relationship between importance and satisfaction exists and whether it is linear or non-linear in nature. The purpose of this study was to determine: 1) the type of relationship that exists between satisfaction and importance in a student satisfaction survey; and 2) the interchangeability of the direct and the inferred means of assessing attribute importance. It was found that in a student satisfaction survey, the relationship between satisfaction and importance is nonlinear. Attributes with very high or very low satisfaction are rated as more important than attributes with middle levels of satisfaction. The direct method of measuring importance was compared to three indirect methods (i.e., based on satisfaction). The three indirect methods differed in terms of how the overall satisfaction was determined (single item rating of general satisfaction, sum score of 25 attributes, and a question about whether the student would enroll at the same institution if given the chance to do it over again). The correlations between the direct and the indirect methods for determining attribute importance were at best only moderate. The indirect method most similar to the direct approach was the one-item general satisfaction question, but common method variance may be a factor in this. It was concluded that the direct and indirect methods should not be considered interchangeable. We recommended that future research determine which method is most valid by considering the relationship of each method to actual attrition.