Structural Properties and Psychometric Qualities of Organizational Self-Reports: Field Tests of Connections Predicted by Cognitive Theory

Abstract

Recent cognitive theories about question interpretation, information retrieval, and judgment-response mapping were used to develop hypotheses about how structural properties of the context of self-report items affects psychometric qualities. The hypotheses were tested in three experiments. In Study 1, a field experiment with 366 security firm employees, manipulation of the evaluative (positive vs. null vs. negative) context of a neutral item embedded in a job affect measure yielded differences in item parameters and correlations with other constructs. In Study 2, a field experiment with 392 chemical processing employees, manipulation of the physical context (“boxed” random vs. uniform groupings) of job attitude and perception items produced differences in internal consistencies and discriminant validities, as well as differences in measurement and structural models. In Study 3, an experiment with 187 undergraduate business students, manipulation of response contexts (low vs. open-ended vs. high ranges of response options) produced differences in reported frequencies of stress-related health symptoms and in correlations with other constructs. Implications are discussed with regard to designing and administering self-report items on organizational surveys.