Eliminating Order Effects in Association Tasks without using Randomisation

Abstract

It has often been observed that changing an item's position in a list can substantially affect the probability that it is chosen. This paper assesses the magnitude of these so-called order effects in brand-attribute association tasks, and examines the confounding roles played by brand usage and question framing. While our main order effect is roughly the same as that observed for similar response formats, we find substantially larger order effects among users of a brand than non-users; and question frames that first ask respondents to create an attribute shortlist before making associations on this reduced set eliminate or greatly reduce the magnitude of the order effect and its interaction with brand usage. These simple modifications to question framings may be useful where randomisation is not feasible.