Obtaining Paired Comparisons Data from Multiple Rank Orders Using Partially Balanced Incomplete Block Designs

Abstract

Complete paired comparisons data was obtained by the method of multiple rank order (MRO) in the context of gathering rank order preferences of grade six students, their parents, and their teachers for instructional objectives. Partially balanced incomplete block designs with two associate classes were used in the M RO instruments instead of the usual balanced incomplete block designs.

The use of partially balanced designs may yield several benefits to a researcher including a reduction in the number of blocks of stimuli to be ranked, a measure of the internal consistency of subject's choices, and a concentration of experimental effort upon comparisons of the most critical stimulus pairs. The benefits and the associated costs of using these designs are discussed in the light of the data obtained in the study.

It is recommended that whenever rank orders or paired comparisons data is called for in a study that serious consideration be given to the use of the MRO method. Furthermore, it is suggested that the overall purposes of a study may best be served by the use of a partially balanced rather than a balanced incomplete block design in the MRO method.