The Dimensionality of Measures Derived from a Nationally Standardized Written Clinical Simulation Examination

Abstract

The literature reflects substantial controversy regarding the dimensionality of various scores derived from a variety of patient management problem tests. The purposes of this study were to determine (1) the dimensionality of 20 problem score measures derived from a nationally standardized written clinical simulation examination and (2) the stability and generalizability of the resulting factor structure across both examinee groups and test forms. Test score data from two independent national samples that completed the same test form were obtained (Ni = 2,812; N2 = 312); a third independent national sample completed a different test form (N3 = 2,546). Each of the three 20 x 20 intercorrelations was subjected to confirmatory factor analysis in an attempt to validate a two-dimensional solution, but such a solution could not be substantiated. Subsequent exploratory factor analyses yielded consistent one-factor solutions for each of the three data sets, demonstrating generalizability and stability of this one-factor structure across both different groups of examinees and different tests forms. This study clearly demonstrated that for the examination under study, there is only one underlying dimension rather than two as suggested in the literature and reflected in the existing test scoring practice.