Developing a Theory of Two Latent Soft Skills Progress Variables using the BEAR Assessment System: Validity Evidence for the Internal Structure of the Social Evaluative in the Workplace Instrument

Abstract

In this article, we report on the development of two latent soft skills progress variables using the Berkeley Evaluation and Assessment Research (BEAR) Assessment System (BAS). The Social Evaluative Reasoning in the Workplace (SER-W) instrument uses comic strip scenarios to depict interactions between employees and customers in entry-level workplace settings. We designed items to elicit evidence of student ability to: (a) identify salient customer social cues, which we term the social cue detection (SPU) variable, and, (b) justify an evaluation on the outcome of the situation depicted in the scenarios. We refer to this as the evaluative inference (EI) variable. Research from the field of autism spectrum disorder was used to develop a theory for building social complexity into the SER-W comic strip scenarios, by manipulating the type, frequency, and co-occurrence of the social cues presented in the scenarios. A unidimensional and multidimensional extension of the Rasch partial credit model were fit to the data. Model comparisons provide empirical support for our hypothesized two-dimensional structure, in which the SPU and EI variables are modeled as separate dimensions. These results are considered in terms of the evidence for the validity of the internal structure of the SER-W dimensions they provide. The article concludes with examples of the practical implications that progress variable research can have on soft skills curriculum development and assessment in the field of special education.