We present the SEMWISE (structural equation modeling for within-subject experiments) approach for analyzing policy capturing data. Policy capturing entails estimating the weights (or utilities) of experimentally manipulated attributes in predicting a response variable of interest (e.g., the effect of experimentally manipulated market-technology combination characteristics on perceived entrepreneurial opportunity). In the SEMWISE approach, a factor model is specified in which latent weight factors capture individually varying effects of experimentally manipulated attributes on the response variable. We describe the core SEMWISE model and propose several extensions (how to incorporate nonbinary attributes and interactions, model multiple indicators of the response variable, relate the latent weight factors to antecedents and/or consequences, and simultaneously investigate several populations of respondents). The primary advantage of the SEMWISE approach is that it facilitates the integration of individually varying policy capturing weights into a broader nomological network while accounting for measurement error. We illustrate the approach with two empirical examples, compare and contrast the SEMWISE approach with multilevel modeling (MLM), discuss how researchers can choose between SEMWISE and MLM, and provide implementation guidelines.
AguinisH.BradleyK. J. (2014). Best practice recommendations for designing and implementing experimental vignette methodology studies. Organizational Research Methods, 17(4), 351–371. doi:10.1177/1094428114547952
2.
Aiman-SmithL.BauerT. N.CableD. M. (2001). Are you attracted? Do you intend to pursue? A recruiting policy-capturing study. Journal of Business and Psychology, 16(2), 219–237. doi:10.1023/A:1011157116322
3.
Aiman-SmithL.ScullenS. E.BarrS. H. (2002). Conducting studies of decision making in organizational contexts: A tutorial for policy-capturing and other regression-based techniques. Organizational Research Methods, 5(4), 388–414. doi:10.1177/109442802237117
4.
BollenK. A. (1989). Structural equations with latent variables. New York, NY: Wiley.
5.
CarusoE. M.RahnevD. A.BanajiM. R. (2009). Using conjoint analysis to detect discrimination: Revealing covert preferences from overt choices. Social Cognition, 27(1), 128–137. doi:10.1521/soco.2009.27.1.128
6.
CatlinJ. R.WangY. (2013). Recycling gone bad: When the option to recycle increases resource consumption. Journal of Consumer Psychology, 23(1), 122=127. doi:10.1016/j.jcps.2012.04.001
7.
CheungG. W.RensvoldR. B. (2001). The effects of model parsimony and sampling error on the fit of structural equation models. Organizational Research Methods, 4(3), 236–264. doi:10.1177/109442810143004
8.
ConnellyB. L.KetchenD. J.GangloffK. A.ShookC. L. (2016). Investor perceptions of CEO successor selection in the wake of integrity and competence failures: A policy capturing study. Strategic Management Journal, 37(10), 2135–2151. doi:10.1002/smj.2430
9.
CuddyA. J.GlickP.BeningerA. (2011). The dynamics of warmth and competence judgments, and their outcomes in organizations. Research in Organizational Behavior, 31, 73–98. doi:10.1016/j.riob.2011.10.004
10.
CurranP. J. (2003). Have multilevel models been structural equation models all along?Multivariate Behavioral Research, 38(4), 529–568. doi:10.1207/s15327906mbr3804_5
11.
EdwardsJ. R. (2001). Multidimensional constructs in organizational behavior research: An integrative analytical framework. Organizational Research Methods, 4(2), 144–192. doi:10.1177/109442810142004
12.
FeingoldA. (2009). Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychological Methods, 14(1), 45–53. doi:10.1037/a0014699
13.
FinkB.Penton-VoakI. (2002). Evolutionary psychology of facial attractiveness. Current Directions in Psychological Science, 11(5), 154–158. doi:10.1111/1467-8721.00190
14.
FiskeS. T.CuddyA. J.GlickP.XuJ. (2002). A model of (often mixed) stereotype content: Competence and warmth respectively follow from perceived status and competition. Journal of Personality and Social Psychology, 82(6), 878–902. doi:10.1037//0022-3514.82.6.878
15.
GelmanA.HillJ. (2007). Data analysis using regression and multilevelhierarchical models (Vol. 1). New York, NY: Cambridge University Press.
16.
GrahamM. E.CableD. M. (2001). Consideration of the incomplete block design for policy-capturing research. Organizational Research Methods, 4(1), 26–45. doi:10.1177/109442810141002
17.
GrégoireD. A.ShepherdD. A. (2012). Technology-market combinations and the identification of entrepreneurial opportunities: An investigation of the opportunity-individual nexus. Academy of Management Journal, 55(4), 753–785. doi:10.5465/amj.2011.0126
18.
HelmJ. L.Castro-SchiloL.OraveczZ. (2017). Bayesian versus maximum likelihood estimation of multitrait-multimethod confirmatory factor models. Structural Equation Modeling: A Multidisciplinary Journal, 24(1), 17–30. doi:10.1080/10705511.2016.1236261
19.
HinkinT. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1), 104–121. doi:10.1177/109442819800100106
20.
JonesD. A.WillnessC. R. (2013). Corporate social performance, organizational reputation, and recruitment. In CableD. M.YuK. Y. T. (Eds.), The Oxford handbook of recruitment (pp. 298–313). Oxford, UK: Oxford University Press.
21.
KarrenR. J.BarringerM. W. (2002). A review and analysis of the policy-capturing methodology in organizational research: Guidelines for research and practice. Organizational Research Methods, 5(4), 337–361. doi:10.1177/109442802237115
22.
KlineR. B. (2016). Principles and practice of structural equation modeling (4th ed.). New York, NY: The Guilford Press.
23.
Kristof-BrownA. L.JansenK. J.ColbertA. E. (2002). A policy-capturing study of the simultaneous effects of fit with jobs, groups, and organizations. Journal of Applied Psychology, 87(5), 985–993. doi:10.1037//0021-9010.87.5.985
24.
LaHuisD. M.HartmanM. J.HakoyamaS.ClarkP. C. (2014). Explained variance measures for multilevel models. Organizational Research Methods, 17(4), 433–451. doi:10.1177/1094428114541701
25.
LanceC. E.ButtsM. M.MichelsL. C. (2006). The sources of four commonly reported cutoff criteria—What did they really say?Organizational Research Methods, 9(2), 202–220. doi:10.1177/1094428105284919
26.
LanceC. E.NobleC. L.ScullenS. E. (2002). A critique of the correlated trait-correlated method and correlated uniqueness models for multitrait-multimethod data. Psychological Methods, 7(2), 228–244. doi:10.1037/1082-989X.7.2.228
27.
MaD. S.CorrellJ.WittenbrinkB. (2015). The Chicago face database: A free stimulus set of faces and norming data. Behavior Research Methods, 47(4), 1122–1135. doi:10.3758/s13428-014-0532-5
28.
Maydeu-OlivaresA.CoffmanD. L. (2006). Random intercept item factor analysis. Psychological Methods, 11(4), 344–362. doi:10.1037/1082-989X.11.4.344
29.
MulaikS. A. (2009). Linear causal modeling with structural equations. Washington, DC: CRC Press.
30.
MuthénL. K.MuthénB. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9(4), 599–620. doi:10.1207/S15328007SEM0904_8
31.
NewmanD. A.LyonJ. S. (2009). Recruitment efforts to reduce adverse impact: Targeted recruiting for personality, cognitive ability, and diversity. Journal of Applied Psychology, 94(2), 298–317. doi:10.1037/a0013472
32.
OlsenJ. E.MartinsL. L. (2016). Racioethnicity, community makeup, and potential employees’ reactions to organizational diversity management approaches. Journal of Applied Psychology, 101(5), 657–672. doi:10.1037/apl0000080
33.
PatzeltH.ShepherdD. A. (2008). The decision to persist with underperforming alliances: The role of trust and control. Journal of Management Studies, 45(7), 1217–1243. doi:10.1111/j.1467-6486.2008.00791.x
34.
PaulhusD. L. (1991). Measurement and control of response bias. In RobinsonJ. P.ShaverP. R.WrightsmanL. S. (Eds.), Measures of personality and social psychological attitudes (pp. 17–60). San Diego, CA: Academic Press.
35.
PloyhartR. E.OswaldF. L. (2004). Applications of mean and covariance structure analysis: Integrating correlational and experimental approaches. Organizational Research Methods, 7(1), 27–65.
36.
RebJ.CropanzanoR. (2007). Evaluating dynamic performance: The influence of salient gestalt characteristics on performance ratings. Journal of Applied Psychology, 92(2), 490–499. doi:10.1037/0021-9010.92.2.490
37.
RhemtullaM. (2016). Population performance of SEM parceling strategies under measurement and structural model misspecification. Psychological Methods, 21(3), 348–368. doi:10.1037/met0000072
38.
RosmanA.LubatkinM.O’NeillH. (1994). Rigidity in decision behaviors: A within-subject test of information acquisition using strategic and financial informational cues. Academy of Management Journal, 37(4), 1017–1033. doi:10.2307/256610
39.
RotundoM.SackettP. R. (2002). The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach. Journal of Applied Psychology, 87(1), 66–80. doi:10.1037/0021-9010.87.1.66
40.
SauerC.AuspurgK.HinzT.LiebigS. (2011). The application of factorial surveys in general population samples: The effects of respondent age and education on response times and response consistency. Survey Research Methods, 5(3), 89–102. doi:10.18148/srm/2011.v5i3.4625#sthash.YYreWVGE.dpuf
41.
ShepherdD. A.PatzeltH.BaronR. A. (2013). “I Care about nature, but…”: Disengaging values in assessing opportunities that cause harm. Academy of Management Journal, 56(5), 1251–1273. doi:10.5465/amj.2011.0776
42.
TomassettiA. J.DalalR. S.KaplanS. A. (2016). Is policy capturing really more resistant than traditional self-report techniques to socially desirable responding?Organizational Research Methods, 19(2), 255–285. doi:10.1177/1094428115627497
43.
VandenbergR. J.LanceC. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. doi:10.1177/109442810031002
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
