The researchers employed a confirmatory tetrad analysis (CTA) using partial least squares–structural equation modeling (PLS-SEM) with Outcome Questionnaire–45.2 (OQ-45) data, examining the measurement model of the OQ-45 scores with a sample of male adult clients (N = 1,558) receiving individual therapy at a university-based community counseling and research center (UBCCRC). Using CTA-PLS, this study examined the reflective and formative nature of each of the OQ-45 items and dimensions. These results identified the innovative second-order formative–formative three-factor model as a best alternative measurement model to represent and calculate the scores of OQ-45 scale.
BauldryS.BollenK. A. (2016). Tetrad: A set of Stata commands for confirmatory tetrad analysis. Structural Equation Modeling: A Multidisciplinary Journal, 23, 921-930.
2.
BentlerP. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238-246.
3.
BeretvasS. N.KearneyL. K. (2003). A shortened form of the Outcome Questionnaire: A validation of scores across groups (A Research Report of the Research Consortium of Counseling and Psychological Services for Higher Education). Austin: University of Texas at Austin, Counseling and Mental Health Center.
4.
BickmanL.SalzerM. S. (1997). Introduction: Measuring quality in mental health services. Evaluation Review, 21, 285-291. doi:10.1177/0193841X9702100301
5.
BludworthJ. L.TraceyT. G.Glidden-TraceyC. (2010). The bilevel structure of the Outcome Questionnaire-45. Psychological Assessment, 22, 350-355. doi:10.1037/a0019187
6.
BollenK. A.LennoxR. (1991). Conventional wisdom on measurement: A structural equation Perspective. Psychological Bulletin, 110, 305-134. http://dx.doi.org/10.1037/0033-2909.110.2.305
7.
BollenK. A.TingK. F. (2000). A tetrad test for causal indicators. Psychological Methods, 5, 3-22.
8.
BrowneM. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62-83.
9.
ChenF.SousaK. H.WestS. G. (2005). Teacher’s corner: Testing measurement invariance of second-order factor models. Sturctural Equation Modeling, 12:3, 471-492. doi: 10.1207/s15328007sem1203_7
10.
de JongK.NugterM. A.PolakM. G.WagenborgJ. E. A.SpinhovenP.HeiserW. J. (2007). The Outcome Questionnaire (OQ-45) in a Dutch population: A cross-cultural validation. Clinical Psychology & Psychotherapy, 14, 288-301. doi:10.1002/cpp.529
11.
EndersC. K. (2010). Applied missing data analysis. New York, NY: Guilford.
12.
GrahamJ. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549-576. doi:10.1146/annurev.psych.58.110405.085530
13.
GuderganS. P.RingleC. M.WendeS.WillA. (2008). Confirmatory tetrad analysis in PLS path modeling. Journal of Business Research, 61, 1238-1249.
14.
HairJ. F.BlackW. C.BabinB. J.AndersonR. E.TathamR. L. (2010). Multivariate data analysis: A global perspective. Upper Saddle River, NJ: Pearson.
15.
HairJ. F.HultG. T. M.RingleC. M.SarstedtM. (2017). A primer on partial least squares structural equation modeling (2nd ed.). Thousand Oaks, CA: Sage.
16.
HairJ. F.SarstedtM.HopkinsL. G.KuppelwieserV. (2014). Partial least squares structural equation modeling (PLS-SEM): An emerging tool in business research. European Business Review, 26, 106-121. doi:10.1108/EBR-10-2013-0128
17.
HairJ. F.SarstedtM.RingleC. M.GuderganS. P. (2018). Advanced issues in partial least squares structural equation modeling (PLS-SEM). Thousand Oaks, CA: Sage.
18.
HatfieldD. R.OglesB. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35, 485-491.
19.
HorowitzL. M.RosenbergS. E.BaerB. A.UrenoG.VillasenorV. S. (1988). Inventory of interpersonal problems: Psychometric properties and clinical applications. Journal of Consulting and Clinical Psychology, 56, 885-892.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.
22.
JarvisC. B.MacKenzieS. B.PodsakoffP. M. (2003). A critical review of construct indicators and measurement model misspecification in marketing and consumer research. Journal of Consumer Research, 30, 199-218. doi:10.1086/376806
23.
KimS.BeretvasN.SherryA. R. (2010). A validation of the factor structure of OQ-45 scores using factor mixture modeling. Measurement and Evaluation in Counseling and Development, 42, 275-295.
24.
KlineR. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press.
25.
KockN. (2017). WarpPLS User Manual: Version 6.0. Laredo, TX: Script Warp Systems.
26.
KockN.LynnG. S. (2012). Lateral collinearity and misleading results in variance-based SEM: An illustration and recommendations. Journal of the Association for Information Systems, 13, 546-580.
27.
LambertM. J. (1983). Introduction to assessment of psychotherapy outcomee: Historical perspective and current issues. In LambertM. J.ChristensenE. R.DejulioS. S. (Eds), The Assessment of Psychotherapy Outcome. New York: John Wiley.
28.
LambertM. J.BurlingameG. M.UmphressV.HansenN. B.VermeerschD. A.ClouseG. C.YancharS. C. (1996). The reliability and validity of the Outcome Questionnaire. Clinical Psychology and Psychotherapy, 3, 249-258.
29.
LambertM. J.GregersenA. T.BurlingameG. M. (2004). The Outcome Questionnaire. In MaruishM. E. (Ed.), The use of psychological testing for treatment planning and outcome assessment: Vol. 3: Instruments for adults (3rd ed., pp. 191-234). Mahwah, NJ: Lawrence Erlbaum.
30.
LambertM. J.HansenN. B.FinchA. E. (2001). Patient-focused research: Using patient outcome data to enhance treatment effects. Journal of Consulting and Clinical Psychology, 69, 159-172.
31.
LambertM. J.LunnenK.UmphressV.HansenN. B.BurlingameG. M. (1994). Administration and scoring manual for the Outcome Questionnaire (OQ-45.1). Salt Lake City, UT: IHC Center for Behavioral Healthcare Efficacy.
32.
LiY.LuoH. (2009). The reliability and validity of the Outcome Questionnaire-Chinese version. Chinese Mental Health Journal, 23, 105-107.
33.
LittleR. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83, 1198-1202. 10.1080/01621459.1988.10478722
34.
Lo CocoG.ChiappelliM.BensiL.GulloS.PrestanoC.LambertM. J. (2008). The factorial structure of the Outcome Questionnaire-45: A study with an Italian sample. Clinical Psychology & Psychotherapy, 15, 418-423. doi:10.1002/cpp.601
35.
MacKenzieS. B.PodsakoffP. M.JarvisC. (2005). The Problem of measurement model misspecification in behavioral and organizational research and some recommended solutions. Journal of Applied Psychology, 90, 710-730.
36.
MuellerR. M.LambertM. J.BurlingameG. M. (1998). Construct validity of the Outcome Questionnaire: A confirmatory factor analysis. Journal of Personality Assessment, 70, 248-262. doi:10.1207/s15327752jpa7002_5
37.
MvududuN. H.SinkC. A. (2013). Factor analysis in counseling research and practice. Counseling Outcome Research and Evaluation, 4, 75-98.
38.
PallantJ. (2016). SPSS survival manual: A step by step guide to data analysis using IBM SPSS (6th ed.). Maidenhead, Berkshire, England: McGraw Hill.
39.
OsborneJ. W. (2013). Best practices in data cleaning. Thousand Oaks, CA: Sage.
40.
RasoolimaneshS. M.NoorS. M.JaafarM. (2017). Positive and negative perceptions of residents toward tourism development: Formative or reflective. In RezaeiS. (Ed.), Quantitative tourism research in Asia: Current status and future directions. Singapore: Springer.
41.
RiceK. G.SuhH.EgeE. (2014). Further evaluation of the Outcome Questionnaire–45.2. Measurement and Evaluation in Counseling and Development, 47, 102-117.
42.
RigdonE. E. (2015). Structural equation modeling: Nontraditional alternatives. In BalakrishnanN.ColtonT.EverittB.PiegorschW.RuggeriF.TeugelsJ. L. (Eds.), Wiley StatsRef: Statistics reference online (pp. 1-12). Chichester, UK: John Wiley. doi:10.1002/9781118445112.stat06528.pub2
SarstedtM.HairJ. F.RingleC. M.ThieleK. O.GuderganS. P. (2016). Estimation issues with PLS and CBSEM: Where the bias lies!Journal of Business Research, 69, 3998-4010.
45.
SchumackerR. E.LomaxR. G. (2016). A beginner’s guide to structural equation modeling. New York, NY: Routledge.
46.
SteigerJ. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 21, 309-331.
47.
TabachnickB. G.FidellL. S. (2013). Using multivariate statistics (6th ed.). Upper Saddle River, NJ: Pearson.
48.
ThalmayerA. G. (2015). Alternative models of the Outcome Questionnaire-45. European Journal of Psychological Assessment, 31, 120-130. doi:10.1027/1015-5759/a000216
49.
TimmanR.de JongK.de Neve-EnthovenN. (2017). Cut-off scores and clinical change indices for the Dutch Outcome Questionnaire (OQ-45) in a large sample of normal and several psychotherapuetic populations. Clinical Psychology and Psychotherapy, 24, 72-81.
50.
TuckerL. R.LewisC. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10.
51.
VeitC. T.WareJ. E. (1983). The structure of psychological distress and well-being in general populations. Journal of Consulting and Clinical Psychology, 51, 730-742. doi:10.1037/0022-006X.51.5.730
52.
VermeerschD. A.WhippleJ. L.LambertM. J.HawkinsE. J.BurchfieldC. M.OkiishiJ. C. (2004). Outcome Questionnaire: Is it sensitive to changes in counseling center clients?Journal of Counseling Psychology, 51, 38-49. doi:10.1037/0022-0167.51.1.38
53.
WennbergP.PhilipsB.de JongK. (2010). The Swedish version of the Outcome Questionnaire (OQ-45): Reliability and factor structure in a substance abuse sample. Psychology and Psychotherapy: Theory, Research and Practice, 83, 325-329.
54.
WilloughbyM.HolochwostS. J.BlantonZ. E.BlairC. B. (2014). Executive functions: Formative versus reflective measurement. Measurement: Interdisciplinary Research & Perspectives, 12(3), 69-95.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
