Factor Structure,Measurement Invariance,and Concurrent Validity of the Penn State Worry Questionnaire Across Development,Psychopathology,and Culture

Abstract

The Penn State Worry Questionnaire (PSWQ) is a widely used assessment of excessive worry. American undergraduate samples have predominately been used to evaluate its factor structure, which may not generalize to other developmental, cultural, and psychopathology populations. The present study tested the PSWQ’s factor structure across three diverse samples: American undergraduate students (n = 3,243), Dutch high school students (n = 3,906), and American adults with psychopathology (n = 384). Exploratory, confirmatory, and multigroup confirmatory factor analyses were conducted. Measurement invariance and concurrent validity were also tested. Method-factor and two-factor models were largely equivalent and superior to a one-factor model. Invariance tests supported configural and metric invariance but only partial scalar invariance. Positively worded items but not negatively worded items demonstrated concurrent validity with anxiety and depression symptom measures and diagnoses. Overall, the PSWQ appears to measure a unitary construct. Present results warrant further testing of the PSWQ across diverse samples.

Keywords

Penn State Worry Questionnaire anxiety depression factor structure measurement invariance

Get full access to this article

View all access options for this article.

References

Akaike

(1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716-723. https://doi.org/10.1109/TAC.1974.1100705

Barahmand

(2008). Age and gender differences in adolescent worry. Personality and Individual Differences, 45(8), 778-783. https://doi.org/10.1016/j.paid.2008.08.006

Basevitz

Pushkar

Chaikelson

Conway

Dalton

(2008). Age-related differences in worry and related processes. International Journal of Aging and Human Development, 66(4), 283-305. https://doi.org/10.2190/AG.66.4.b

Beck

J. G.

Stanley

M. A.

Zebb

B. J.

(1995). Psychometric properties of the Penn State Worry Questionnaire in older adults. Journal of Clinical Geropsychology, 1(1), 33-42. https://www.researchgate.net/publication/232439962_Psychometric_properties_of_the_Penn_State_Worry_Questionnaire_in_older_adults

Bentler

P. M.

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238-246. https://doi.org/10.1037/0033-2909.107.2.238

Bentler

P. M.

(1995). EQS structural equations program manual (Vol. 6). Multivariate Software.

Bredemeier

Spielberg

J. M.

Silton

R. L.

Berenbaum

Heller

Miller

G. A.

(2010). Screening for depressive disorders using the Mood and Anxiety Symptoms Questionnaire Anhedonic Depression Scale: A receiver-operating characteristic analysis. Psychological Assessment, 22(3), 702-710. https://doi.org/10.1037/a0019915

Bredemeier

Warren

S. L.

Berenbaum

Miller

G. A.

Heller

(2016). Executive function deficits associated with current and past major depressive symptoms. Journal of Affective Disorders, 204(November), 226-233. https://doi.org/10.1016/j.jad.2016.03.070

Brown

T. A.

(2003). Confirmatory factor analysis of the Penn State Worry Questionnaire: Multiple factors or method effects? Behaviour Research and Therapy, 41(12), 1411-1426. https://doi.org/10.1016/s0005-7967(03)00059-7

10.

Brown

T. A.

Antony

M. M.

Barlow

D. H.

(1992). Psychometric properties of the Penn State Worry Questionnaire in a clinical anxiety disorders sample. Behaviour Research and Therapy, 30(1), 33-37. https://doi.org/10.1016/0005-7967(92)90093-v

11.

Buckby

J. A.

Yung

A. R.

Cosgrave

E. M.

Killackey

E. J.

(2007). Clinical utility of the Mood and Anxiety Symptom Questionnaire (MASQ) in a sample of young help-seekers. BMC Psychiatry, 7(1), 1-7. https://doi.org/10.1186/1471-244X-7-50

12.

Castillo

Macrini

Cheniaux

Landeira-Fernandez

(2010). Psychometric properties and latent structure of the Portuguese version of the Penn State Worry Questionnaire. Spanish Journal of Psychology, 13(1), 431-443. https://doi.org/10.1017/s113874160000398x

13.

Cattell

R. B.

(1978). The scientific use of factor analysis in behavioral and life sciences. Plenum Press. https://doi.org/10.1007/978-1-4684-2262-7

14.

Chen

F. F.

(2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), 464-504. https://doi.org/10.1080/10705510701301834

15.

Chorpita

B. F.

Tracey

S. A.

Brown

T. A.

Collica

T. J.

Barlow

D. H.

(1997). Assessment of worry in children and adolescents: An adaptation of the Penn State worry questionnaire. Behaviour Research and Therapy, 35(6), 569-581. https://doi.org/10.1016/S0005-7967(96)00116-7

16.

Crocker

L. D.

Heller

Spielberg

J. M.

Warren

S. L.

Bredemeier

Sutton

B. P.

Banich

M. T.

Miller

G. A.

(2012). Neural mechanisms of attentional control differentiate trait and state negative affect. Frontiers in Psychology, 3, Article 298. https://doi.org/10.3389/fpsyg.2012.00298

17.

De Beuckelaer

Swinnen

. (2011). Biased latent variable mean comparisons due to measurement non-invariance: A simulation study. In Davidov

Schmidt

Billiet

(Eds.), Cross-cultural analysis: Methods and applications (pp. 117-148). Taylor & Francis Group.

18.

Ehring

Watkins

E. R.

(2008). Repetitive negative thinking as a transdiagnostic process. International Journal of Cognitive Therapy, 1(3), 192-205. https://doi.org/10.1521/ijct.2008.1.3.192

19.

Engels

A. S.

Heller

Spielberg

J. M.

Warren

S. L.

Sutton

B. P.

Banich

M. T.

Miller

G. A.

(2010). Co-occurring anxiety influences patterns of brain activity in depression. Cognitive, Affective, & Behavioral Neuroscience, 10(1), 141-156. https://doi.org/10.3758/CABN.10.1.141

20.

First

M. B.

Spitzer

R. L.

Gibbon

Williams

J. B. W.

(2002). SCID-I/NP: Structured clinical interview for DSM-IV-TR Axis I disorders, research version, non-patient edition. Biometrics Research, New York State Psychiatric Institute.

21.

Fresco

D. M.

Heimberg

R. G.

Mennin

D. S.

Turk

C. L.

(2002). Confirmatory factor analysis of the Penn State Worry Questionnaire. Behaviour Research and Therapy, 40(3), 313-323. https://doi.org/10.1016/S0005-7967(00)00113-3

22.

George

Mallery

(2003). SPSS for Windows step by step: A simple guide and reference. 11.0 update (4th ed.). Allyn & Bacon.

23.

Gladstone

G. L.

Parker

G. B.

Mitchell

P. B.

Malhi

G. S.

Wilhelm

K. A.

Austin

M.-P.

(2005). A Brief Measure of Worry Severity (BMWS): Personality and clinical correlates of severe worriers. Journal of Anxiety Disorders, 19(8), 877-892. https://doi.org/10.1016/j.janxdis.2004.11.003

24.

González

Ibáñez

Barrera

(2017). Rumination, worry and negative problem orientation: Transdiagnostic processes of anxiety, eating behavior and mood disorders. Acta Colombiana de Psicología, 20(2), 42-52. http://dx.doi.org/10.14718/ACP.2017.20.2.3

25.

Gorsuch

(1983). Factor analysis. Lawrence Erlbaum Associates.

26.

Gouin

J. P.

Hantsoo

Kiecolt-Glaser

J. K.

(2008). Immune dysregulation and chronic stress among older adults: A review. Neuroimmunomodulation, 15(4-6), 251-259. https://doi.org/10.1159/000156468

27.

Hazlett-Stevens

Ullman

J. B.

Craske

M. G.

(2004). Factor structure of the Penn State Worry Questionnaire: Examination of a method factor. Assessment, 11(4), 361-370. https://doi.org/10.1177/1073191104269872

28.

Hong

R. Y.

Riskind

J. H.

Cheung

M. W.-L.

Calvete

González-Díez

Atalay

A. A.

Curzik

Jokic-Begic

del Palacio-Gonzalez

Mihić

Samac

Sica

Sugiura

Khatri

Kleiman

E. M.

(2017). The Looming Maladaptive Style Questionnaire: Measurement invariance and relations to anxiety and depression across 10 countries. Journal of Anxiety Disorders, 49(June), 1-11. https://doi.org/10.1016/j.janxdis.2017.03.004

29.

Hopko

D. R.

Stanley

M. A.

Reas

D. L.

Wetherell

J. L.

Beck

J. G.

Novy

D. M.

Averill

P. M.

(2003). Assessing worry in older adults: Confirmatory factor analysis of the Penn State Worry Questionnaire and psychometric properties of an abbreviated model. Psychological Assessment, 15(2), 173-183. https://doi.org/10.1037/1040-3590.15.2.173

30.

Hur

Heller

Kern

J. L.

Berenbaum

(2017). A bi-factor approach to modeling the structure of worry and rumination. Journal of Experimental Psychopathology, 8(3), 252-264. https://doi.org/10.5127/jep.057116

31.

Jeon

J. W.

Kim

Roh

(2017). Exploratory and confirmatory factor analysis of the Korean version of the Penn State Worry Questionnaire. Anxiety and Mood, 13(2), 86-92. https://www.koreascience.or.kr/article/JAKO201734158359254.page

32.

Keogh

Reidy

(2000). Exploring the factor structure of the Mood and Anxiety Symptom Questionnaire (MASQ). Journal of Personality Assessment, 74(1), 106-125. https://doi.org/10.1207/S15327752JPA740108

33.

Leys

Delacre

Mora

Y. L.

Lakens

Ley

(2019). How to classify, detect, and manage univariate and multivariate outliers, with emphasis on pre-registration. International Review of Social Psychology, 32(1), 5. http://doi.org/10.5334/irsp.289

34.

Lim

Y. J.

Kim

Y.-H.

Lee

E.-H.

Kwon

S.-M.

(2008). The Penn State Worry Questionnaire: Psychometric properties of the Korean version. Depression & Anxiety, 25(10), E97-E103. https://doi.org/10.1002/da.20356

35.

Liu

Zhong

(2020). Structure and psychometric properties of the Penn State Worry Questionnaire for Children in Chinese adolescents. Journal of Abnormal Child Psychology, 48(11), 1499-1510. https://doi.org/10.1007/s10802-020-00680-1

36.

Madian

Bredemeier

Heller

Miller

G. A.

Warren

S. L.

(2019). Repetitive negative thought and executive dysfunction: An interactive pathway to emotional distress. Cognitive Therapy and Research, 43(2), 464-480. https://doi.org/10.1007/s10608-018-9966-x

37.

Marsh

H. W.

(1986). The bias of negatively worded items in rating scales for young children: A cognitive-developmental phenomenon. Developmental Psychology, 22(1), 37-49. https://doi.org/10.1037/0012-1649.22.1.37

38.

Marsh

H. W.

(1996). Positive and negative global self-esteem: A substantively meaningful distinction or artifactors? Journal of Personality and Social Psychology, 70(4), 810-819. https://doi.org/10.1037//0022-3514.70.4.810

39.

McEvoy

P. M.

Watson

Watkins

E. R.

Nathan

(2013). The relationship between worry, rumination, and comorbidity: Evidence for repetitive negative thinking as a transdiagnostic construct. Journal of Affective Disorders, 151(1), 313-320. https://doi.org/10.1016/j.jad.2013.06.014

40.

Meyer

T. J.

Miller

M. L.

Metzger

R. L.

Borkovec

T. D.

(1990). Development and validation of the Penn State Worry Questionnaire. Behaviour Research and Therapy, 28(6), 487-495. https://doi.org/10.1016/0005-7967(90)90135-6

41.

Milfont

T. L.

Fischer

(2010). Testing measurement invariance across groups: Applications in cross-cultural research. International Journal of Psychological Research, 3(1). https://doi.org/10.21500/20112084.857

42.

Muris

Roelofs

Rassin

Franken

Mayer

(2005). Mediating effects of rumination and worry on the links between neuroticism, anxiety and depression. Personality and Individual Differences, 39(6), 1105-1111. https://doi.org/10.1016/j.paid.2005.04.005

43.

Nuevo

Mackintosh

M. A.

Gatz

Montorio

Wetherell

J. L.

(2007). A test of the measurement invariance of a brief version of the Penn State Worry Questionnaire between American and Spanish older adults. International Psychogeriatrics, 19(1), 89-101. https://doi.org/10.1017/S1041610206003450

44.

Nunnally

J. C.

Bernstein

I. H.

(1994). Psychometric theory. McGraw-Hill.

45.

Olatunji

B. O.

Schottenbauer

M. A.

Rodriguez

B. F.

Glass

C. R.

Arnkoff

D. B.

(2007). The structure of worry: Relations between positive/negative personality characteristics and the Penn State Worry Questionnaire. Journal of Anxiety Disorders, 21(4), 540-553. https://doi.org/10.1016/j.janxdis.2006.08.005

46.

Pallesen

Nordhus

I. H.

Carlstedt

Thayer

J. F.

Johnsen

T. B.

(2006). A Norwegian adaptation of the Penn State Worry Questionnaire: Factor structure, reliability, validity and norms. Scandinavian Journal of Psychology, 47(4), 281-291. https://doi.org/10.1111/j.1467-9450.2006.00518.x

47.

Păsărelu

C. R.

Dobrean

Balazsi

Predescu

Şipos

Lupu

(2017). The Penn State Worry Questionnaire for Children: Age, gender and clinical Invariance. Child Psychiatry & Human Development, 48(3), 359-369. https://doi.org/10.1007/s10578-016-0663-2

48.

Rosenberg

(1965). Society and the adolescent self-image. Princeton University Press.

49.

Rosseel

(2012). lavaan: An R Package for structural equation modeling. Journal of Statistical Software, 48(2), 1-36. https://doi.org/10.18637/jss.v048.i02

50.

Ruiz

F. J.

Monroy-Cifuentes

Suárez-Falcón

J. C.

(2018). Penn State Worry Questionnaire-11 validity in Colombia and factorial equivalence across gender and nonclinical and clinical samples. Anales De Psicología/Annals of Psychology, 34(3), 451-457. https://doi.org/10.6018/analesps.34.3.300281

51.

Sadeh

N. S.

Spielberg

J. M.

Warren

S. L.

Miller

G. A.

Heller

(2014). Aberrant neural connectivity during emotional processing associated with posttraumatic stress. Clinical Psychological Science, 2(6), 748-755. https://doi.org/10.1177/2167702614530113

52.

Satorra

(2000). Scaled and adjusted restricted tests in multi-sample analysis of moment structures. In Heijmans

R. D. H.

Pollock

D. S. G.

Satorra

(Eds.), Innovations in multivariate statistical analysis: A Festschrift for Heinz Neudecker (pp. 233-247). Springer. https://doi.org/10.1007/978-1-4615-4603-0_17

53.

Satorra

Bentler

P. M.

(1994). Corrections to test statistics and standard errors in covariance structure analysis. In von Eye

Clogg

C. C.

(Eds.), Latent variables analysis: Applications for developmental research (pp. 399-419). Sage.

54.

Schwarz

(1978). Estimating the dimension of a model. Annals of Statistics, 6(2), 461-464. https://doi.org/10.1214/aos/1176344136

55.

Silton

R. L.

Heller

Engels

A. S.

Towers

D. N.

Spielberg

J. M.

Edgar

J. C.

Sass

S. M.

Stewart

J. L.

Sutton

B. P.

Banich

M. T.

Miller

G. A.

(2011). Depression and anxious apprehension distinguish frontocingulate cortical activity during top-down attentional control. Journal of Abnormal Psychology, 120(2), 272-285. https://doi.org/10.1037/a0023204

56.

Silton

R. L.

Heller

Towers

D. N.

Engels

A. S.

Spielberg

J. M.

Edgar

J. C.

Sass

S. M.

Stewart

J. L.

Sutton

B. P.

Banich

M. T.

Miller

G. A.

(2010). The time course of activity in dorsolateral prefrontal cortex and anterior cingulate cortex during top-down attentional control. NeuroImage, 50(3), 1292-1302. https://doi.org/10.1016/j.neuroimage.2009.12.061

57.

Spielberg

J. M.

Heller

Silton

R. L.

Stewart

J. L.

Miller

G. A.

(2011). Approach and avoidance profiles distinguish dimensions of anxiety and depression. Cognitive Therapy and Research, 35(4), 359-371. https://doi.org/10.1007/s10608-011-9364-0

58.

Spielberg

J. M.

Miller

G. A.

Engels

A. S.

Herrington

J. D.

Sutton

B. P.

Banich

M. T.

Heller

(2011). Trait approach and avoidance motivation: Lateralized neural activity associated with executive function. Neuroimage, 54(1), 661-670. https://doi.org/10.1016/j.neuroimage.2010.08.037

59.

Spielberg

J. M.

Miller

G. A.

Warren

S. L.

Engels

A. S.

Crocker

L. D.

Sutton

B. P.

Heller

(2012). Trait motivation moderates neural activation associated with goal pursuit. Cognitive, Affective, & Behavioral Neuroscience, 12(2), 308-322. https://doi.org/10.3758/s13415-012-0088-8

60.

Spielberg

J. M.

Miller

G. A.

Warren

S. L.

Sutton

B. P.

Banich

Heller

(2014). Transdiagnostic dimensions of anxiety and depression moderate motivation-related brain networks during goal maintenance. Depression & Anxiety, 31(10), 805-813. https://doi.org/10.1002/da.22271

61.

Stark

Chernyshenko

O. S.

Drasgow

(2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91(6), 1292-1306. https://doi.org/10.1037/0021-9010.91.6.1292

62.

Startup

Lavender

Oldershaw

Stott

Tchanturia

Treasure

Schmidt

(2013). Worry and rumination in anorexia nervosa. Behavioural and Cognitive Psychotherapy, 41(3), 301-316. https://doi.org/10.1017/S1352465812000847

63.

Steiger

J. H.

Lind

J. C.

(1980). Statistically based tests for the number of common factors [Paper presentation]. Annual Meeting of the Psychometric Society, Iowa City, IA.

64.

Stöber

(1995). Besorgnis: Ein Vergleich dreier Inventare zur Erfassung allgemeiner Sorgen. [Worrying: A comparison of three questionnaires concerning everyday worries]. Zeitschrift Für Differentielle Und Diagnostische Psychologie, 16(1), 50-63. https://www.researchgate.net/profile/Joachim_Stoeber/publication/232500263_Besorgnis_Ein_Vergleich_dreier_Inventare_zur_Erfassung_allgemeiner_Sorgen/links/09e4151370f7813928000000/Besorgnis-Ein-Vergleich-dreier-Inventare-zur-Erfassung-allgemeiner-Sorgen.pdf

65.

Tang

J. Y.

Rodgers

R. F.

Ballou

(2016). Ethnic differences in the effectiveness of cognitive behavioral therapy combined with medication: Comparing Asian American and white psychiatric patients. Asian Journal of Psychiatry, 24(December), 46-50. https://doi.org/10.1016/j.ajp.2016.08.012

66.

Titus

C. E.

DeShong

H. L.

(2020). Thought control strategies as predictors of borderline personality disorder and suicide risk. Journal of Affective Disorders, 266(April), 349-355. https://doi.org/10.1016/j.jad.2020.01.163

67.

Topper

Emmelkamp

P. M. G.

Watkins

Ehring

(2014). Development and assessment of brief versions of the Penn State Worry Questionnaire and the Ruminative Response Scale. British Journal of Clinical Psychology, 53(4), 402-421. https://doi.org/10.1111/bjc.12052

68.

Tucker

L. R.

Lewis

(1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1-10. https://doi.org/10.1007/BF02291170

69.

Tully

P. J.

Cosh

S. M.

Baune

B. T.

(2013). A review of the affects of worry and generalized anxiety disorder upon cardiovascular health and coronary heart disease. Psychology, Health & Medicine, 18(6), 627-644. https://doi.org/10.1080/13548506.2012.749355

70.

van Buuren

Groothuis-Oudshoorn

. (2011). mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1-67. https://doi.org/10.18637/jss.v045.i03

71.

van de Schoot

Lugtig

Hox

(2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486-492. https://doi.org/10.1080/17405629.2012.686740

72.

van Rijsoort

Emmelkamp

Vervaeke

. (1999). The Penn State Worry Questionnaire and the Worry Domains Questionnaire: Structure, reliability and validity. Clinical Psychology & Psychotherapy, 6(4), 297-307. https://doi.org/10.1002/(SICI)1099-0879(199910)6:4<297::AID-CPP206>3.0.CO;2-E

73.

Vuong

Q. H.

(1989). Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica, 57(2), 307-333. https://doi.org/10.2307/1912557

74.

Warren

S. L.

Bost

K. K.

Roisman

G. I.

Silton

R. L.

Spielberg

J. M.

Engels

A. S.

Choi

Sutton

B. P.

Miller

G. A.

Heller

(2010). Effects of adult attachment and emotional distractors on brain mechanisms of cognitive control. Psychological Science, 21(12), 1818-1826. https://doi.org/10.1177/0956797610388809

75.

Warren

S. L.

Crocker

L. D.

Spielberg

J. M.

Engels

A. S.

Banich

M. T.

Sutton

Miller

G. A.

Heller

(2013). Cortical organization of inhibition-related functions and modulation by psychopathology. Frontiers in Human Neuroscience, 7, Article 271. https://doi.org/10.3389/fnhum.2013.00271

76.

Warren

S. L.

Heller

Miller

G. A.

(2021). The structure of executive dysfunction in depression and anxiety. Journal of Affective Disorders, 279(January), 208-216. https://doi.org/10.1016/j.jad.2020.09.132

77.

Watson

Weber

Assenheimer

J. S.

Clark

L. A.

Strauss

M. E.

McCormick

R. A.

(1995). Testing a tripartite model: I. Evaluating the convergent and discriminant validity of anxiety and depression symptom scales. Journal of Abnormal Psychology, 104(1), 3-14. https://doi.org/10.1037//0021-843x.104.1.3

78.

Weijters

Baumgartner

Schillewaert

(2013). Reversed item bias: An integrative model. Psychological Methods, 18(3), 320-334. https://doi.org/10.1037/a0032121

79.

Yuniardi

Psikologi

(2020). Should we shoot intolerance of uncertainty for anxiety disorders? Data from the UK and Indonesia. International Journal of Psychosocial Rehabilitation, 24(8), 5134-5146. https://doi.org/10.37200/IJPR/V24I8/PR280524

80.

Zainal

N. H.

Newman

M. G.

Hong

R. Y.

(2019). Cross-cultural and gender invariance of transdiagnostic processes in the United States and Singapore. Assessment. Advance online publication. https://doi.org/10.1177/1073191119869832

81.

Zhong

Wang

Liu

(2009). Penn State Worry Questionnaire: Structure and psychometric properties of the Chinese version. Journal of Zhejiang University: Science. B, 10(3), Article 211. https://doi.org/10.1631/jzus.B0820189

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB