Comparing the Validity of the Rorschach Performance Assessment System and Exner’s Comprehensive System to Differentiate Patients and Nonpatients

Abstract

Building on the comparative nonpatient study of Pianowski et al., we examine data from the Rorschach Performance Assessment System (R-PAS) and Comprehensive System (CS) in 100 nonpatients and 100 patients, 50 of each per system. Replicating their results but now in a patient sample, R-PAS produced more patient protocols having an optimal number of responses (R) for interpretation and eliminated the need for readministration due to low R. The R-PAS protocols were also much less variable in R, despite having about 2.5 more responses. Extending their results, we document that the primary markers of psychopathology in each system validly differentiate patients from nonpatients. However, R-PAS produced stronger effects. Finally, Complexity added to the valid discrimination of patients from nonpatients just for R-PAS, with patients producing less complex and rich records. The more erratic variability in R for the CS produced larger Complexity standard deviations (SDs) that obscured these genuine differences in people. We discuss implications for research and applied practice, along with directions for future research.

Keywords

assessment psychopathology severity Rorschach performance tests

Get full access to this article

View all access options for this article.

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing.

Borenstein

Hedges

L. V.

Higgins

J. P. T.

Rothstein

H. R.

(2009). Introduction to meta-analysis. John Wiley.

Carvalho

L. F.

Primi

(2019). Manual técnico do Inventário Dimensional Clínico da Personalidade 2 (IDCP-2) e versão triagem (IDCP-triagem) [Technical Manual of the Dimensional Clinical Personality Inventory 2 (IDCP-2) and its Screening version (IDCP-screening)]. Pearson.

Cicchetti

D. V.

(1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

Coxe

West

S. G.

Aiken

L. S.

(2009). The analysis of count data: A gentle introduction to Poisson regression and its alternatives. Journal of Personality Assessment, 91(2), 121–136. https://doi.org/10.1080/00223890802634175

Curran

P. J.

West

S. G.

Finch

J. F.

(1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16.

Dean

K. L.

Viglione

D. J.

Perry

Meyer

G. J.

(2007). A method to optimize the response range while maintaining Rorschach Comprehensive System validity. Journal of Personality Assessment, 89, 149–161. https://doi.org/10.1080/00223890701468543

Diener

M. J.

Hilsenroth

M. J.

Shaffer

S. A.

Sexton

J. E.

(2011). A meta-analysis of the relationship between the Rorschach Ego Impairment Index (EII) and psychiatric severity. Clinical Psychology & Psychotherapy, 18, 464–485. https://doi.org/10.1002/cpp.725

10.

Dzamonja-Ignjatovic

Smith

B. L.

Jocic

D. D.

Milanovic

(2013). A comparison of new and revised Rorschach measures of schizophrenic functioning in a Serbian clinical sample. Journal of Personality Assessment, 95, 471–478. https://doi.org/10.1080/00223891.2013.810153

11.

Exner

J. E.

(1974). The Rorschach: A comprehensive system (Vol. 1). Wiley.

12.

Exner

J. E.

(1983). Continuing research [1983 Alumni Newsletter]. Rorschach Workshops.

13.

Exner

J. E.

(1997). Rorschach workshops and the future (Alumni Newsletter). Rorschach Workshops.

14.

Exner

J. E.

(2003). The Rorschach: A comprehensive system: Vol. 1. Basic foundations (4th ed.). Wiley.

15.

Hosseininasab

Meyer

G. J.

Viglione

D. J.

Mihura

J. L.

Berant

Resende

A. C.

…Mohammadi

M. R.

(2019). The effect of CS administration or an R-Optimized alternative on R–PAS Variables: A meta-analysis of findings from six studies. Journal of Personality Assessment, 101(2), 199–212. https://doi.org/10.1080/00223891.2017.1393430

16.

Kay

S. R.

Fiszbein

Opler

L. A.

(1987). The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin, 13(2), 261–276. https://doi.org/10.1093/schbul/13.2.261

17.

Kleiger

J. H.

Mihura

J. L.

(2021). Developments in the Rorschach assessment of disordered thinking and communication. Rorschachiana, 42(2), 265–280. https://doi.org/10.1027/1192-5604/a000132

18.

MacCallum

R. C.

Zhang

Preacher

K. J.

Rucker

D. D.

(2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7(1), 19–40. https://doi.org/10.1037/1082-989X.7.1.19

19.

Meyer

G. J.

(1992). Response frequency problems in the Rorschach: Clinical and research implications with suggestions for the future. Journal of Personality Assessment, 58, 231–244. https://doi.org/10.1207/s15327752jpa5802_2

20.

Meyer

G. J.

(1993). The impact of response frequency on the Rorschach constellation indices and on their validity with diagnostic and MMPI-2 criteria. Journal of Personality Assessment, 60, 153–180. https://doi.org/10.1207/s15327752jpa6001_13

21.

Meyer

G. J.

(1997). On the integration of personality assessment methods: The Rorschach and MMPI. Journal of Personality Assessment, 68, 297–330. https://doi.org/10.1207/s15327752jpa6802_5

22.

Meyer

G. J.

(2022). Understanding complexity on the Rorschach as a construct and a formally scored variable [Manuscript submitted for publication]. Department of Psychology, University of Toledo.

23.

Meyer

G. J.

Erdberg

Shaffer

T. W.

(2007). Toward international normative reference data for the Comprehensive System. Journal of Personality Assessment, 89, S201–S216. https://doi.org/10.1080/00223890701629342

24.

Meyer

G. J.

Giromini

Viglione

D. J.

Reese

J. B.

Mihura

J. L.

(2015). The association of gender, ethnicity, age, and education with Rorschach scores. Assessment, 22, 46–64. https://doi.org/10.1177/1073191114544358

25.

Meyer

G. J.

Hilsenroth

M. J.

Baxter

Exner

J. E.

Jr. Fowler

J. C.

Piers

C. C.

Resnick

(2002). An examination of interrater reliability for scoring the Rorschach Comprehensive System in eight data sets. Journal of Personality Assessment, 78, 219–274. https://doi.org/10.1207/S15327752JPA7802_03

26.

Meyer

G. J.

Hosseininasab

Viglione

D. J.

Mihura

J. L.

Berant

Resende

A. C.

Reese

(2020). The effect of CS administration or an R-Optimized alternative on potential projective material in Rorschach responses from six studies and a meta-analysis of their findings. Journal of Personality Assessment, 102(1), 135–146. https://doi.org/10.1080/00223891.2018.1492926

27.

Meyer

G. J.

Hsiao

W.-C.

Viglione

D. J.

Mihura

J. L.

Abraham

L. M.

(2013). Rorschach scores in applied clinical practice: A survey of perceived validity by experienced clinicians. Journal of Personality Assessment, 95, 351–365. https://doi.org/10.1080/00223891.2013.770399

28.

Meyer

G. J.

Mihura

J. L.

(2021). Rorschach Performance Assessment System (R-PAS) for assessing disordered thinking and perception. In Weiner

I. B.

Kleiger

J. H.

(Eds.), Psychological assessment of disordered thinking and perception (pp. 151–168). American Psychological Association.

29.

Meyer

G. J.

Viglione

D. J.

Mihura

J. L.

(2017). Psychometric foundations of the Rorschach Performance Assessment System (R-PAS). In Erard

Evans

(Eds.), The Rorschach in multimethod forensic practice (pp. 23–91). Routledge.

30.

Meyer

G. J.

Viglione

D. J.

Mihura

J. L.

Erard

R. E.

Erdberg

(2011). Rorschach Performance Assessment System: Administration, coding, interpretation, and technical manual. Rorschach Performance Assessment System.

31.

Mihura

J. L.

Meyer

G. J.

Dumitrascu

Bombel

(2013). The validity of individual Rorschach variables: Systematic reviews and meta-analyses of the comprehensive system. Psychological Bulletin, 139(3), 548–605. https://doi.org/10.1037/a0029406

32.

Mihura

J. L.

Roy

Graceffo

R. A.

(2017). Psychological assessment training in clinical psychology doctoral programs. Journal of Personality Assessment, 99(2), 153–164. https://doi.org/10.1080/00223891.2016.1201978

33.

Moore

R. C.

Viglione

D. J.

Rosenfarb

I. S.

Patterson

T. L.

Mausbach

B. T.

(2013). Rorschach measures of cognition relate to everyday and social functioning in schizophrenia. Psychological Assessment, 25, 253–263. . https://doi.org/10.1037/a0030546

34.

Nascimento

R. S. G. F

. (2007). Rorschach Comprehensive System data for a sample of 409 adult nonpatients from Brazil. Journal of Personality Assessment, 89(Suppl. 1), S35–S41. https://doi.org/10.1080/00223890701582905

35.

Pianowski

Meyer

G. J.

de Villemor-Amaral

A. E.

(2016a). The impact of R-optimized administration modeling procedures on Brazilian normative reference values for Rorschach Scores. Journal of Personality Assessment, 98(4), 408–418. https://doi.org/10.1080/00223891.2016.1148701

36.

Pianowski

Meyer

G. J.

de Villemor-Amaral

A. E.

(2016b). Potential projective material on the Rorschach: Comparing Comprehensive System protocols to their modeled R-Optimized administration counterparts. Journal of Personality Assessment, 98, 398–407. https://doi.org/10.1080/00223891.2016.1147451

37.

Pianowski

Meyer

G. J.

de Villemor-Amaral

A. E.

Zuanazzi

A. C.

do Nascimento

R. S. G. F

. (2021). Does the Rorschach Performance Assessment System (R-PAS) differ from the Comprehensive System (CS) on variables relevant to interpretation? Journal of Personality Assessment, 103(1), 132–147. https://doi.org/10.1080/00223891.2019.1677678

38.

Shaffer

T. W.

Erdberg

Meyer

G. J.

(2007). International reference samples for the Rorschach Comprehensive System [Special issue]. Journal of Personality Assessment, 89(Suppl. 1), S201–S216. https://doi.org/10.1080/00223890701629342

39.

W.-S.

Viglione

D. J.

Green

E. E.

Tam

W.-C. C.

J.-A.

Chang

Y.-T.

(2015). Cultural and linguistic adaptability of the Rorschach Performance Assessment System as a measure of psychotic characteristics and severity of mental disturbance in Taiwan. Psychological Assessment, 27, 1273–1285. https://doi.org/10.1037/pas0000144

40.

Ventura

Green

M. F.

Shaner

Liberman

R. P

. (1993). Training and quality assurance with the Brief Psychiatric Rating Scale: “The drift busters.” International Journal of Methods in Psychiatric Research, 3(4), 221–244. https://doi.org/10.1016/j.psychres.2013.07.001

41.

Viglione

D. J.

Giromini

Gustafson

Meyer

G. J.

(2014). Developing continuous variable composites for Rorschach measures of thought problems, vigilance, and suicide risk. Assessment, 21, 42–49. https://doi.org/10.1177/1073191112446963

42.

Viglione

D. J.

Meyer

G. J.

(2008). An overview of Rorschach psychometrics for forensic practice. In Gacono

C. B.

Evans

F. B.

(Eds.), Handbook of forensic Rorschach psychology (pp. 21–53). Routledge.

43.

Viglione

D. J.

Meyer

G. J.

Jordan

R. J.

Converse

G. L.

Evans

MacDermott

Moore

R. C.

(2015). Developing an alternative Rorschach administration method to optimize the number of responses and enhance clinical inferences. Clinical Psychology and Psychotherapy, 22, 546–558. https://doi.org/10.1002/cpp.1913

44.

Viglione

D. J.

Perry

Giromini

Meyer

G. J.

(2011). Revising the Rorschach Ego Impairment Index to accommodate recent recommendations about improving Rorschach validity. International Journal of Testing, 11, 349–364. https://doi.org/10.1080/15305058.2011.589019

45.

Viglione

D. J.

Perry

Jansak

Meyer

Exner

J. E.

Jr. (2003). Modifying the Rorschach human experience variable to create the human representational variable. Journal of Personality Assessment, 81(1), 64–73. https://doi.org/10.1207/S15327752JPA8101_06

46.

Winterstein

B. P.

Silvia

P. J.

Kwapil

T. R.

Kaufman

J. C.

Reiter-Palmon

Wigert

(2011). Brief assessment of schizotypy: Developing short forms of the Wisconsin Schizotypy Scales. Personality and Individual Differences, 51(8), 920–924. https://doi.org/10.1016/j.paid.2011.07.027

47.

World Health Organization. (1992). The ICD-10 classification of mental and behavioural disorders: Clinical descriptions and diagnostic guidelines. World Health Organization. https://apps.who.int/iris/handle/10665/37958