Reliability generalization (RG) was used to study five versions of the Working Alliance Inventory (WAI), including scores from 12 different scales. Sixty-seven internal consistency estimates, six interrater reliability estimates, and four study characteristics were analyzed. In general, reliability estimates of WAI scale scores appear to be robust. Mean reliability estimates ranged, in this sample of studies, from .79 to .97, with a modal estimate of .92. Variability in reliability estimates was, based on simple bivariate correlations, associated with client and therapist sample size for WAI total scores (observer version). Implications for measuring alliance using the WAI and conducting future RG studies on psychotherapy process measures are discussed.
Get full access to this article
View all access options for this article.
References
1.
Andrusyna, B. A. , Tang, T. Z., DeRubeis, R. J., & Luborsky, L. (2001). The factor structure of the Working Alliance Inventory in cognitive-behavioral therapy. Journal of Psychotherapy Practice and Research, 10, 173-178.
2.
Bordin, E. S. (1979). The generalizability of the psychoanalytic concept of the working alliance.Psychotherapy: Theory, Research, and Practice, 16, 252-260.
3.
Burkard, A. W. , Ponterotto, J. G., Reynolds, A. L., & Alfonso, V. C. (1999). White counselor trainees’racial identity and working alliance perceptions. Journal of Counseling and Development, 77, 324-329.
4.
Caruso, J. C. (2000). Reliability generalization of the NEO personality scales. Educational and Psychological Measurement, 60, 236-254.
5.
Castonguay, L. G. , Goldfried, M. R., Wiser, S., Raue, P. J., & Hayes, A. M. (1996). Predicting the effect of cognitive therapy for depression: A study of unique and common factors. Journal of Consulting and Clinical Psychology, 64, 497-504.
6.
Cecero, J. J. , Fenton, L. R., Frankforter, T. L., Nich, C., & Carroll, K. M. (2001). Focus on therapeutic alliance: The psychometric properties of six measures across three treatments. Psychotherapy, 38, 1-11.
7.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284-290.
8.
Crocker, L. , & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
9.
Dawis, R. V. (1987). Scale construction. Journal of Counseling Psychology, 34, 481-489.
10.
Feldt, L. S. , & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105-146). Washington, DC: American Council on Education.
11.
Fenton, L. R. , Cecero, J. J., Nich, C., Frankforter, T. L., & Carroll, K. M. (2001). Perspective is everything: The predictive validity of six working alliance instruments. Journal of Psychotherapy Practice and Research, 10, 262-268.
12.
Florsheim, P. , Shotorbani, S., Guest-Warnick, G., Barratt, T., & Hwang, W. (2000). Role of the working alliance in the treatment of delinquent boys in community-based programs. Journal of Clinical Child Psychology, 29, 94-107.
13.
Forchuk, C. (1995). Developmentof nurse-clientrelationships: Whathelps?Journal of the American Psychiatric Nurses Association, 1, 146-151.
14.
Gaston, L. (1990). The concept of the alliance and its role in psychotherapy: Theoretical and empirical considerations. Psychotherapy, 27, 143-153.
15.
Greenson, R. R. (1965). The working alliance and the transference neurosis. Psychoanalyis Quarterly, 34, 155-181.
16.
Hartley, D. , & Strupp, H. (1983). The therapeutic alliance: Its relationship to outcome in brief psychotherapy. In J. Masling (Ed.), Empirical studies of psychoanalytic theories (Vol. 1, pp. 1-27). Hillsdale, NJ: Erlbaum.
17.
Henson, R. K. , Kogan, L. R., & Vacha-Haase, T. (2001). A reliability generalization study of the Teacher Efficacy Scale and related instruments. Educational and Psychological Measurement, 61, 404-420.
18.
Hill, C. E. , & Williams, E. N. (2001). The process of individual therapy. In S. D. Brown & R. W. Lent(Eds.), Handbookof counseling psychology (3rd ed., pp. 670-710). New York: Wiley.
19.
Horvath, A. O. (1981). An exploratory study of the working alliance: Its measurement and relationship to outcome. Unpublished doctoral dissertation, University of British Columbia, Vancouver, Canada.
20.
Horvath, A. O. , & Greenberg, L. S. (1986). Development of the Working Alliance Inventory. In L. S. Greenberg & W. M. Pinsoff (Eds.), The psychotherapeutic process: A research handbook (pp. 529-556). New York: Guilford.
21.
Horvath, A. O. , & Greenberg, L. S. (1989). Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 36, 223-233.
22.
Horvath, A. O. , & Greenberg, L. S. (Eds.). (1994). The working alliance: Theory, research, and practice. New York: Wiley & Sons.
23.
Horvath, A. O. , & Luborsky, L. (1993). The role of the therapeutic alliance in psychotherapy. Journal of Consulting and Clinical Psychology, 61, 561-573.
24.
Horvath, A. O. , & Symonds, D. (1991). Relation between working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology, 38, 139-149.
25.
Hoyt, W. T. (2002). Bias in participant ratings of psychotherapy process: An initial generalizability study. Journal of Counseling Psychology, 49, 35-46.
26.
Kanninen, K. , Salo, J., & Punamäki, R. (2000). Attachment patterns and working alliance in trauma therapy for victims of political violence. Psychotherapy Research, 10, 435-449.
27.
Kivlighan, D. M., Jr. , Clements, L., Blake, C., Arnzen, A., & Brady, L. (1993). Counselor sex role orientation, flexibility, and working alliance formation. Journal of Counseling and Development, 72, 95-100.
28.
Kokotovic, A. M. , & Tracey, T. J. (1990). Working alliance in the early phase of counseling. Journal of Counseling Psychology, 37, 16-21.
29.
Lambert, M. J. , & Bergin, A. E. (1994). The effectiveness of psychotherapy. In A. E. Bergin & S. L. Garfield (Eds.), Handbookof psychotherapy and behavior change (4th ed., pp. 143-189). New York: Wiley & Sons.
30.
Long, J. R. (2001). Goal agreement and early therapeutic change. Psychotherapy, 38, 219-232.
31.
Luborsky, L. , Crits-Christoph, P., Alexander, L., Margolis, M., & Cohen, M. (1983). Two helping alliance methods of predicting outcomes of psychotherapy. Journal of Nervous and Mental Disease, 171, 480-491.
32.
Marmar, C. R. , Weiss, D. S., & Gaston, L. (1989). Towards validation of the California Therapeutic Alliance Rating System. Journal of Consulting and Clinical Psychology, 1, 46-52.
33.
Martin, D. J. , Garske, J. P., & Davis, K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68, 438-450.
34.
Marziali, E. (1984). Prediction of outcome of brief psychotherapy from therapist interpretive interventions. Archives of General Psychiatry, 41, 301-305.
35.
Meier, S. T. , & Davis, S. R. (1990). Trends in reporting psychometric properties of scales used in counseling psychology research. Journal of Counseling Psychology, 37, 113-115.
36.
Moseley, D. (1983). The therapeutic relationship and its association with outcome. Unpublished master’s thesis, University of British Columbia, Vancouver, Canada.
37.
Multon, K. D. , Kivlighan, D. M., Jr., & Gold, P. B. (1996). Changes in counselor adherence over the course of training. Journal of Counseling Psychology, 43, 356-363.
38.
Patton, M. J. , & Kivlighan, D. M., Jr. (1997). Relevance of the supervisory alliance to the counseling alliance and to treatment adherence in counselor training. Journal of Counseling Psychology, 44, 108-115.
39.
Pedhazur, E. J. , & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Erlbaum.
40.
Raue, P. J. , Castonguay, L. G., & Goldfried, M. R. (1993). The working alliance: A comparison of two therapies. Psychotherapy Research, 3, 197-207.
41.
Raue, P. J. , Goldfried, M. R., & Barkhan, M. (1997). The therapeutic alliance in psychodynamic-interpersonal and cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 65, 582-587.
42.
Raue, P. J. , Putterman, J. T., Goldfried, M. R., & Wolitzky, D. L. (1995). Effect of rater orientation on the evaluation of therapeutic alliance. Psychotherapy Research, 5, 337-342.
43.
Rosen, M. I. , Desai, R., Bailey, M., Davidson, L., & Rosenheck, R. (2001). Consumer experience with payeeship provided by a community mental health center. Psychiatric Rehabilitation Journal, 25, 190-195.
44.
Rowley, G. L. (1976). The reliability of observational measures. American Educational Research Journal, 13, 51-59.
45.
Saunders, S. M. , Howard, K. I., & Orlinsky, D. E. (1989). The Therapeutic Bond Scales: Psychometric characteristics and relationship to treatment effectiveness. Psychological Assessment, 1, 323-330.
46.
Sexton, H. (1996). Process, life events, and symptomatic change in brief eclectic psychotherapy. Journal of Consulting and Clinical Psychology, 64, 1358-1365.
47.
Smith, M. L. , Glass, G. V., & Miller, T. I. (1980). The benefits of psychotherapy. Baltimore: Johns Hopkins University Press.
48.
Solomon, P. , Draine, J., & Delaney, M. A. (1995). The working alliance and consumer case management. Journal of Mental Health Administration, 22, 126-134.
49.
Thompson, B. (1994). Guidelines for authors. Educational and Psychological Measurement, 54, 837-847.
50.
Thompson, B. , & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174-195.
51.
Tichenor, V. , & Hill, C. E. (1989). A comparison of six measures of working alliance. Psychotherapy, 26, 195-199.
52.
Tokar, D. M. , Hardin, S. I., Adams, E. M., & Brandel, I. W. (1996). Clients’expectations about counseling and perceptions of the working alliance. Journal of College Student Psychotherapy, 11, 9-26.
53.
Tracey, T. J. , & Kokotovic, A. M. (1989). Factor structure of the Working Alliance Inventory. Psychological Assessment, 1, 207-210.
54.
Tryon, G. S. , & Kane, A. S. (1995). Client involvement, working alliance, and type of therapy termination. Psychotherapy Research, 5, 189-198.
55.
Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement, 58, 6-20.
56.
Vacha-Haase, T. , Kogan, L. R., Tani, C. R., & Woodall, R. A. (2001). Reliability generalization: Exploring reliability coefficients of MMPI clinical scales scores. Educational and Psychological Measurement, 61, 45-59.
57.
Vacha-Haase, T. , Kogan, L. R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational and Psychological Measurement, 60, 509-522.
58.
Vacha-Haase, T. , Ness, C., Nilsson, J., & Reetz, D. (1999). Practices regarding reporting of reliability coefficients: A review of three journals. Journal of Experimental Education, 67, 335-341.
59.
Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Erlbaum.
60.
Wilkinson, L. , & American Psychological Association Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
61.
Wolfe, B. E. , & Goldfried, M. R. (1988). Research on psychotherapy integration: Recommendations and conclusions from an NIMH workshop. Journal of Consulting and Clinical Psychology, 56, 448-451.
62.
Yin, P. , & Fan, X. (2000). Assessing the reliability of the Beck Depression Inventory scores: Reliability generalization across studies. Educational and Psychological Measurement, 60, 201-223.
