Generalizability theory was used to assess the reliability of the Dartmouth Assertive Community Treatment Scale (DACTS), which was developed to assess treatment reliability to assertive community treatment (ACT). Program staff and local evaluators who were participating in a national demonstration program to serve homeless mentally ill clients provided data. The total scale score for the DACTS demonstrated acceptable internal consistency and interrater reliability. Although the stability of the total DACTS score was quite low, many items on the DACTS were quite stable. The generalizability analyses provide additional detail on the effects of site, occasion, and site by occasion interactions on the reliability of the DACTS.
Get full access to this article
View all access options for this article.
References
1.
Bond, G. R., D. R. Becker, R. E. Drake, and K. M. Vogler. 1997. A fidelity scale for the individual placement and support model of supported employment. Rehabilitation Counseling Bulletin40:275-284.
2.
Bond, G. R., L. Evans, M. Salyers, J. Williams, and H. W. Kim. Forthcoming. Measurement of fidelity in psychiatric rehabilitation. Mental Health Services Research.
3.
Burns, B. J., and A. B. Santos. 1995. Assertive community treatment: An update of randomized trials. Psychiatric Services46 (7): 669-675.
4.
Calsyn, R. J. Forthcoming. A checklist for critiquing treatment fidelity studies. Mental Health Services Research.
5.
Cronbach, L. J., and G. C. Gleser. 1965. Psychological tests and personnel decisions. Urbana: University of Illinois Press.
6.
Cronbach, L. J., G. C. Gleser, H. Nanda, and N. Rajaratnam. 1972. The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: John Wiley.
7.
Dixon, L. B., N. Krauss, E. Kernan, A. F. Lehman, and B. R. DeForge. 1995. Modifying the PACT model to serve homeless persons with severe mental illness. Psychiatric Services46 (7): 684-688.
8.
Fekete, D. M., G. R. Bond, and M. P. Salyers. 1997. ACT fidelity scale: A practical application in a statewide project. Poster session at the annual conference of the American Public Health Association, Indianapolis, IN.
9.
Harachi, T. W., R. D. Abbott, R. F. Catalano, K. P. Haggerty, and C. B. Fleming. 1999. Opening the black box: Using process evaluation measures to assess implementation and theory building. American Journal of Community Psychology27 (5):711-731.
10.
Johnsen, M., L. Samberg, R. Calsyn, M. Blasinky, W. Landow, and H. Goldman. 1999. Case management models for persons who are homeless and mentally ill: The ACCESS demonstration project. Community Mental Health Journal35(4): 325-346.
11.
Lucca, A. M. Forthcoming. A clubhouse fidelity index: Preliminary reliability and validity results. Mental Health Services Research.
12.
McGrew, J. H., and G. R. Bond. 1995. Critical ingredients of assertive community treatment: Judgments of the experts. Journal of Mental Health Administration22 (2): 113-125.
13.
McGrew, J. H., G. R. Bond, L. Dietzen, and M. Salyers. 1994. Measuring the fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology62 (4): 670-678.
14.
McHugo, G. J., R. E. Drake, G. B. Teague, and H. Xie. 1999. Fidelity to assertive community treatment and client outcomes in the New Hampshire dual disorders study. Psychiatric Services50 (6): 818-824.
15.
Moncher, F. J., and R. J. Prinz. 1991. Treatment fidelity in outcome studies. Clinical Psychology Review11:247-266.
16.
Morse, G. A., R. J. Calsyn, G. Allen, B. Tempelhoff, and R. Smith. 1992. Experimental comparison of the effects of three treatment programs for homeless mentally ill people. Hospital and Community Psychiatry43 (10): 1005-1010.
17.
Morse, G. A., R. J. Calsyn, W. D. Klinkenberg, M. L. Trusty, F. Gerber, R. Smith, B. Tempelhoff, and L. Ahmad. 1997. An experimental comparison of three types of case management for homeless mentally ill persons. Psychiatric Services48 (4): 497-503.
18.
Olfson, M.1990. Assertive community treatment: An evaluation of the experimental evidence. Hospital and Community Psychiatry41 (6): 634-641.
19.
Randolph, F., M. Blasinky, W. Leginski, L. B. Parker, and H. H. Goldman. 1997. Creating integrated service systems for homeless persons with mental illness: The ACCESS program. Psychiatric Services48 (3): 369-373.
20.
Searle, S. R., G. Casella, and C. E. McCulloch. 1992. Variance components. New York: John Wiley.
21.
Shavelson, R. J., and N. M. Webb. 1991. Generalizability theory: A primer. Newbury Park, CA: Sage.
22.
Solomon, P.1992. The efficacy of case management services for severely mentally disabled clients. Community Mental Health28 (3): 163-180.
23.
SPSS, Inc.1997. SPSS Advanced Statistics 7.5. Chicago: SPSS, Inc.
24.
Sussman, S., C. W. Dent, E. R. Galaif, A. W. Stacy, T. Newman, M. A. Moss, M. Hennesy, S. Craig, M. A. Pentz, and T.R. Simon. 1997. Implemetation and process evaluation of a student “school-as-community” group. Evaluation Review21: 94-123
25.
Teague, G., G. R. Bond, and R. E. Drake. 1998. Program fidelity in assertive community treatment: Development and use of a measure. American Journal of Orthopsychiatry68 (2): 216-232.
26.
Teague, G. B., R. E. Drake, and T. H. Ackerson. 1995. Evaluating use of continuous treatment teams for persons with mental illness and substance abuse. Psychiatric Services46 (7): 689-695.
27.
Test, M. A.1992. Training in community living. In Handbook of psychiatric rehabilitation, edited by R. P. Liberman. Boston: Allyn and Bacon.
