This article explores how substantive and methodological differences between English- and German-language psychotherapy outcome studies may have contributed to observed differences in their effectiveness. Data from the meta-analyses by Smith, Glass, and Miller and Wittmann and Matt were reanalyzed to examine the extent to which substantive and methodological differences can account for the difference in the overall effect size of psychotherapy published in English- and German-language sources. The findings indicate that differences in the study of simple phobias, the use of trait inventories as outcome measures, and the selection of effect sizes within studies are likely to have led Smith et al. to overestimate, or conversely, Wittmann and Matt to underestimate the benefits of psychotherapy. Such results suggest that average effect estimates derived for English- and German-language psychotherapy outcomes studies depend on, and have to be interpreted in, the cultural context in which the research is being conducted and reviewed. The comparison between different classes of interventions has to take into account how this context shapes the effect derived for such classes.
Get full access to this article
View all access options for this article.
References
1.
Andrews, G. , & Harvey, R. (1981). Does psychotherapy benefit neurotic patients?Archives of General Psychiatry, 38, 1203-1208.
2.
Boruch, R. F. , & Gomez, H. (1979). Measuring impact: Power theory in social program evaluation. In L. E. Datta & R. Perloff (Eds.), Improving evaluation (pp. 139-170). Beverly Hills: Sage.
3.
Cooper, H. M. (1984). The integrative research review: A systematic approach (Applied Social Research Methods Series: Vol. 2). Beverly Hills, CA: Sage.
4.
Coursol, A. , & Wagner, E. E. (1986). Effect of positive findings on submission and acceptance rates: A note on meta-analysis bias. Professional Psychology: Research and Practice, 17, 136-137.
5.
Eaton, W. W. , Kramer, M., Anthony, J. C., Dryman, A., Shapiro, S., & Locke, B. Z. (1989). The incidence of specific DIS/DSMIII mental disorders: Data from the NIMH epidemiologic catchment area program. Acta Psychiatrica Scandinavica, 79, 163-178.
6.
Glass, G. V. , McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
7.
Hand, I. , & Wittchen, H. U. (Eds.). (1986). Panic and phobias. Berlin: Springer.
8.
Hedges, L. V. , & Olkin, I. (1985). Statistical methods for meta -analysis. New York: Academic Press.
9.
Hoaglin, D. C. , Mosteller, F., & Tukey, J. W. (Eds.). (1983). Understanding robust and exploratory data analysis. New York: Wiley.
10.
Hunter, J. E. , Schmidt, F. L. (1990). Methods of meta-analysis. Correcting error and bias in research findings. Newbury Park, CA: Sage.
11.
Hunter, J. E. , Schmidt, F. L., & Jackson, G. B. (1982). Meta-analysis: Cumulating findings across research. Beverly Hills, CA: Sage.
12.
Light, R. J. , & Pillemer, D. B. (1984). Summing up. The science of reviewing research. Cambridge, MA: Harvard University Press.
13.
Matt, G. E. (1988). Behandlungseffekte und Studienmerkmale: Eine meta-analytische Untersuchung zur Praxis der Psychotherapieeffektforschung [Treatment effects and study characteristics: A meta-analytic study of the practice of psychotherapy outcome research.]Gruppendynamik, 18, 345-360.
14.
Matt, G. E. (1989). Decision rules for selecting effect sizes in meta-analysis: A review and reanalysis of psychotherapy outcome studies. Psychological Bulletin, 105, 106-115.
15.
Matt, G. E. , & Navarro, A. M. (1990, October). Meta-analyses of the effectiveness of psychotherapy: Practice and improvement. Paper presented at the annual meeting of the American Evaluation Association, Washington, DC.
16.
Matt, G. E. , & Wittmann, W. W. (1985). Zum Status Quo deutschsprachiger Psychotherapieeffektforschung der Jahre 1971-1982. [The present state of psychotherapy outcome research in German-speaking countries, 1971-1982.]Zeitschrift für Klinische Psychologie, 14, 293-312.
17.
Miller, R. C. , & Berman, J. S. (1983). The efficacy of cognitive behavior therapies: A quantitative review of the research evidence. Psychological Bulletin, 94, 39-53.
18.
Orwin, R. G. , & Cordray, D. S. (1985). Effects of deficient reporting on meta-analysis: A conceptual framework and reanalysis. Psychological Bulletin, 97, 134-147.
19.
Regier, D. A. , Boyd, J. H., Burke, J. D., Rae, D. S., Myers, J. K., Kramer, M.Robins, L. N., George, L. K., & Locke, B. Z. (1988). One-month prevalence of mental disorders in the United States. Archives of General Psychiatry, 45, 977-986.
20.
Rosenthal, R. (1978). Interpersonal expectancy effects: The first 345 studies. Behavioral and Brain Sciences, 3, 377-415.
21.
Rosenthal, R. (1984). Meta-analytic procedures for social research (Applied Social Research Methods Series: Vol. 6). Beverly Hills, CA: Sage.
22.
Rosenthal, R. , & Rubin, D. B. (1986). Meta-analytic procedures for combining studies with multiple effect sizes. Psychological Bulletin, 99, 400-406.
23.
Shapiro, D. A. , & Shapiro, D. (1982). Meta-analysis of comparative outcome studies: A replication and refinement. Psychological Bulletin, 92, 581-604.
24.
Smith, M. L. (1980). Integrating studies of psychotherapy outcomes. In R. Rosenthal (Ed.), Quantitative assessment of research domains (pp. 47-62). San Francisco: Jossey-Bass.
25.
Smith, M. L. , & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752-760.
26.
Smith, M. L. , Glass, G. V., & Miller, T. I. (1980). The benefits of psychotherapy. Baltimore, MD: Johns Hopkins University Press.
27.
Strube, M. J. (1985). Combining and comparing significance levels from nonindependent hypothesis tests. Psychological Bulletin, 97, 334-341.
28.
Wanous, J. P. , Sullivan, S. E., & Malinak, J. (1989). The role of judgment calls in meta-analysis. Journal of Applied Psychology, 74, 259-264.
29.
Wittmann, W. W. , & Matt, G. E. (1986). Meta-Analyse als Integration von Forschungsergebnissen am Beispiel deutschprachiger Arbeiten zur Effektivitat von Psychotherapie [Meta-analysis as a method for integrating psychotherapy studies in German-speaking countries]. Psychologische Rundschau, 37, 20-40.
30.
Wolf, F. (1986). Meta-analysis. Quantitative methods for research synthesis (Quantitative Applications in the Social Sciences, No. 59). Beverly Hills, CA: Sage.
31.
Yeaton, W. H. , & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments. Strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156-167.
