Sage Journals: Discover world-class research

Abstract

One of the most important effect sizes used in meta-analysis is the standardized mean difference (SMD). In this article, the conditions under which SMD effect sizes based on different measures of the same construct are directly comparable are investigated. The results show that SMD effect sizes from different measures of the same construct are directly comparable if and only if (Condition 1) there is a linear relationship between the true scores, and (Condition 2) there is a special linear relationship between the magnitude of the error variances of the involved measures. If only Condition 1 is met, then the reliability-corrected SMD effect sizes from the different measures are directly comparable. However, if Condition 1 is not met, then there will be nonlinear relationships between the true score metrics of two or more of the measures and systematic variability between SMD effect sizes that reflects these nonlinear relationships.

Keywords

meta-analysis effect size standardized mean difference (SMD)reliability-corrected standardized mean difference

Get full access to this article

View all access options for this article.

References

Bryk, A. , & Raudenbush, S. (1992).Hierarchical linear models. Newbury Park, CA: Sage.

Cooper, H. , & Hedges, L. (1994).Research synthesis as a scientific enterprise. In Handbookof research synthesis (pp. 3-14). New York: Russell Sage.

Durlak, J. , & Lipsey, M. (1991). A practitioner's guide to meta-analysis.American Journal of Community Psychology,19(3), 291-332.

Feldt, L. , & Brennan, R. (1989). Reliability. In R. Linn (Ed.),Educational measurement (2nd ed., pp. 105-146). New York: Macmillan.

Gleser, L. (1982). Discussion of “equating using the confirmatory factor analysis model.” In P. Holland & D. Rubin (Eds.),Test equating (pp. 259-270). New York: Academic Press.

Hambleton, R. , & Swaminathan, H. (1985).Item response theory: Principles and applications. Boston: Kluwer/Nijhoff.

Hedges, L. , & Olkin, I. (1985).Statistical methods for meta-analysis. New York: Academic Press.

Hunt, M. (1997).How science takes stock. New York: Russell Sage.

Hunter, J. , & Schmidt, F. (1990).Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.

10.

Lipsey, M. , & Wilson, D. (2001).Practical meta-analysis. Thousand Oaks, CA: Sage.

11.

Lord, F. (1980).Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

12.

Lord, F. , & Novick, M. (1968).Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

13.

Peterson, N. , Kolen, M. , & Hoover, H. (1989).Scaling, norming,and equating.In R. Linn (Ed.),Educational measurement (2nd ed., pp. 221-262). New York: Macmillan.

14.

Rosenthal, H. (1994). Parametric measures of effect size. In H. Cooper & L. Hedges (Eds.),Handbook of research synthesis (pp. 231-244). New York: Russell Sage.

15.

Solow, D. (2002).How to read and do proofs. New York: John Wiley.

16.

Wang, W. , & Chen, H. (2004).The standardized mean difference within the framework of item response theory.Educational and Psychological Measurement,64(2), 201-223.

17.

Wilson, D. , & Lipsey, M. (2001). The role of method in treatment effectiveness research: Evidence from meta-analysis.Psychological Methods,6(4), 413-429.

18.

Wonnacott,T. , & Wonnacott,R. (1981).Regression: A second course in statistics. New York: John Wiley.

The Comparability of the Standardized Mean Difference Effect Size Across Different Measures of the Same Construct

Abstract

Keywords

Get full access to this article

References