Power and Reliability: The Case of Homogeneous True Score Regression Across Treatments

Abstract

As part of their investigation of the relation between reliability of measures and statistical power, Nicewander and Price (1983) examined the situation in which true score regressions are homogeneous across treatments. They derived an equation indicating that the measure, X or Y, yielding the greater power depends on the reliabilities (p _XX'. and p _YY', respectively) and the squared linear correlation between true scores for X and Y (p_T(X)T(Y) ² ). Typically, however, researchers cannot compute a direct estimate of p_T(X)T(Y) ² The authors eliminate this problem by (a) pointing out situations in which the value of p_T(X)T(Y) ² is either irrelevant or 1.0 and (b) demonstrating that their equation otherwise can be expressed in terms of p _XX' and p _XY only.

Get full access to this article

View all access options for this article.

References

Allen, M. J. and Yen, W. M. (1979). Introduction to measurement theory. Belmont, CA: Wadsworth.

Cleary, T. A. and Linn, R. L. (1969). Error of measurement and the power of a statistical test. British Journal of Mathematical and Statistical Psychology, 22, 49-55.

Fleiss, J. L. (1976). Comment on Overall and Woodward's asserted paradox concerning the measurement of change. Psychological Bulletin, 83, 774-775.

Gulliksen, H. (1950). Theory of mental tests. New York: Wiley.

Nicewander, W. A. and Price, J. M. (1978). Dependent variable reliability and the power of statistical tests. Psychological Bulletin, 85, 405-409.

Nicewander, W. A. and Price, J. M. (1983). Reliability of measurement and the power of statistical tests: Some new results. Psychological Bulletin, 94, 524-533.

Overall, J. E. and Woodward, J. A. (1975). Unreliability of difference scores: A paradox for the measurement of change. Psychological Bulletin, 82, 85-86.

Overall, J. E. and Woodward, J. A. (1976). Reassertion of the paradoxical power of tests of significance based on unreliable difference scores. Psychological Bulletin, 83, 776-777.

Sutcliffe, J. P. (1958). Error of measurement and the sensitivity of a test of significance. Psychometrika, 23, 9-17.

10.

Sutcliffe, J. P. (1980). On the relationship of reliability to statistical power. Psychological Bulletin, 88, 509-515.

11.

Zimmerman, D. W. and Williams, R. H. (1986). Note on the reliability of experimental measures and the power of significance tests. Psychological Bulletin, 100, 123-124.