The medical sciences offer several helpful tools to the program evaluator for outcome analysis. Where a single point on the range of a dependent variable clearly defines attainment of outcome success, the number needed to treat (NNT) offers an indicator of a program’s efficacy. L’Abbé plots compare several programs in terms of the event rates that constitute the NNTs.
If the nature of an outcome variable is not amenable to such a demarcation, then the evaluator may extend the NNT concept by calculating and plotting event rates on a continuous, ordinal basis. Graphical analysis of these curves may involve program comparisons, inclusion of other relevant variables, and visualization of the test statistic for inferential purposes.
Get full access to this article
View all access options for this article.
References
1.
Anderson, D. R., Burnham, K. P., & Thompson, W. L. (2000). Null hypothesis testing: Problems, prevalence, and an alternative. Journal of Wildlife Management, 64, 912-923.
2.
Bailar, J. C., III, & Mosteller, F. (1988). Guidelines for statistical reporting in articles for medical journals. Annals of Internal Medicine, 108, 266-273.
3.
Bender, R. (2001). Calculating confidence intervals for the number needed to treat. Controlled Clinical Trials, 22, 102-110.
4.
Chow, S. L. (1996). Statistical significance: Rationale, validity and utility. London: Sage.
5.
Cicchetti, D. V. (1998). Role of null hypothesis significance testing (NHST) in the design of neuropsychologic research. Journal of Clinical and Experimental Neuropsychology, 20, 293-295.
6.
Cliff, N. (1996). Ordinal methods for behavioral data analysis. Mahwah, NJ: Lawrence Erlbaum.
7.
Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.
8.
Cook, R. J., & Sackett, D. L. (1995). The number needed to treat: A clinically useful measure of treatment effect. British Medical Journal, 310, 452-454.
9.
Germano, J. D. (1999). Ecology, statistics, and the art of misdiagnosis: The need for a paradigm shift. Environmental Reviews, 7, 167-190.
10.
Gibbons, J. D., & Chakraborti, S. (1992). Nonparametric statistical inference (3rd ed.). New York: Marcel Dekker.
11.
Glass, G. V., & Hopkins, K. D. (1984). Statistical methods in education and psychology. Englewood Cliffs, NJ: Prentice-Hall.
12.
Guyatt, G. H., Juniper, E. F., Walter, S. D., Griffith, L. E., & Goldstein, R. S. (1998). Interpreting treatment effects in randomised trials. British Medical Journal, 316, 690-693.
13.
Hallberg, B., Kinkoph, S., & Ray, B. (1997). Using Microsoft Excel 97, bestseller edition. Indianapolis, IN: Que Corporation.
14.
Hanes, J. C. (2000). Tapestry over time: A tableau approach to the visualization of multivariate dynamics. Unpublished doctoral dissertation, University of North Carolina-Greensboro, Greensboro, NC.
15.
Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.). (1997). What if there were no significance tests?Mahwah, NJ: Lawrence Erlbaum.
16.
Hettmansperger, T. P., McKean, J. W., & Sheather, S. J. (2000). Robust nonparametric methods. Journal of the American Statistical Association, 95, 1308-1312.
17.
Howard, D. C. P., & Howard, P. A. (2000). Towards sustainability of human services: Assessing community self-determination and self-reliance. The Canadian Journal of Program Evaluation, 15, 25-40.
18.
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.
19.
Jaeger, R. M. (1983). Statistics: A spectator sport. Beverly Hills, CA: Sage.
20.
Kanji, G. K. (1993). 100 Statistical tests. Thousand Oaks, CA: Sage.
21.
Kazdin, A. E. (1999). The meanings and measurement of clinical significance. Journal of Consulting and Clinical Psychology, 67, 332-339.
22.
King, G., Tomz, M., & Wittenberg, J. (2000). Making the most of statistical analyses: Improving interpretation and presentation. American Journal of Political Science, 44, 341-355.
23.
Kirk, R. E. (2001). Promoting good statistical practices: Some suggestions. Educational and Psychological Measurement, 61, 213-218.
24.
L’Abbé, K.A., Detsky, A. S., & O’Rourke, K. (1987). Meta-analysis in clinical research. Annals of Internal Medicine, 107, 224-233.
25.
Laupacis, A., Sackett, D. L., & Roberts, R. S. (1988). An assessment of clinically useful measures of the consequences of treatment. The New England Journal of Medicine, 318, 1728-1733.
26.
Maher, B. (1998). When the coefficient hits the clinic: Effect size and the size of the effect. Behavioral and Brain Sciences, 21, 211-212.
27.
McCulloch, B. D., & Wilson, B. (1999). On the accuracy of statistical procedures in Microsoft Excel 97. Computational Statistics & Data Analysis, 31, 27-37.
28.
McLean, J. E., & Ernest, J. M. (1998). The role of statistical significance testing in educational research. Research in the Schools, 5, 15-22.
29.
McQuay, H. J., & Moore, R. A. (1997). Using numerical results from systematic reviews in clinical practice. Annals of Internal Medicine, 126, 712-720.
30.
Michell, J. (1999). Measurement in psychology: Critical history of a methodological concept. Cambridge, UK: Cambridge University Press.
31.
Murray, L. W., & Dosser, D. A., Jr. (1987). How significant is a significant difference? Problems with the measurement of magnitude of effect. Journal of Counseling Psychology, 34, 68-72.
32.
Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301.
33.
Nix, T. W., & Barnette, J. J. (1998). The data analysis dilemma: Ban or abandon. A review of null hypothesis significance testing. Research in the Schools, 5, 3-14.
34.
Norman, G. R., & Streiner, D. L. (1997). PDQ statistics (2nd ed.). St. Louis: Mosby.
35.
Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology, 25, 241-286.
36.
Posavac, E. J. (1998). Toward more informative uses of statistics: Alternatives for program evaluators. Evaluation and Program Planning, 21, 243-254.
37.
Riffenburgh, R. H. (1999). Statistics in medicine. San Diego: Academic Press.
38.
Shaver, J. P. (1993). What statistical significance testing is, and what it is not. Journal of Experimental Education, 61, 293-316.
39.
Thompson, B. (1993). The use of statistical significance tests in research: Bootstrap and other alternatives. Journal of Experimental Education, 61, 361-377.
40.
Velleman, P., & Wilkinson, L. (1994). Nominal, ordinal, interval, and ratio typologies are misleading. In I. Borg & P. P. Mohler (Eds.), Trends and perspectives in empirical social research (pp. 161-177). Berlin: Walter de Gruyter.
