Articles published in professional journals are expected to comply with the 1994 publication manual of the American Psychological Association, which acknowledges that p values are not acceptable measures of effect size magnitudes, and explicitly encourages authors to report effect sizes. This “encouragement” is also consistent with recent criticisms of common misuses of statistical significance tests. The present article examined effect size reporting in 23 quantitative articles reported in Exceptional Children. It was found that effect sizes are rarely being reported, though exemplary reporting practices were also noted Reasons why the APA “encouragement” has been ineffective are explored and it is suggested that an editorial policy requiring effect size reporting similar to that invoked at several journals, including some APA journals, may be useful.
Get full access to this article
View all access options for this article.
References
1.
American Psychological Association. (1994). Publication manual of the American Psychological Association (4th ed.). Washington, DC: Author.*
2.
AtkinsonR. C., & JacksonG. B. (Eds.). (1992). Research and education reform: Roles for the Office of Educational Research and Improvement.Washington, DC: National Academy of Sciences. (ERIC Document Reproduction Service No. ED 343 961)
3.
BennettT., DelucaD., & BrunsD. (1997). Putting inclusion into practice: Perspectives of teachers and parents.Exceptional Children, 64, 115–131.
4.
BenzM. R., YovanoffP., & DorenB. (1997). School-to-work components that predict postschool success for students with and without disabilities.Exceptional Children, 63, 151–165.
5.
BorgenF. H., & SelingM. J. (1978). Uses of discriminant analysis following MANOVA: Multivariate statistics for multivariate purposes.Journal of Applied Psychology, 63, 689–697.
6.
CarnineD. (1997). Bridging the research-to-practice gap.Exceptional Children, 63, 513–521.
CohenJ. (1965). Some statistical issues in psychological research. In WolmanB. B. (Ed.), Handbook of clinical psychology (pp. 95–121). New York: McGraw-Hill.*
9.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.*
10.
CohenJ. (1994). The earth is round (p < .05).American Psychologist, 49, 997–1003.
11.
The Council for Exceptional Children. (1997). Author Guidelines.Exceptional Children, 64, 139–140.
12.
FavazzaP., & OdomS. L. (1997). Promoting positive attitudes of kindergarten-age children toward people with disabilities.Exceptional Children, 63, 405–418.
13.
FriedmanH. (1982). Simplified determinations of statistical power, magnitude of effect and research sample size.Educational and Psychological Measurement, 42, 521–526.
14.
GallM. D., BorgW. R., & GallJ. P. (1996). Educational research: An introduction (6th ed.). White Plains, NY: Longman.*
15.
GlassG. V. (1979). Policy for the unpredictable (uncertainty research and policy).Educational Researcher, 8(9), 12–14.
16.
HallB. W., WardA. W., & ComerC. B. (1988). Published educational research: An empirical study of its quality.Journal of Educational Research, 81, 182–189.
17.
Heldref Foundation. (1997). Guidelines for contributors.Journal of Experimental Education, 65, 95–96.
18.
HubertyC. (1994). Applied discriminant analysis.New York: Wiley and Sons.*
19.
KeselmanH. J., HubertyC. J., LixL. M., OlejnikS., CribbieR., DonahueB., KowalchukR. K., LowmanL. L., PetoskeyM. D., KeselmanJ. C., & LevinJ. R. (in press). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOVA analyses.Review of Educational Research.
20.
KirkR. (1996). Practical significance: A concept whose time has come.Educational and Psychological Measurement, 56, 746–759.
21.
KnappT. R. (1978). Canonical correlation analysis: A general parametric significance testing system.Psychological Bulletin, 85, 410–416.
22.
MorrisseyP. A. (1997). Response to bridging the research-to-practice gap.Exceptional Children, 63, 531–532.
23.
MurphyK. R. (1997). Editorial.Journal of Applied Psychology, 82, 3–5.
24.
NelsonN., RosenthalR., & RosnowR. L. (1986). Interpretation of significance levels and effect sizes by psychological researchers.American Psychologist, 41, 1299–1301.
25.
OakesM. (1986). Statistical inference: A commentary for the social and behavioral sciences.New York: Wiley.*
26.
OlejnikS. F. (1984). Planning educational research: Determining the necessary sample size.Journal of Experimental Education, 53, 40–48.
27.
PomplumM. (1997). When students with disabilities participate in cooperative groups.Exceptional Children, 64, 49–58.
28.
RosenthalR., & GaitoJ. (1963). The interpretation of level of significance by psychological researchers.Journal of Psychology, 55, 33–38.
29.
Saint-LaurentL., DionneJ., GiassonJ., RoyerE., SimardC., & PierardB. (1998). Academic achievement effects of an in-class service model on students with and without disabilities.Exceptional Children, 64, 239–253.
30.
SchmidtF. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for the training of researchers.Psychological Methods, 1, 115–129.
31.
SextonD., LobmanM., ConstansT., SnyderP., & ErnestJ. (1997). Early interventionists' perspectives of multicultural practices with African-American families.Exceptional Children, 63, 313–328.
32.
ShaverJ. (1985). Chance and nonsense.Phi Delta Kappan, 67(1), 57–60.
33.
SnyderP., & LawsonS. (1993). Evaluating results using corrected and uncorrected effect size estimates.Journal of Experimental Education, 61, 334–349.
34.
SnyderP. A., & ThompsonB. (in press). Use of tests of statistical significance and other analytic choices in a school psychology journal: Review of practices and suggested alternatives.School Psychology Quarterly.
35.
SontagJ. C. (1997). Contextual factors influencing the sociability of preschool children with disabilities in integrated and segregated classrooms.Exceptional Children, 63, 389–404.
36.
SydoriakD., & FieldsM. J. (1997). Response to bridging the research-to-practice gap.Exceptional Children, 63, 529–530.
37.
ThompsonB. (1992). Two and one-half decades of leadership in measurement and evaluation.Journal of Counseling and Development, 70, 434–438.
38.
ThompsonB. (1993). The use of statistical significance tests in research: Bootstrap and other alternatives.Journal of Experimental Education, 61, 361–377.
39.
ThompsonB. (1994a). Guidelines for authors.Educational and Psychological Measurement, 54, 837–847.
40.
ThompsonB. (1994b, February). Why multivariate methods are usually vital in research: Some basic concepts. Paper presented as a Featured Speaker at the biennial meeting of the Southwestern Society for Research in Human Development (SWSRHD), Austin, TX. (ERIC Document Reproduction Service No. ED 367 687)
41.
ThompsonB. (1995). Stepwise regression and stepwise discriminant analysis need not apply here: A guidelines editorial.Educational and Psychological Measurement, 55, 525–534.
ThompsonB. (1998a, April). Five methodology errors in educational research: The pantheon of statistical significance and other faux pas. Invited address presented at the annual meeting of the American Educational Research Association, San Diego.
44.
ThompsonB. (1998b). Review of What if there were no significance tests?Educational and Psychological Measurement, 58, 332–344.
45.
ThompsonB. (in press-a). If statistical significance tests are broken/misused, what practices should supplement or replace them?Theory & Psychology.
46.
ThompsonB. (in press-b). Journal editorial policies regarding statistical significance tests: Heat is to fire as p is to importance.Educational Psychology Review.
47.
ThompsonB., & SnyderP. A. (1998). Statistical significance and reliability analyses in recent JCD research articles.Journal of Counseling and Development, 76, 436–441.
48.
TryonW. W. (1998). The inscrutable null hypothesis.American Psychologist, 53, 796.
49.
TuckmanB. W. (1990). A proposal for improving the quality of published educational research.Educational Researcher, 19(9), 22–24.
50.
VaughnS., MoodyS. W., & SchummJ. S. (1998). Broken promises: Reading instruction in the resource room.Exceptional Children, 64, 211–225.
51.
VockellE. L., & AsherW. (1974). Perceptions of document quality and use by educational decision makers and researchers.American Educational Research Journal, 11, 249–258.
52.
WandtE. (1967). An evaluation of educational research published in journals (Report of the Committee on Evaluation of Research). Washington, DC: American Educational Research Association.*
53.
WardA. W., HallB. W., & SchrammC. E. (1975). Evaluation of published educational research: A national survey.American Educational Research Journal, 12, 109–128.
54.
Wenz-GrossM., & SipersteinG. N. (1997). Importance of social support in the adjustment of children with learning problems.Exceptional Children, 63, 183–193.
55.
Whitney-ThomasJ., & Hanley-MaxwellC. (1997). Packing the parachute: Parents' experiences as their children prepare to leave high school.Exceptional Children, 63, 75–87.
56.
ZuckermanM., HodginsH. S., ZuckermanA., & RosenthalR. (1993). Contemporary issues in the analysis of data: A survey of 551 psychologists.Psychological Science, 4, 49–53.
