This research note examined contemporary issues for meta-analysis with an example from the communication disorders literature. A significant proportion of experimental and quasi-experimental research results are not published in the mainstream literature, which suggests that publication bias is a potential problem for synthesis studies. Nonetheless, 50% of synthesis studies (meta-analyses) in communication disorders do not include statistical or visual procedures, e.g., fail-safe Ns or funnel plots, to reinforce the integrity of results. Tests for publication bias are recommended as routine procedures for meta-analyses.
Get full access to this article
View all access options for this article.
References
1.
American Speech-Language-Hearing Association. (1996) ASHA annual convention program.Rockville, MD: Author.
2.
AmlaniA. M. (2001) Efficacy of directional microphone hearing aids: Meta-analytic perspective. Journal of the American Academy of Audiology, 12, 202–214.
3.
AndrewsG.GuitarB., & HowieP. (1980) Meta-analysis of the effects of stuttering treatment. Journal of Speech and Hearing Disorders, 45, 287–307.
4.
BeggC. B. (1994) Publication bias. In CooperH. & HedgesL. V. (Eds.), The handbook of research synthesis.New York: Russell Sage Found. Pp. 399–409.
5.
BeggC. B., & MazumbarM. (1994) Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088–1101.
6.
BhogalS. K.TeasellR., & SpeechleyM. (2003) Intensity of aphasia therapy, impact on recovery. Stroke, 34, 987–993.
7.
BoutsenF.CannitoM. P.TaylorM., & BenderB. (2002) Botox treatment in adductor spasmodic dysphonia: Meta-analysis. Journal of Speech, Language, and Hearing Research, 45, 469–481.
8.
CarsonK. P., & SchriesheimC. A. (1990) The usefulness of the “fail-safe” statistic in meta-analysis. Educational and Psychological Measurement, 50, 233–244.
9.
CasbyM. W. (2001) Otitis media and language development: Meta-analysis. American Journal of Speech-Language Pathology, 10, 65–80.
10.
ChardD. J.VaughnS., & TylerB. J. (2002) A synthesis of research on effective interventions for building reading fluency with elementary students with learning disabilities. Journal of Learning Disabilities, 35, 386–406.
11.
EggerM., & SmithG. D. (1997) Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315, 615–634.
12.
FeyereisenP. (1997) A meta-analytic procedure shows an age-related decline in picture naming: Comments on Goulet, Ska, and Kahn (1994). Journal of Speech, Language, and Hearing Research, 40, 1328–1333.
13.
HunterJ. E., & SchmidtF. L. (1990) Methods of meta-analysis: Correcting error and bias in research findings.Newbury Park, CA: Sage.
14.
LapsdatJ. C. (1991) Pragmatic language skills of students with language and/or learning disabilities: Quantitative synthesis. Journal of Learning Disabilities, 24, 147–158.
15.
LightR. J.SingerJ. D., & WillettJ. B. (1994) The visual presentation and interpretation of meta-analysis. In CooperH. & HedgesL. V. (Eds.), The handbook of research synthesis.New York: Russell Sage Found. Pp. 439–453.
16.
MacaskillP.WalterS. D., & IrwigL. (2001) A comparison of methods to detect publication bias in meta-analysis. Statistics in Medicine, 20, 641–654.
17.
McGueM. (2000) Authorship and intellectual property. In SalesB. D. & FolkmanS. (Eds.), Ethics in research with human participants.Washington, DC: American Psychological Association. Pp. 75–95.
18.
MurtaughP. A. (2002) Journal quality, effect size, and publication bias in meta-analysis. Ecology, 83, 1162–1166.
19.
NyeC.FosterS. H., & SeamanD. (1987) Effectiveness of language intervention with the language/learning disabled. Journal of Speech and Hearing Disorders, 52, 348–357.
20.
OrwinR. G. (1983) A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8, 157–159.
21.
PhamB.PlattR.McAuleyL.KlassenT. P., & MoherD. (2001) Is there a “best” way to detect and minimize publication bias? An empirical evaluation. Evaluation and the Health Professions, 24, 109–125.
22.
RobeyR. R. (1994) The efficacy of treatment for aphasic persons: Meta-analysis. Language, 47, 582–608.
23.
RobeyR. R. (1998) A meta-analysis of clinical outcomes in the treatment of aphasia. Journal of Speech, Language, and Hearing Research, 41, 172–187.
24.
RobeyR. R., & DaleboutS. D. (1998) A tutorial on conducting meta-analyses of clinical outcome research. Journal of Speech, Language, and Hearing Research, 41, 1227–1241.
25.
RoggmanL. A.LangloisJ. H.Hubbs-TaitL., & Rieser-DannerL. A. (1994) Infant daycare, attachment, and the “file drawer problem.”Child Development, 65, 1429–1443.
26.
RosenthalR. (1979) The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–641.
27.
WangM. C., & BushmanB. J. (1998) Using the normal quantile plot to explore meta-analytic data sets. Psychological Methods, 3, 46–54.
28.
WhurrR.LorchM. P., & NyeC. (1992) A meta-analysis of studies carried out between 1946 and 1998 concerned with the efficacy of speech and language therapy treatment for aphasic patients. European Journal of Disorders of Communication, 27, 1–17.
29.
WortmanP. M. (1994) Judging research quality. In CooperH. & HedgesL. V. (Eds.), The handbook of research synthesis.New York: Russell Sage Found. Pp. 98–109.
