Abstract
Martin, L.T., Burns, R.M. & Schonlau, M. (2010). Mental disorders among gifted and nongifted youth: A selected review of the epidemiologic literature. Gifted Child Quarterly, 54, 31-41. DOI: 10.1177/0016986209352684
An error impacting the results of our analysis occurred when we inadvertently flipped the interpretation of anxiety scores in the Tong and Yewchuk (1996) study. Upon reanalysis with the corrected number, we found the effect was not significant at the 5% significance level, but there is evidence of an effect at the 10% significance level. Rather than dichotomizing the interpretation based on p < .05, some prefer reporting the “degree of evidence”: there was weak evidence (.05 < p < .10) of a difference in anxiety between gifted and nongifted students in this meta-analysis. Below is the corrected Figure 2 based on this statement (using a 90% CI). In recent years, the “ASA Statement on Statistical Significance and p-Values” (Wasserstein & Lazar, 2016) and others have warned against dichotomizing results into significant and not significant using p-values as the sole marker of association. We are encouraged to emphasize effect size in addition to commenting on the p-value. Our revised effect size is ES = -0.48, meaning that the level of anxiety is reduced by almost half a standard deviation. This is considered a moderate or medium effect size. The results remain suggestive of an association that may warrant additional study.
(Corrected) Forest plot for anxiety studies based on the random effect model.a
Tong, J., & Yewchuk, C. (1996). Self-concept and sex-role orientation in gifted high school students. Gifted Child Quarterly, 40, 15-23. https://doi.org/10.1177/001698629604000103
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129-133. https://doi.org/10.1080/00031305.2016.1154108
Editor’s Note
The original Martin et al. (2010) article included the statement, “We conducted all analyses using Stata10 with p < .05 considered significant” (p. 34). All plots in the original article presented the results with 95% CI and significance is reported for analyses with p < .05. Following the corrected analysis, the authors chose to present the data using an altered metric, assuming a revised statistical significance threshold of p < .10 and a plot with 90% CI for the effect sizes. Had the error been caught before the article’s publication in 2010, the authors would likely have claimed, “gifted children [did not have] significantly lower levels of anxiety compared with their nongifted peers” (p. 37), as was the case for depression in their study. The confidence interval for the overall effect size in the corrected analysis included zero in the 95% CI plot, but did not in the 90% CI plot. A simple correction would have maintained the assumptions presented in the original article.
It should be noted that, although the point estimate of the overall effect (-0.48) can be considered a moderate or a medium effect size (Cohen, 1988), the wide range in the confidence interval (-0.92, -0.04) suggests the possibility that the effect size could be very large, moderate, or negligible. The range was similarly large for 95% CI: (-1.00, 0.04). Thus, the results are not definitive and further study is indicated.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.