Considering Statistical Logic in Otology

Abstract

Objectives:

Illustrate strong and weak statistical concepts used in otolaryngologic papers submitted for publication regardless of acceptance.

Methods:

For 1 calendar year, the type and number of statistical concerns were recorded for 56 otologic papers submitted for publication to 1 of 5 journals.

Results:

Ninety-nine concerns were found in 32 of the 56 papers. Reviewers identified several concerns in many papers. For some papers, data reanalysis was possible with different statistical tests, but in none was the conclusion altered. A summary of the findings with some overlap of the categories follows. Conclusions not supported by data – 13. Confusion of correlation with causation – 13. “Fishing trip” – 12. Overgeneralization – 9. Multiple t tests for related outcomes – 8. Multiple tests - inappropriate tests – 8. Parametric tests were used but underlying assumptions clearly violated – 7. Conclusion inappropriate or absent – 7. Lack of required power analysis – 4. Inappropriate assignment to groups when assessing group differences – 3. Partial data analysis – 3. Reviewer requests power analysis when not needed – 2. Reviewer requests inappropriate tests - 2.

Conclusions:

Application of statistics in otology papers was surprisingly good. The most common flaws in statistical interpretation in submitted papers were (1) that the conclusion reported was not supported by the data or (2) confusion between association and causation. Even if statistics were applied inappropriately, thoughtful reviewers were generally able to detect errors in statistical logic even if they did not feel that they were qualified as statisticians.