Whatever Happened to the T-Test?

Abstract

A review of the 1977 and the 1994 British Medical Journal (BMJ) was undertaken to determine how the application of statistics to medical research may have changed over the last two decades. The measures of outcome chosen were statistical (co-)authorship of, and references to, the statistical literature in papers and the statistical techniques and methodologies applied in the research presented. The 17 years between the two issues of the BMJ witnessed a considerable change as regards any of the chosen parameters mentioned, with a doubling in the number of papers with statistical (co-)authorship, a five-fold increase in the number of papers containing statistical references, and references to statistical software in more than one third of the papers in 1994, compared to no more than in 1% in 1977.

Despite a 50% reduction in papers representing medical research which had not undergone any statistical evaluation, this less than rigorous, in a scientific “evidence-based medicine” sense, type of work, still represented 21% of published papers in 1994. Many desirable changes, however, were seen in the reporting of statistical considerations. Thus, for example, the habit of quoting means ± “something” had all but disappeared in 1994 (replaced with more meaningful indications of dispersion/precision), p-values without any mention of a statistical test is on the way out, and statistical significance is now more frequently defined. Simple statistical techniques were still applied virtually to the same extent in 1994 as in 1977, but much recent research is now subjected to more sophisticated statistical evaluation, partly through the availability of such techniques (in themselves a result of statistical research), partly perhaps because of an appreciation of the usefulness of some of these newer techniques in the context of ever more complex medical research.