Beyond the P Value: The Clinical Story of Pain after Inguinal Hernia Repair

Abstract

Background:

Clinical studies often define their findings as statistically significant based solely on a P value of less than .05. In hernia surgery, pain intensity is a key patient-reported outcome, commonly measured using the visual analogue scale (VAS). However, recent research indicates that, despite achieving statistical significance, a difference of less than 1 point on the VAS lacks clinical relevance. Our study aims to investigate how the leading journals in surgery and abdominal wall hernia surgery report superiority regarding postoperative pain using the VAS for inguinal hernia repair (IHR).

Methods:

We searched Hernia Journal, Surgical Endoscopy, Annals of Surgery, Surgery, World Journal of Surgery, and JAMA for comparative articles analyzing IHR. Our search was performed according to the recent guidelines, comprising articles published since 2018. We included all the studies that used the VAS pain scale as a pain intensity analysis tool. Two authors independently screened the articles, analyzing the presence of a clinical relevance cutoff definition for the VAS tool and of statistical significance, and if it suggested superiority of an intervention or control regarding the pain intensity based solely on the P value, despite having a VAS difference of less than 1 point.

Results:

The initial search resulted in 169 articles, of which 45 articles were included. Among the studies, 21 (46.7%) were randomized controlled trials (RCTs), 15 (33.3%) were cohort studies, and 9 (20%) were meta-analyses. Fifteen (33.3%) studies compared surgical techniques, 19 (42.2%) studies analyzed surgical materials, while the other studies compared anesthetic methods, patient characteristics, and intraoperative aspects such as hernia sac and internal orifice management. Surprisingly, none of the studies defined a clinical relevance cutoff for the VAS pain scale. Twenty-five (55.6%) studies presented a statistically significant difference between the groups analyzed. Among these studies, only 11 (44%) presented a VAS difference of 1 point in their analyses. However, among the 14 studies that presented no clinically relevant VAS difference, 12 (85.7%) supported a difference in pain intensity based solely on the P value, while only 2 (14.3%) highlighted that their results may not be of clinical relevance, both being meta-analyses. Interestingly, these findings were more evident among the randomized studies, of which nine (42.9%) RCTs presented statistical significance. However, only two (22.2%) among those statistically significant results presented a VAS difference of more than 1 point. Interestingly, none of the other seven RCTs highlighted that their results may not be of clinical significance.

Conclusion:

Our findings revealed that the widely recommended clinical relevance cutoff of a 1-point difference on the VAS tool is often overlooked in inguinal hernia clinical studies. Moreover, authors frequently focus on statistical significance, even when their results lack meaningful clinical relevance.