The credibility of subgroup analyses reported in stroke trials is low: A systematic review

Abstract

Background:

Subgroup analyses are widely used to evaluate the heterogeneity of treatment effects in randomized clinical trials. However, there is a limited investigation of the quality of prespecified and reported subgroup analyses in stroke trials. This study evaluated the credibility of subgroup analyses in stroke trials.

Methods and analysis:

We searched Medline/PubMed, Embase, the Cochrane Central Register of Controlled Trials, and the Web of Science from inception to 24 March 2021. Three reviewers screened, extracted, and analyzed the data from the publications. Primary publications of stroke trials that reported at least one subgroup effect and had published corresponding study protocols were included. The Instrument for Assessing the Credibility of Effect Modification Analyses (ICEMAN) was used to examine the quality of the subgroup effects reported, with each subgroup effect assigned a credibility rating ranging from very low to high. Subgroup effects with two or more “definitely no” responses received a low credibility rating. The risk of bias was assessed using the Cochrane Risk-of-Bias tool for randomized trials version 2.

Results:

Seventy-four articles met the inclusion criteria and reported a combined total of 647 subgroup effects. The median sample size was 1264 (interquartile range (IQR): 380–3876), and the median number of subgroups prespecified in the protocol was 6 (IQR: 2–10). Sixty-one (82%) studies used the univariate test of interaction. Of the total 647 subgroup effects reported in these studies, 319 (49%) were reported in acute stroke trials, while 423 (65%) had low credibility.

Conclusion:

The quality of subgroup analysis reporting in stroke trials remains poor. More effort is needed to train trialists on the best methods for designing and performing subgroup analyses, and how to report the results.

Trial registration number:

We prospectively registered the review with International Prospective Register for Systematic Reviews (registration number: CRD42020223133)

Keywords

Systematic review randomized controlled trial stroke subgroup analysis ICEMAN

Key messages

1. What is the issue and what we know so far about it?

■ Credibility of subgroup effects in clinical trials remains a major challenge because of the perils of subgroups that include increased risk of false-positive and false-negative conclusions

■ This study aims to assess the credibility of reported subgroup analyses in published stroke trials using the Instrument for Assessing the Credibility of Effect Modification Analyses (ICEMAN) criteria

2. What are the key findings from this study?

■ The credibility of reported subgroup analyses in stroke trials remained generally low.

■ Most stroke trials still do not report on the justification or direction for effect modification for the selected subgroups

■ Many of the studies that were reviewed used the incorrect statistical method to assess subgroup effects

3. What are the implications of the findings?

■ More effort is needed to train trialists on the best methods for designing and performing subgroup analyses, and how to report the results

■ Journals need to include the requirements to assess the credibility of reported subgroup effects as part of their reporting

Recommendations

1. Trialists must be better educated on the best methods for conducting subgroup analyses to prevent avoid misleading results

2. Biological and sample size considerations should take precedence to minimize data dredging

3. Journal editors and professional societies play an important role in improving the credibility of reported subgroup analyses

4. Checklists, such as the ICEMAN checklist, should be considered necessary for the publication of trial protocols and subsequent subgroup findings

Introduction

Evaluation of treatment effect heterogeneity is an integral part of the analysis of clinical trials.^1–3 Identifying subgroups with different treatment effects generate new hypotheses for future clinical trial research and enables interventions to be tailored to specific patients. Subgroup analyses are commonly performed in stroke randomized controlled trials (RCTs) to evaluate the heterogeneity of treatment effects.^4,5 Reporting the overall treatment effect without accounting for subgroup effects can be misleading when patients with different characteristics respond differently to the same intervention. In addition, incorrectly reported subgroup analyses lead to wrong conclusions, which harm clinical policies and practices.^6,7

Several studies have drawn the attention of the clinical trials community to the prevalence and impact of incorrect reporting of subgroup analysis to support claims of treatment effect heterogeneity.^8–11 Subgroup analysis was incorporated into the Consolidated Standards of Reporting Trials (CONSORT) statement in response to these findings.¹² Despite the addition of subgroup analysis to the CONSORT guidelines, selective reporting of significant subgroup analyses, a lack of prior evidence on potentially relevant subgroups, and a failure to use appropriate statistical analysis remain major concerns.^13,14

Many authors have developed guidelines and checklists to improve the conduct and reporting of subgroup analysis findings.^15–19 However, these checklists vary in the length and type of criteria included, which creates ambiguity in their applications. Schandelmaier et al.²⁰ recently created the Instrument for Assessing the Credibility of Effect Modification Analyses (ICEMAN) checklist, a shorter five-item Likert-validated checklist for assessing the credibility of subgroup analyses in RCTs. The ICEMAN checklist reduces the number of recommendations and assigns a credibility rating to each subgroup effect ranging from very low to high. While Farrokhyar et al.²¹ mentioned ICEMAN as a detailed checklist for systematic reviews, Kilpeläinen et al.²² used the ICEMAN checklist to investigate the credibility of subgroup effects in urology trials. Until now, the credibility of subgroup analyses reported in stroke trials has not been examined. This study aims to assess the credibility of reported subgroup analyses in published stroke trials.

Methods

Design and registration

This systematic review followed the guidelines outlined in the Cochrane Handbook of Systematic Reviews of Interventions.²³ The review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.²⁴ The review protocol was prospectively registered at PROSPERO (registration number: CRD42020223133). The review was conducted using the Studies, Data, Methods, and Outcomes (SDMO) framework.²⁵ The study design (S) focused on RCTs, and the data (D) were primary stroke trial results published in any biomedical journal. The method (M) contrasts the prespecified analyses in protocol to those in the published results, and the outcome (O) was the reporting quality subgroup analysis in stroke clinical trials. The three medical subject headings (MeSH) used were RCTs, stroke trials, and subgroup analysis/effect modifications (Tables S1–S4: Supplementary Materials).

Eligibility criteria and search strategy

Eligible studies are included in this review if they (1) reported at least one subgroup analysis along with the primary outcome findings in the main publication, (2) had published corresponding study protocols, and (3) were RCTs. Non-human trials, systematic reviews, literature reviews, meta-analyses, gray literature, conference abstracts, and publications in languages other than English were excluded. Studies whose protocols were not published or made available elsewhere were also excluded. Four electronic medical databases were searched for relevant publications from inception until 24 March 2021, namely, (1) Medline/PubMed, (2) Embase, (c) the Cochrane Central Register of Controlled Trials (CENTRAL), and (3) Web of Science. The identified studies were imported into Covidence, a web-based application for conducting systematic reviews.²⁶ Covidence eliminated duplicate studies and provided independent reviewers with a customized review template.

Selection of studies and data extraction

Three reviewers (A.A., J.A., and A.O.) independently screened titles and abstracts in duplicate to identify articles that met the inclusion criteria. These reviewers completed the full-text review of each eligible study and retrieved the necessary data. Disagreements among reviewers were resolved via a consensus process involving T.S. and B.K.M. Data extracted from each included study were the first author’s name and country, year of publication, trial sample size, type of intervention, the prespecified subgroup analysis in the original protocol, and the number of subgroups analyzed in the primary publication. Other data extracted were the subgroup effect studied, the predicted direction of the effect modification, whether the authors justified the subgroup effect, and whether they used an appropriate statistical test.

Assessment of subgroup analyses’ credibility and risk of bias

The reviewers independently graded the credibility of each reported subgroup effect in duplicate using ICEMAN criteria. The ICEMAN questions (and answers) with the overall ratings are listed (Table S5 and Figure S1). Each subgroup effect was assigned a credibility rating ranging from “Very Low” to “High” based on the responses to the questions. A subgroup effect was assigned a “Very low” credibility rating if all the responses were “definitely no” or “probably no,” and a “Low credibility” rating if there were at least two “definitely no” responses. Furthermore, a subgroup effect was given a moderate credibility rating when there was only one “definitely no” or two “probably no” responses, and a high credibility rating when none of the responses were “definitely no” or “probably no.” Finally, the reviewers used the Cochrane Risk-of-Bias tool for randomized trials version 2 (RoB-2) to critically assess the risk of bias in duplicate for each study.²⁷

Data analysis

After rating the effects with the ICEMAN, a descriptive analysis of the eligible subgroup effects was performed. These analyses were stratified by the type of stroke trial (acute stroke, primary, and prevention), sample size, and credibility. Trend analysis was performed on the extracted meta-data using three time periods: publications before 2010, between 2010 and 2014, and publications after 2014. The time intervals were selected to account for scientific progress in stroke trials. The revised CONSORT statement was published in 2010, and 2015 was recognized as the year of endovascular treatment due to many landmark stroke trials.^12,28 Furthermore, the descriptive analysis of the risk of bias based on the five domains of the RoB-2 was provided. All analyses were conducted in R 4.2.0 and Stata 17.0.^29,30

Results

A total of 9234 studies were imported into Covidence from the four electronic databases, with 2889 duplicates removed and 6345 articles screened. The reviewers eliminated 5798 publications as irrelevant at the title and abstract screening stage. After excluding 473 studies at the full-text review stage, 74 publications met the inclusion criteria (Table S6). There was substantial agreement among the reviewers at the title and abstract screening, with an interrater agreement κ = 0.69. See the PRISMA workflow diagram in Figure 1. Also, the PRISMA checklist is shown in Table S7.

Figure 1.

The PRISMA flow diagram of study selection.

Characteristics of included studies

Table 1 describes the characteristics of the included studies. The 74 studies included in the review had a total sample size of 249,305 trial participants and 647 subgroup effects, with a median study sample size of 1264 (IQR: 380–3876). There were 42 (57%) acute stroke trials and 73 (99%) were multicenter trials. Twenty-five studies (34%) used the modified Rankin scale as the primary outcome measure, while 32 (43%) used time-to-event outcomes. Thirty studies (40%) had favorable results (i.e. significant treatment effect based on the primary outcome). The median number of subgroups prespecified in the protocol was 6 (IQR: 2–10), and the median number of subgroups examined was 8 (IQR: 5–12). The univariate test of interaction was used in 61 studies (82%).

Table 1.

The main characteristics of included articles (n = 74).

Characteristics	N (%)
Year of publications: n (%)
2015+	45 (61)
2010–2014	17 (23)
Before 2010	12 (16)
Journal impact factor: n (%)
<10.00	12 (16)
10.00–50.00	15 (20)
>50.00	47 (64)
Type of intervention: n (%)
Acute stroke treatment	42 (57)
Primary prevention	11 (15)
Secondary prevention	21 (28)
Primary outcome: n (%)
Modified Rankin scale	25 (34)
Recurrent events or death	32 (43)
Change in blood pressure	4 (5)
Others	13 (18)
Geographical region: n (%)
Europe	22 (30)
North America	11 (15)
Asia	8 (11)
Oceania	1 (1)
Multiple	32 (43)
Type of trial design: n (%)
Superiority	65 (88)
Non-inferiority	9 (12)
Trial design classification: n (%)
Explanatory	60 (81)
Pragmatic	14 (19)
Number of centers: n (%)
Multi-center	73 (99)
Single center	1 (1)
Source of funding: n (%)
Non-industry	46 (62)
Industry	20 (27)
Mixed	8 (11)
Was the main effect is significant? n (%)
Yes	30 (41)
No	44 (59)
Test of interaction^a: n (%)
Yes	61 (82)
No	13 (18)
Sample size
Total	249,305
Median (Q1–Q3)*	1264 (380–3876)
Sample category: n (%)
<200	10 (13)
200–999	21 (28)
1000–5000	28 (37)
>5000	15 (20)
Prespecified number of subgroup analysis: median (Q1–Q3)	6 (2–10)
Number of subgroups analyzed: median (Q1–Q3)	8 (5–12)

Q1: first quartile; Q3: third quartile.

One-at-a-time interaction.

Subgroup effect credibility and trend analysis

Tables 2 and 3 describe the characteristics and the credibility of subgroup effect reported in trial publications. Three hundred nineteen (49%) subgroup effects were in acute stroke trials, 421 (65%) subgroup effects were prespecified in the protocols, 185 (29%) subgroup effects were in analyses that took statistical power into account, and 40 (6%) subgroup effects were significant (Table 2). Six (1%) subgroup effects had an expected direction of effect, the reasons for conducting the analysis were provided in 146 (23%) subgroup effects, and 27 (4%) subgroup effects were in analyses that considered fewer subgroup effects (Table 3). The scatterplot comparing sample size to the number of subgroup effects examined is shown in Figure 2. Overall, 34 (5%) subgroup effects were very low in credibility, 423 (65%) were low in credibility, 184 (29%) were moderate in credibility, and 6 (1%) were high in credibility. Trend analyses of reported subgroup effects by year of publication show that 12 (11%) subgroup effects reported in trials published before 2010 had moderate credibility rating, while 130 (35%) subgroup effects reported in trials published after 2014 had moderate credibility rating (Figure 3).

Table 2.

Characteristics of subgroup effect reported in trial publications (n = 647).

Characteristics	N (%)
Prespecified subgroup effect: n (%)
Acute stroke treatment	319 (49)
Primary prevention	114 (18)
Secondary prevention	214 (33)
Prespecified subgroup effect: n (%)
Yes	421 (65)
No	226 (35)
Was the subgroup effect described? n (%)
Yes	354 (55)
No	293 (45)
Was direction effect anticipated: n (%)
Yes	6 (1)
No	641 (99)
Arbitrary categorized of continuous subgroup effects^a: n (%)
Yes	237 (37)
No	12 (2)
Not applicable	398 (61)
Was statistical power considered? n (%)
Yes	185 (29)
No	462 (71)
Is subgroup effect significant? n (%)
Yes	40 (6)
No	607 (94)

Categorized the subgroup effect without providing justification.

Table 3.

The credibility of subgroup effects (n = 647).

Characteristics	N (%)
Was the direction of the effect correctly hypothesized? n (%)
Probably no	229 (35)
Probably yes	412 (64)
Definitely yes	6 (1)
Was the effect supported by prior evidence? n (%)
Little or no support	501 (77)
Some support	146 (23)
Was there any effect due to the test of interaction? n (%)
Chance a very likely explanation	607 (94)
Chance a likely explanation or unclear	11 (2)
Chance may not explain	29 (4)
Was a small number of effects considered? n (%)
Definitely no	390 (60)
Probably no or unclear	230 (36)
Definitely yes	27 (4)
Avoided arbitrary cut-points for subgroup effect? n (%)
Probably no or unclear	237 (37)
Probably yes	12 (2)
Definitely yes	398 (61)
Subgroup effect rating^a: n(%)
Very low credibility	34 (5)
Low credibility	423 (65)
Moderate credibility	184 (29)
High credibility	6 (1)

ICEMAN: Instrument for Assessing the Credibility of Effect Modification Analyses.

Rating was performed using the ICEMAN checklists.

Figure 2.

Scatterplot of sample size against the number of subgroup effects analyzed.

Figure 3.

Trend analysis of quality of reporting among the included studies.

Risk of bias

Three bias risk domains were at least 90% low risk (Figure S2). Many studies used an allocation sequence, concealed treatment allocation, and balanced baseline covariates. Few studies used the prospective randomized open-blinded endpoint (PROBE) study design, making blinding participants and trial personnel difficult. However, this is a design feature of the study rather than a deviation from the intended intervention domain. The measurement of the outcome and the selection of the reported result domains had reduced risk relative to other domains (i.e. 74% and 55%, respectively). Using an ineffective outcome measure and the failure to blind the outcome assessors led to a high risk of bias in the measurement of the outcome domain. The discrepancy between the prespecified analyses in the protocol and the published analyses was the source of the increased risk within the selection of reported results domain.

Discussion

Evidence from this review revealed that the credibility of reported subgroup analyses in stroke trials remained generally poor. Despite the plethora of checklists available to guide subgroup analysis, majority of stroke trials still do not report on the rationale or direction for effect modification for the selected subgroups. In addition, most reviewed studies used the incorrect statistical method (i.e. univariate interaction), resulting in multiple statistical significance tests and an inflated overall Type II error.

The poor credibility of report subgroup analyses highlights the need for a multipronged approach to address this issue. Regrettably, the low credibility of published subgroup analyses observed in this review of stroke trials is consistent with findings from systematic reviews of published trials in other disciplines. The used of the ICEMAN checklist in other studies have showed that subgroup effects had a very low to low credibility rating. Kilpeläinen et al.²² criticized the poor conduct and reporting of subgroup analyses in urology trials. Using data from the well-known Prostate Cancer Intervention Versus Observation Trial (PIVOT) trial³¹ that has influenced clinical practice guidelines,³² they demonstrated the use of ICEMAN for assessing the credibility of findings from subgroup analyses and argued that results of subgroup analysis of this trial had low credibility rating. Furthermore, Saragiotto et al.³³ examine the credibility of subgroup analyses in back pain trials and concluded that subgroup analyses in these published trials had low credibility rating. Similarly, Wallach et al.³⁴ discovered that efforts to verify statistically significant subgroup differences claimed in many RCTs are uncommon, and when they do occur, the claimed subgroup differences are not replicated. Here we provide a few recommendations to address this issue. First, there is need for more education of trialists on the best methods for performing subgroup analyses to minimize misleading results. Given that many reported subgroup analyses are underpowered, biological and sample size considerations should be used to guide the selection of few subgroups that will be conducted to avoid data dredging.^35–37 On the other hand, editorial boards of journals and professional societies play an important gate-keeping role in improving the credibility of reported subgroup analyses. We recommend medical journals adopt editorial policies that mandate authors to report subgroup analyses consistent with an approved checklist, such as ICEMAN checklist, as part of their requirements for publishing trial protocols and subsequent findings.

Our review included many studies to ensure that the credibility of stroke trial subgroup effects reporting was thoroughly examined. Rather than focusing on a single intervention, the review was designed to include all stroke trial settings (primary, secondary, and acute stroke treatment). A major limitation of this study is the generalizability of our study findings. This review of published stroke trials was not exhaustive since it was restricted to the top four medical databases excluded studies in languages other than English. It is possible we might have missed other relevant studies. Second, this review focused on reported results from primary publications from each trial and excluded trials in which subgroup analyses are only reported in detail in a separate paper. Finally, our review excluded 88 trials without published protocols (as a separate manuscript or as a supplementary document to the primary publication). These excluded studies are more likely to be trials published in journals where publication of study protocols is not mandated at all, or published before medical journals mandated trial protocol publication. Nevertheless, we believe that this study is inclusive of high-quality stroke trials and the reported findings are likely to be unchanged even if eligibility criteria are relaxed to be inclusive of all stroke trials.

In conclusion, this review highlights the need for improvement in the credibility of subgroup analyses in stroke trials. Although there have been some trends toward improved reporting of subgroup analyses in recent years, more work is needed to achieve a reasonable standard for reporting subgroup findings. Reporting guidelines, such as the ICEMAN checklists, are recommended to guide the choice of subgroups to be investigated for treatment heterogeneity and the appropriate analyses to be conducted to improve the credibility of reported subgroup analyses in stroke trials.

Supplemental Material

sj-docx-1-wso-10.1177_17474930231168517 – Supplemental material for The credibility of subgroup analyses reported in stroke trials is low: A systematic review

Supplemental material, sj-docx-1-wso-10.1177_17474930231168517 for The credibility of subgroup analyses reported in stroke trials is low: A systematic review by Ayoola Ademola, Lehana Thabane, Joel Adekanye, Ayooluwanimi Okikiolu, Samuel Babatunde, Mohammed A Almekhlafi, Bijoy K Menon, Michael D Hill, Kevin A Hildebrand and Tolulope T Sajobi in International Journal of Stroke

Footnotes

Authors’ contribution

AA, LT, and TTS conceptualized the study; AA drafted the initial version of the manuscript; AA, JA, AO, and SB participated in the review and data extraction. All authors read, critically revised, and approved the final manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: AA received doctoral funding from the Prevention of Post-Traumatic Contractures with Ketotifen II (PERK II) (supported by the United States Army Medical Research Acquisition Activity, United States Department of Defense). Also, AA received the Eyes High International Doctoral Scholarship and the Alberta Graduate Excellence Scholarship from the University of Calgary.

ORCID iDs

Ayoola Ademola

Michael D Hill

Availability of data and materials

The data obtained or analyzed during this study were included in the manuscript (and its supplementary information files).

Supplemental material

Supplemental material for this article is available online.

References

Angus

Chang

CCH

. Heterogeneity of treatment effect: estimating how the effects of interventions vary across individuals. JAMA 2021; 326: 2312–2313.

Tanniou

van der Tweel

Teerenstra

Roes

LCB

. Subgroup analyses in confirmatory clinical trials: time to be specific about their purposes. BMC Med Res Methodol 2016; 16: 20.

Vijan

. Evaluating heterogeneity of treatment effects. Biostat Epidemiol 2020; 4: 98–104.

Rothwell

. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet 2005; 365: 176–186.

Wang

Lagakos

Ware

Hunter

Drazen

. Statistics in medicine—reporting of subgroup analyses in clinical trials. New Engl J Med 2007; 357: 2189–2194.

Brookes

Whitley

Peters

Mulheran

Egger

Davey Smith

. Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives. Health Technol Assess 2001; 5: 1–56.

Schulz

Grimes

. Multiplicity in randomised trials II: subgroup and interim analyses. Lancet 2005; 365: 1657–1661.

Randomized trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2(Second International Study of Infarct Survival) Collaborative Group. J Am Coll Cardiol 1988; 12: A3–A13.

Hirji

Fagerland

. Outcome based subgroup analysis: a neglected concern. Trials 2009; 10: 33–33.

10.

Horton

. From star signs to trial guidelines. Lancet 2000; 355: 1033–1034.

11.

Sleight

. Debate: subgroup analyses in clinical trials: fun to look at—but don’t believe them. Curr Control Trials Cardiovasc Med 2000; 1: 25–27.

12.

Schulz

Altman

Moher

, et al. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Medicine 2010; 8: 18.

13.

Evans

. Common statistical concerns in clinical trials. J Exp Stroke Transl Med 2010; 3: 1–7.

14.

Gabler

Duan

Raneses

, et al. No improvement in the reporting of clinical trial subgroup effects in high-impact general medical journals. Trials 2016; 17: 320.

15.

Dijkman

Kooistra

Bhandari

. How to work with a subgroup analysis. Can J Surg 2009; 52: 515–522.

16.

Guillemin

. Primer: the fallacy of subgroup analysis. Nat Clin Pract Rheumatol 2007; 3: 407–413.

17.

Oxman

Guyatt

. A consumer’s guide to subgroup analyses. Ann Intern Med 1992; 116: 78–84.

18.

Rothwell

. Treating individuals 2. Subgroup analysis in randomised controlled trials: importance, indications, and interpretation. Lancet 2005; 365: 176–186.

19.

Sun

Briel

Walter

Guyatt

. Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ 2010; 340: pc117.

20.

Schandelmaier

Briel

Varadhan

, et al. Development of the Instrument to assess the credibility of effect modification analyses (ICEMAN) in randomized controlled trials and meta-analyses. Canad Med Assoc J 2020; 192: E901–E906.

21.

Farrokhyar

Skorzewski

Phillips

, et al. When to believe a subgroup analysis: revisiting the 11 criteria. Eye 2022; 36: 2075–2077.

22.

Kilpeläinen

Tikkinen

KAO

Guyatt

Vernooij

RWM

. Evidence-based urology: subgroup analysis in randomized controlled trials. Eur Urol Focus 2021; 7: 1237–1239.

23.

Cumpston

Page

, et al. Updated guidance for trusted systematic reviews: a new edition of the cochrane handbook for systematic reviews of interventions. Cochrane Database Syst Rev 2019; 10: ED000142.

24.

Page

McKenzie

Bossuy

, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71.

25.

Munn

Stern

Aromataris

Lockwood

Jordan

. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol 2018; 18: 5–5.

26.

Jessica

. Product review: covidence (Systematic Review Software). J Canad Health Libr Assoc 2014; 35: 68–71.

27.

Sterne

JAC

. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019; 366.

28.

Muir

. Stroke in 2015: the year of endovascular treatment. Lancet Neurol 2016; 15: 2–3.

29.

StataCorp. Stata Statistical Software: Release 17. College Station, TX: Statacorp LLC, 2021.

30.

Team

. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria, 2021, https://www.R-project.org/

31.

Wilt

Langsetmo

, et al. Radical prostatectomy or observation for clinically localized prostate cancer: extended follow-up of the prostate cancer intervention versus observation trial (PIVOT). Eur Urol 2020; 77: 713–724.

32.

Mottet

van de Bergh

RCN

Briers

, et al. EAU-EANM-ESTRO-ESUR-SIOG Guidelines on Prostate Cancer-2020 Update. Part 1: screening, diagnosis, and local treatment with curative intent. Eur Urol 2021; 79: 243–262.

33.

Saragiotto

Maher

Moseley

, et al. A systematic review reveals that the credibility of subgroup claims in low back pain trials was low. J Clin Epidemiol 2016; 79: 3–9.

34.

Wallach

Sullivan

Trepanowski

Sainani

Steyerberg

Ioannidis

JPA

. Evaluation of evidence of statistical support and corroboration of subgroup claims in randomized clinical trials. JAMA Intern Med 2017; 177: 554–560.

35.

Faber

Fonseca

. How sample size influences research outcomes. Dental Press J Orthod 2014; 19: 27–29.

36.

Fan

Song

Bachmann

. Justification and reporting of subgroup analyses were lacking or inadequate in randomized controlled trials. J Clin Epidemiol 2019; 108: 17–25.

37.

Smith

Ebrahim

. Data dredging, bias, or confounding. BMJ 2002; 325: 1437–1438.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.22 MB