Sage Journals: Discover world-class research

Abstract

Objective:

We aim to evaluate the quality of patient-reported outcomes included in randomized control trials for the treatment of congestive heart failure using the International Society for Quality of Life Research (ISOQOL) checklist, a validated tool for critically appraising the quality of patient-reported outcomes.

Design:

We performed a cross-sectional analysis of 65 randomized control trials with patient-reported outcomes for drug intervention trials for treating congestive heart failure.

Setting:

N/A.

Participants:

N/A.

Main outcome measures:

The primary outcome of this study was to evaluate the reporting completeness of patient-reported outcomes in congestive heart failure clinical trials with drug interventions according to the ISOQOL checklist.

Results:

Our search returned 1114 studies, of which, 65 are included in the analysis. The average completion of the ISOQOL reporting standards was 44.51%. Higher completion of the ISOQOL patient-reported outcome standards was observed in the clinical trials with patient-reported outcomes as primary endpoints compared to the clinical trials with patient-reported outcomes as a secondary endpoint. The multivariable regression model showed that clinical trials with patient-reported outcomes as a primary endpoint had a 21.46% better completion percentage (t = 4.45, p ⩽ 0.001) when controlling for PRO recording duration and trial registration. Eight (8/65, 12.31%) of the clinical trials met the satisfaction criteria of completing two-thirds of the ISOQOL patient-reported outcomes reporting standards. All of these RCTs had a patient-reported outcome as a primary endpoint.

Conclusion:

Our analysis of the reporting of patient-reported outcomes in congestive heart failure clinical trials with drug interventions suggests that the quality of reporting is suboptimal. This evidence of substandard reporting of patient-reported outcomes is disconcerting as it reduces the transparency of randomized control trials, which are considered the foundation of evidenced-based medicine. Inadequate reporting may result in clinicians implementing misrepresented or incomplete evidence into clinical practice. Validated reporting tools, such as the ISOQOL, can be used by trialists and clinicians alike to improve and critically appraise the reporting of patient-reported outcomes in randomized control trials.

Trial registration:

N/A.

Keywords

congestive heart failure ISOQOL Reporting Standards patient-reported outcomes randomized controlled trials

Introduction

The American Heart Association estimates that 6.2 million American adults have congestive heart failure (CHF). This estimate is projected to increase to over 8 million adults by 2030.^1–3 In 2014, 1.1 million emergency department visits and 1 million hospitalizations were attributed to primary CHF with a mean cost of $11,552 per admission and a total estimated cost of greater than $11 billion.⁴ Worldwide, it is estimated that more than 64 million people have CHF, affecting 1%–2% of all adults.^5,6 CHF may also increase the risk of complications such as arrhythmia, thromboembolism, hepatic dysfunction, muscle wasting, and pulmonary congestion.⁷ Given the high prevalence of CHF, accompanied by its escalating societal costs and associated comorbidities, research efforts are essential for investigating therapies that decrease morbidity and mortality. Moreover, because CHF directly affects the quality of life and daily function of patients, equal efforts should focus on quality-of-life improvement and other patient-reported outcomes (PROs). PROs are defined as measurements based on reports directly from patients about their health condition and treatment experiences, without interpretation by clinicians or others. PROs are crucial in assessing the impact of treatments from the patient’s perspective. In CHF trials, commonly used PRO measures include the Kansas City Cardiomyopathy Questionnaire (KCCQ), which assesses the patient’s health status, and the Minnesota Living with Heart Failure Questionnaire (MLHFQ), which measures the impact of heart failure on quality of life.^8,9 For investigations of efficacy and quality of life, results from randomized clinical trials (RCTs) are frequently used to explore the methods and strategies for potential improvements in these outcomes.

RCTs are widely regarded as the foundation of evidence-based medicine through their study design that reduces bias, decreases confounders, and allows for the exploration of causal relationships between interventions and outcomes.¹⁰ To illustrate their importance, RCTs are often used as strong supporting evidence for recommendations underpinning clinical practice guideline development. For example, a clinical practice guideline for the management of heart failure recommends the use of ACE inhibitors for patients with prior or current symptoms of chronic heart failure with a reduced ejection fraction based on the work of an RCT.¹¹ While RCTs are commonly used to investigate therapeutic efficacy, this study design is also effective for examining PROs. A recent systematic review of PROs found that few RCTs examined the quality of life parameters as primary outcomes¹²; however, when these outcomes are selected as primary endpoints, patients report greater satisfaction with study results.¹³ Thus, because of their widespread importance to patient care, RCTs must be clearly and completely reported, carefully outlining their methodology and decision-making.^14,15

Still, trialists will intermittently forgo the rigorous study design mandated by RCTs, giving way to variability and insufficient outcome reporting. This deficiency has led to the development of the Consolidated Standards of Reporting Trials (CONSORT) statement.¹⁶ The CONSORT statement created evidenced-based recommendations to be used by authors as a tool to ensure complete and appropriate reporting of trials. Several extensions of the CONSORT statement have been developed to improve different aspects of the original guidelines.^17,18 One of those extensions was developed by the International Society for Quality of Life Research (ISOQOL) to improve the reporting of PROs in RCTs.¹⁸ In addition to the ISOQOL guidelines, the CONSORT-PRO (Consolidated Standards of Reporting Trials–Patient-Reported Outcomes) extension provides minimum standards for reporting PROs in clinical trials. While CONSORT-PRO is widely recognized, this study used the ISOQOL checklist due to its comprehensive approach, offering a more detailed assessment of PRO reporting, particularly in the context of CHF trials. In this study, we focus on examining the reporting quality of RCTS that evaluate PROs for individuals with CHF using the ISOQOL screening guideline.

Methods

This methodology review (which used elements of systematic review methodology, when applicable) did not qualify as human research as outlined in the United States Code of Federal Regulations and did not require institutional review board oversight. The writing of this manuscript was directed by a modified reporting guideline of the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA). This guideline was developed by Murad and Wang¹⁹ for the reporting of epidemiological methodology research.

Study objectives

The primary objective of this study was to evaluate the completeness of PRO reporting in CHF studies with drug interventions according to the ISOQOL reporting guidelines. We selected the ISOQOL reporting guidelines as the primary framework for analysis due to its detailed and specific approach to evaluating PRO reporting. While the CONSORT-PRO guidelines establish minimum reporting standards, we believe ISOQOL offers a more comprehensive evaluation of reporting completeness, which is crucial for assessing the quality of PRO reporting in CHF clinical trials. Our secondary objectives were to determine factors that influence the completeness of PRO reporting and how primary or secondary endpoints affect the completeness of PRO reporting. We evaluated the change in completeness of PRO reporting prior to and after the publication of CONSORT-PRO in 2013 (allowing 1 year for guidelines to be implemented).

PubMed search

PubMed, which includes the MEDLINE collection, was searched by DR and JP. Our search string was created using a combination of medical subject heading (MESH) terms, key clinical phrases, and PRO-specific phrases used in a previous systematic review.²⁰ The following search string was deployed as follows: (“Heart Failure”[Mesh] AND “Heart failure” OR “Congestive Heart failure” OR “CHF” OR “Decompensated heart failure”) AND (“quality of life” OR “health related quality of life” OR “health status” OR “health outcomes” OR “psychosocial” OR “psychological” OR “depression” OR “anxiety” OR “emotional” OR “distress” OR “social functioning” OR “social wellbeing” OR “patient reported symptom” OR “patient reported outcomes” OR pain OR fatigue OR “PRO” OR “PROs” OR “HRQL” OR “QOL” OR “HRQOL” OR “symptom distress” OR “symptom burden” OR “symptom assessment” OR “functional status” OR coping OR socialization). To narrow this search to RCTs, we applied the filters for RCTs. We also limited our search to English-only publications and studies published on or after January 1st, 2010.

Eligibility

Eligible publications in this study included all types of RCTs including, but not limited to, cluster designs, crossover designs, and parallel arm studies with PROs. Other study designs lacking participant randomization including cohort, case–control, case reports, chart review, systematic reviews, meta-analyses, observational studies, animal studies, and letters to the editor were, thus, excluded. We also excluded trials with a sample size of less than 50 participants as these smaller trials are considered inadequately powered for PRO analyses (consistent with previous studies^20–23), or trials involving participants with other medical conditions. According to the 2017 CHF clinical practice guidelines, pharmacological interventions are a common treatment modality for CHF.¹¹ Thus, in this study, only RCTs focused on pharmacological interventions were included. All other intervention modalities were excluded.

Screening

The search string returns were exported into Rayyan,²⁴ a systematic review screening tool, for the first screening phase, including title and abstract screening. DR independently screened the titles and abstracts of all records twice using the aforementioned criteria. All excluded records were tagged with a reason for exclusion. Any difficult or questionable records were also reviewed by a second investigator (JP) and decisions about inclusion were made jointly through discussion. Following title and abstract screening, all included studies underwent a secondary full-text screening by both investigators (DR and JP) in a masked, independent fashion. Any differences found during the screening were discussed between DR and JP and a mutual decision was made by discussion. Two third-party researchers were available for consultation; however, this process was not needed. A list of included and excluded records can be found on Open Science Framework(https://osf.io/5zyp7/?view_only=a10612e0a0b34c53802e68dad4561d7f)

Data extraction

The investigators (DR and JP) who completed the screening also performed data extraction. Using the ISOQOL reporting guidelines as a reference,¹⁸ a Google form was developed that included all 28 checklist items, as well as the year of publication, country of first author, trial registration status, trial phase, PRO measure (quality of life, anxiety, depression, symptoms, functional status, etc.), length of PRO measure, statistical significance between treatment arms, conflict of interest, and funding statements. The inclusion of these characteristics was based on findings reported in previous studies.^20–23 Both researchers completed the form for each publication independently and masked. After completing data extraction, DR and JP reconciled any differences in the extracted data. Two other investigators (MV and CW) were available for consultation; however, it was not needed. The Google form used for data extraction is available at https://osf.io/5zyp7/?view_only=a10612e0a0b34c53802e68dad4561d7f.

Data analysis

For our primary objective, sufficient completion of the ISOQOL reporting standards was determined using two different approaches. As used by Mercieca-Bebber et al.,²⁰ our first approach considered adequate reporting as the completion of two-thirds of the ISOQOL reporting items. The second approach consisted of a tiered system to determine the level of completeness, which was based upon a previous study by Efficace et al.²⁵ The three tiers included mostly complete (⩾66% item completeness), partially complete (⩾33%, ⩽66% item completeness), and incomplete (⩽33% item completeness). For our secondary objective, we performed a multivariable regression analysis to determine the associations’ percent of PRO reporting completeness and RCTs length of PRO assessment, trial registration, whether PROs were primary or secondary endpoints, and whether the study was published prior to or after 2014.

Results

The initial search string returned 1114 studies. After a primary screening, 1021 studies were excluded. A secondary screening was performed resulting in the exclusion of 28 additional studies. Approximately 65 RCTs met the inclusion criteria and are included in the analysis. A summary of the exclusion process is shown in Figure 1.

Figure 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram for included studies.

RCT characteristics

RCT characteristics are fully reported in Table 1. A PRO was reported as a primary endpoint in 29 RCTs (44.62%) and reported as a secondary endpoint in 30 RCTs (46.15%). In six of the RCTs (9.23%), the PRO was not listed as a primary or a secondary endpoint. Quality of life was the PRO that was most commonly reported with 62 of the 65 RCTs (95.38%) reporting on it. Symptoms were the second most common PRO reported with 15 RCTs (23.08%) reporting on it. In these studies, 36 RCTs (55.38%) had a statistically significant difference in the PRO. The most common PRO that had a statistical difference between the arms was quality of life (33/65, 50.77%).

Table 1.

Baseline characteristics of randomized controlled trials.

Characteristic	PRO endpoint N (%)		Total N (%)
	Primary N (%)	Secondary N (%)
	29 (44.6)	36 (55.4)
Year of publication (N = 65)
2010	2 (6.9)	4 (11.1)	6 (9.2)
2011	2 (6.9)	3 (8.3)	5 (7.7)
2012	2 (6.9)	2 (5.6)	4 (6.2)
2013	6 (20.7)	4 (11.1)	10 (15.4)
2014	3 (10.3)	3 (8.3)	6 (9.2)
2015	4 (13.8)	2 (5.6)	6 (9.2)
2016	2 (6.9)	6 (16.7)	8 (12.3)
2017	4 (13.8)	4 (11.1)	8 (12.3)
2018	1 (3.4)	2 (5.6)	3 (4.6)
2019	1 (3.4)	5 (13.9)	6 (9.2)
2020	2 (6.9)	1 (2.8)	3 (4.6)
Country of first author (N = 65)
Country			#
United States	11 (37.9)	14 (11.3)	25 (38.5)
Italy	4 (13.8)	3 (2.4)	7 (10.8)
Germany	1 (3.4)	4 (3.2)	5 (7.7)
China	2 (6.9)	2 (1.6)	4 (6.2)
United Kingdom	1 (3.4)	2 (1.6)	3 (4.6)
Greece	2 (6.9)	1 (0.8)	3 (4.6)
Canada	2 (6.9)	1 (0.8)	3 (4.6)
Netherlands	0 (0)	2 (1.6)	2 (3.1)
Other	6 (20.7)	7 (5.6)	13 (20)
Trial registration (N = 65)
Yes	15 (51.7)	28 (12.2)	43 (66.2)
No	14 (48.3)	8 (3.5)	22 (33.8)
Registration site (N = 43)
Clinicaltrials.gov	11 (37.9)	20 (8.7)	31 (72.1)
ChiCTR	1 (3.4)	1 (0.4)	2 (4.7)
ISRCTN	3 (10.3)	5 (2.2)	8 (18.6)
CTRI	0 (0)	1 (0.4)	1 (2.3)
FONDECYT	0 (0)	1 (0.4)	1 (2.3)
Other	0 (0)	0 (0)	0 (0)
Trial phase (N = 43)
Phase I	0 (0)	0 (0)	0 (0)
Phase II	2 (6.9)	10 (4.3)	12 (27.9)
Phase II/III	0 (0)	2 (0.9)	2 (4.7)
Phase III	11 (37.9)	5 (2.2)	16 (37.2)
Phase IV	1 (3.4)	6 (2.6)	7 (16.3)
N/A	1 (3.4)	5 (2.2)	6 (14.0)
Length of PRO assessment (N = 65)
Up to 1 month	5 (17.2)	2 (5.6)	7 (10.8)
Up to 6 months	11 (37.9)	25 (69.4)	36 (55.4)
Up to a year	3 (10.3)	8 (22.2)	11 (16.9)
More than a year	9 (31)	1 (2.8)	10 (15.4)
Not reported	1 (3.4)	0 (0)	1 (1.5)
PROs studied (N = 65)
Quality of life	27 (93.1)	35 (97.2)	62 (95.4)
Anxiety	5 (17.2)	2 (5.6)	7 (10.8)
Symptoms	9 (31)	6 (16.7)	15 (23.1)
Depression	7 (24.1)	4 (11.1)	11 (16.9)
Pain	6 (20.7)	1 (2.8)	7 (10.8)
Psychosocial	1 (3.4)	2 (5.6)	3 (4.6)
Physical function	4 (13.8)	3 (8.3)	7 (10.8)
Statistical difference between treatment arms of PROs (N = 65)
Yes	24 (82.8)	12 (33.3)	36 (55.4)
No	5 (17.2)	24 (66.7)	29 (44.6)
Significant PROs (N = 36)
Quality of life	23 (79.3)	10 (8)	33 (50.8)
Anxiety	4 (13.8)	0 (0)	4 (6.2)
Symptoms	7 (24.1)	4 (3.2)	11 (16.9)
Depression	5 (17.2)	1 (0.8)	6 (9.2)
Pain	4 (13.8)	0 (0)	4 (6.2)
Psychosocial	1 (3.4)	1 (0.8)	2 (3.1)
Physical function	3 (10.3)	1 (0.8)	4 (6.2)
Funding statement (N = 65)
Public (e.g., government grant)	11 (37.9)	19 (8.3)	30 (46.2)
Private (e.g., foundation)	2 (6.9)	3 (1.3)	5 (7.7)
Industry	14 (48.3)	16 (7)	30 (46.2)
Hospital	0 (0)	2 (0.9)	2 (3.1)
Not mentioned	5 (17.2)	2 (0.9)	7 (10.8)
Other	0 (0)	0 (0)	0 (0)
Conflict of interest statement (N = 65)
Yes—the statement says that there is no conflict of interest	3 (10.3)	6 (2.6)	9 (13.8)
Yes—the statement says that there are one or more conflicts of interest	21 (72.4)	27 (11.7)	48 (73.8)
No—there is no conflict of interest statement	5 (17.2)	3 (1.3)	8 (12.3)

PRO, patient-reported outcome.

PRO measures

Within the 65 RCTs, there were 45 unique PRO measures used. The most common PRO measures used were the KCCQ with 35 RCTs (53.85%), the MLHFQ with 25 RCTs (38.46%), and the EQ-5D with 16 RCTs (24.62%). All of the RCTs reported which PRO measure they used.

Adherence to ISOQOL PRO reporting standards

Table 2 shows the adherence to ISOQOL PRO reporting standards. Of all 65 studies, the average completion of the ISOQOL reporting standards was 44.51% (SD = 17.85%). Items commonly reported include the PRO as an outcome in the abstract (61/65, 93.85%), the schedule for data collection (58/65, 89.23%), and the baseline characteristics (53/65, 81.54%). The items that were reported the least include discussing generalizability issues with the PRO measure (10/65, 15.38%) and discussing the limitations with the PRO measure (12/65, 18.46%).

Table 2.

ISOQOL guideline adherence.

ISOQOL subgroups	ISOQOL item	Adherence	PRO endpoint N (%)		Total N (%)
			Primary N (%)	Secondary N (%)
			29 (44.6)	36 (55.4)
Title/abstract
	The title of the paper states it is an RCT with a PRO	Yes	19 (65.5)	—	—
		No	10 (34.5)	—	—
	The PRO is identified as an outcome in the abstract	Yes	29 (100)	27 (90)	61 (93.8)
		No	0 (0)	3 (10)	4 (6.2)
Introduction/background/objectives
	The introduction contains a summary of PRO research that is relevant to the RCT	Yes	21 (72.4)	—	—
		No	8 (27.6)	—	—
	The PRO hypothesis is stated and is specific to the relevant PRO domain(s) if applicable	Yes	15 (51.7)	9 (30)	26 (40)
		No	14 (48.3)	21 (70)	39 (60)
	Additional details regarding the hypothesis are provided including the rationale for the selected domain(s), time points for assessment, and direction(s) of change	Yes	2 (6.9)	—	—
		No	27 (93.1)	—	—
Methods
Outcomes	The mode of data collection and PRO tool administration (telephone, email, etc.) is provided	Yes	9 (31)	9 (30)	20 (30.8)
		No	20 (69)	21 (70)	45 (69.2)
	A citation for the original development of the PRO instrument is provided	Yes	17 (58.6)	—	—
		No	12 (41.4)	—	—
	The rationale for the choice of the PRO instrument is provided	Yes	20 (69)	6 (20)	28 (43.1)
		No	9 (31)	24 (80)	37 (56.9)
	Evidence of PRO reliability and validity is provided or cited	Yes	20 (69)	7 (23.3)	28 (43.1)
		No	9 (31)	23 (76.7)	37 (56.9)
	A planned HRQL data collection schedule is provided	Yes	27 (93.1)	27 (90)	58 (89.2)
		No	2 (6.9)	3 (10)	7 (10.8)
	The PRO is identified in the trial protocol; post hoc analyses are reported	Yes	24 (82.8)	19 (63.3)	46 (70.8)
		No	1 (3.4)	5 (16.7)	8 (12.3)
		Protocol not found	4 (13.8)	6 (20)	11 (16.9)
	The status of the PRO as a primary or secondary outcome is reported	Yes	29 (100)	30 (100)	59 (90.77)
		No	0 (0)	0 (0)	6 (9.23)
	Windows for valid PRO responses are justified and specified as being appropriate for the clinical context	Yes	23 (79.3)	—	—
		No	6 (20.7)	—	—
Sample size	A power/sample size calculation relevant to the PRO and based on clinical rationale (expected effect size) is provided	Yes	7 (24.1)	—	—
		No	22 (75.9)	—	—
Statistical analysis	Evidence is provided for PRO-appropriate statistical analysis for each hypothesis tested	Yes	8 (27.6)	5 (16.7)	13 (20)
		No	21 (72.4)	25 (83.3)	52 (80)
	Statistical approaches for missing data and the extent of missing data are provided	Yes	10 (34.5)	3 (10)	13 (20)
		No	19 (65.5)	27 (90)	52 (80)
	The manner in which multiple comparisons have been addressed is reported	Yes	4 (13.8)	—	—
		No	25 (86.2)	—	—
Results
Participant flow	A flow diagram or description of participant allocation and those lost to follow-up is provided for PROs	Yes	10 (34.5)	7 (23.3)	17 (26.2)
		No	19 (65.5)	23 (76.7)	48 (73.8)
	The reasons for missing data are explained	Yes	10 (34.5)	6 (20)	16 (24.6)
		No	19 (65.5)	24 (80)	49 (75.4)
Baseline data	Patient characteristics are described including baseline PRO scores	Yes	27 (93.1)	24 (80)	53 (81.5)
		No	2 (6.9)	6 (20)	12 (18.5)
Estimation and outcomes	PRO data analysis accounts for survival differences between treatment arms if relevant	Yes	12 (41.4)	—	—
		No	8 (27.6)	—	—
		N/A	9 (31)	—	—
	All PRO domains and items are reported (not just statistically significant)	Yes	12 (41.4)	—	—
		No	17 (58.6)	—	—
	The proportion of patients achieving pre-defined responder definitions is provided when relevant	Yes	7 (24.1)	—	—
		No	1 (3.4)	—	—
Discussion
Limitations	The limitations of PRO components are discussed	Yes	11 (37.9)	1 (3.3)	12 (18.5)
		No	18 (18)	29 (96.7)	53 (81.5)
Generalizability	Generalizability issues uniquely related to the PRO results are discussed, if applicable	Yes	10 (34.5)	0 (0)	10 (15.4)
		No	19 (65.5)	30 (100)	55 (84.6)
Interpretation	The clinical significance of the PRO findings is discussed	Yes	19 (65.5)	8 (26.7)	28 (43.1)
		No	10 (34.5)	22 (73.3)	37 (56.9)
	The PRO results are discussed in the context of other clinical trial outcomes	Yes	21 (72.4)	7 (23.3)	29 (44.6)
		No	8 (27.6)	23 (76.7)	36 (55.4)
Other information
Protocol	A copy of the instrument is present if it has not been published previously	Yes	2 (6.9)	—	—
		PRO instrument previously published	27 (93.1)	—	—
		No	0 (0)	—	—

ISOQOL, International Society for Quality of Life Research; PRO, patient-reported outcome; RCT, randomized clinical trial.

All of the 29 RCTs that had a PRO as the primary endpoint provided the PRO measurement tool or the tool was previously published. In addition, 23 of these RCTs (79.31%) specified and justified the window for valid PRO responses, 21 RCTs (72.41%) included a summary of PRO research in the introduction, and 19 RCTs (65.52%) specified which PRO is measured in the title of the paper. Only two of these RCTs (6.90%) provided a hypothesis that specified directionality and timing and four RCTs (13.79%) discussed how they addressed multiple comparisons.

Higher completion of the ISOQOL PRO standards was observed in the RCTs with PROs as primary endpoints compared to the RCTs with PROs as a secondary endpoint. The RCTs with a PRO as a primary endpoint had an average completion rate of 55.54% (SD = 17.11%). This is compared to RCTs with a PRO as a secondary endpoint with an average completion rate of 38.24% (SD = 12.33%). The lowest completion rate was seen in the RCTs that did not specify if the PRO was a primary or secondary endpoint with a completion rate of 22.55% (SD = 5.78). The item with the largest difference between primary and secondary endpoints was providing evidence for the validity and reliability of the PRO measure with 20 RCTs (68.97%) with primary endpoints and 7 RCTs (23.33%) with secondary endpoints reporting it.

Eight (8/65, 12.31%) of the RCTs met the satisfaction criteria of completing two-thirds of the ISOQOL PRO reporting standards. All of these RCTs had a PRO as a primary endpoint. These studies had an average completion rate of 76.79%. Using the tiered completeness approach, 8 (8/65, 12.31%) were mostly complete, 39 (39/65, 60.00%) were partially complete, and 18 (18/65, 27.69%) were incomplete.

Influence on adherence

The multivariable regression model showed that the percentage of completeness of ISOQOL reporting items was not significantly different between RCTs PRO duration (−3.17%, t = −0.68, p = 0.502) or registration status (7.29%, t = 1.63, p = 0.111). However, reporting completeness was significantly different between studies with primary or secondary endpoints—RCTs with PROs as the primary endpoint showed a 21.46% better completion percentage (t = 4.45, p ⩽ 0.001), when controlling for PRO recording duration and trial registration.

Discussion

Our analysis of the reporting of PROs in CHF RCTs with drug interventions suggests that the quality of reporting is suboptimal. This evidence of substandard reporting of PROs is disconcerting as it reduces the transparency of RCTs, which are considered the foundation of evidence-based medicine.²⁶ Inadequate reporting may result in clinicians implementing misrepresented or incomplete evidence into clinical practice. Poor reporting practices result in the loss of valuable data, diminish the utility of research findings, and ultimately impede the translation of evidence into clinical practice. This notion is supported by Chalmers and Glasziou, who identified avoidable waste in the production and reporting of research evidence, emphasizing that incomplete or unusable reports represent a significant source of inefficiency in biomedical research. Similarly, Glasziou et al. have called for improved reporting standards to reduce waste and enhance the reliability of published research findings. Our study reinforces the need for strict adherence to validated reporting guidelines, such as ISOQOL and CONSORT-PRO, to ensure that PRO data are fully and accurately reported, thus maximizing their value for clinical decision-making and policy development.^27,28 Our primary finding—that PRO reporting in CHF RCTs with drug interventions showed suboptimal reporting—is supported by three similar studies across different fields of medicine that similarly used the two-third item completion threshold.^20–22 These studies concluded that overall reporting of PRO findings is insufficient, noting that high-quality reporting is crucial for the appropriate interpretation and appraisal of PRO results for use in clinical practice. Therefore, improvement in PRO reporting among RCTs is warranted given their importance in clinical decision-making and healthcare policy development.

In this investigation, we found that publications with PROs as the primary endpoint were associated with a statistically significant increase in adherence to the ISOQOL reporting guidelines. Of note, secondary endpoint PROs did not mirror the statistically significant adherence to ISOQOL reporting seen with PROs as primary endpoints. While our study focused on the ISOQOL guidelines for a detailed evaluation of PRO reporting in CHF trials, we acknowledge the important role of CONSORT-PRO as a widely recognized standard for PRO reporting. Future research could benefit from a comparative analysis using both ISOQOL and CONSORT-PRO guidelines to explore the impact of these reporting standards on the quality and completeness of PRO data. This finding is considerable, as better reporting is associated with transparent and reproducible research practices, decreased bias, and improved generalizability of results.²⁹ Omission of PROs from clinical trials is regrettably a common occurrence.^30–33 The absence of these outcomes decreases patient engagement in healthcare decision-making leading to worsening medical conditions secondary to lack of comfort with available knowledge about a drug treatment.³⁴ As the adoption of more patient engagement metrics is used by healthcare organizations and governments, so too should a rise be seen in RCTs reporting primary endpoint PROs, strictly adhering to ISOQOL and other PRO reporting guidelines.

Given that RCTs have begun to increasingly use PROs along with safety and efficacy data for drug approval, an expected increase would be seen in the quality of the PROs as more analyses are performed. However, the results of studies examining the quality of PROs over the years are inconclusive. The findings of our study of PROs in RCTs for CHF noted that there were no statistically significant differences in reporting quality over time, mirroring the findings of Mercieca-Bebber et al.²⁰ Conversely, Rees et al.²³ report that certain criteria (the clinical significance of PRO findings and information about instrument administration) have increased over time. Both studies indicate that the overall quality of reporting of PROs remains inadequate. Both the Federal Drug Administration and the European Medicine Agencies have issued guidance to trialists on the implementation and interpretation of PROs used to endorse drug approval and appropriate pharmaceutical labeling.^35,36 Yet, neither of these organizations mentions the use of validated reporting quality guidelines, such as the ISOQOL. We recommend that as PROs become more commonplace, a greater emphasis should be placed on these PROs adhering to externally verifiable reporting guidelines, thus ensuring the reliability of these outcomes on patients and the public.

Lingering concerns for wider adoption of PROs is clinician understanding of the results and integration into clinical decision-making. While PRO measures have been shown to increase clinician time spent discussing these outcomes with patients,³⁷ studies have failed to show an influence on treatment selection.³⁸ Reasons for this finding include unfamiliarity with PRO research design, skepticism about the generalizability of the results, and lack of clinician time and resources. These questions have led to numerous published articles meant to aid clinicians in the interpretation of PRO results. Still, this places the burden upon clinicians to familiarize themselves with differing methods of deconstructing PROs to make sense of true clinical benefit, which complicates matters rather than clarifying them. For common diseases, such as CHF, sifting through the multitude of PROs included within RCTs is not a feasible task for the busy clinician. A solution to this quandary is the universal implementation of verified reporting quality guidelines, like the ISOQOL, when submitting a manuscript or drug approval. This mandate would encourage trialists to observe the items underpinning a high-quality PRO, as well as provide a score based on the number of items met that is easily discernible to clinicians.

Strengths and limitations

Strengths of our study include the data extraction method, in which all studies were coded independently and blinded by two researchers, increasing the reproducibility of our results. We selected the ISOQOL guideline as this is an internationally recognized measure of PROs that has been the basis for the development of other PRO reporting standards such as the CONSORT-PRO extension.^18,20,39 In addition, we elected to include multiple PRO types rather than focusing only on health-related quality of life. Limitations of our study include the exclusion of non-English RCTs which may fail to account for all RCTs evaluating drug interventions for CHF, but the effect of their absence has been suggested to be non-significant.⁴⁰ In addition, the ISOQOL-PRO reporting standards may be a source of subjectivity in the interpretation of the reporting items, such as the reporting of PRO statistical analysis and missing data, which requires some interpretation to determine the quality of reporting. Lastly, we only used PubMed as the source for identifying relevant records. While PubMed is a comprehensive and widely recognized database for biomedical literature, this approach may have excluded relevant studies indexed in other databases such as Embase, Scopus, or the Cochrane Library. This could potentially limit the breadth and diversity of our included studies, leading to a narrower scope of findings.

Conclusion

Overall, PRO reporting in RCTs of pharmaceutical interventions for CHF remains suboptimal. The underreporting of PROs may lead to significant errors in interpreting data from these studies, which ultimately will impact patient care. It is vital that the results of PROs in RCTs are reported at the highest quality for not only patient safety but also for the practice of evidence-based medicine. This problem may be solved by a collective effort from researchers, clinicians, and journal editors to adhere to internally recognized standards for reporting PROs such as the ISOQOL guidelines. Together, the use of PRO quality guidelines optimizes research and positively facilitates the implementation of RCT PROs into clinical practice.

Footnotes

Acknowledgements

None.

Declarations

ORCID iD

Christian Hemmerich

References

Heidenreich

Albert

Allen

, et al. Forecasting the impact of heart failure in the United States: a policy statement from the American Heart Association. Circ Heart Fail 2013; 6(3): 606–619.

Yancy

Jessup

Bozkurt

, et al. 2013 ACCF/AHA guideline for the management of heart failure: executive summary: a report of the American College of Cardiology Foundation/American Heart Association Task Force on practice guidelines. Circulation 2013; 128(16): 1810–1852.

Virani

Alonso

Benjamin

, et al. Heart disease and stroke statistics—2020 update: a report from the American Heart Association. Circulation 2020; 141: e139–e596.

Jackson

Tong

King

, et al. National burden of heart failure events in the United States, 2006 to 2014. Circ Heart Fail 2018; 11(12): e004873.

Groenewegen

Rutten

Mosterd

, et al. Epidemiology of heart failure. Eur J Heart Fail 2020; 22(8): 1342–1356.

GBD 2017 Disease and Injury Incidence and Prevalence Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018; 392(10159): 1789–1858.

Watson

Gibbs

Lip

. ABC of heart failure. Clinical features and complications. BMJ 2000; 320(7229): 236–239.

Green

Porter

Bresnahan

, et al. Development and evaluation of the Kansas City Cardiomyopathy Questionnaire: a new health status measure for heart failure. J Am Coll Cardiol 2000; 35(5): 1245–1255.

Spertus

Jones

Sandhu

, et al. Interpreting the Kansas City Cardiomyopathy Questionnaire in clinical trials and clinical care: JACC state-of-the-art review. J Am Coll Cardiol 2020; 76(20): 2379–2390.

10.

Hariton

Locascio

. Randomised controlled trials—the gold standard for effectiveness research. BJOG 2018; 125(13): 1716.

11.

Yancy

Jessup

Bozkurt

, et al. 2017 ACC/AHA/HFSA focused update of the 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology/American Heart Association task force on clinical practice guidelines and the Heart Failure Society of America. J Card Fail 2017; 23(8): 628–651.

12.

Gaudry

Messika

Ricard

, et al. Patient-important outcomes in randomized controlled trials in critically ill patients: a systematic review. Ann Intensive Care 2017; 7(1): 28.

13.

Murad

Shah

Van Houten

, et al. Individuals with diabetes preferred that future trials use patient-important outcomes and provide pragmatic inferences. J Clin Epidemiol 2011; 64(7): 743–748.

14.

Chen

Zhang

, et al. Quality of reporting in randomized controlled trials of therapeutic cardiovascular medical devices. Surgery 2019; 165(5): 965–969.

15.

Kabisch

Ruckes

Seibert-Grafe

, et al. Randomized controlled trials: part 17 of a series on evaluation of scientific publications. Dtsch Arztebl Int 2011; 108(39): 663–668.

16.

Schulz

Altman

Moher

; CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Int J Surg 2011; 9(8): 672–677.

17.

Schulz

Altman

Moher

; CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. Obstet Gynecol 2010; 115(5): 1063–1070.

18.

Brundage

Blazeby

Revicki

, et al. Patient-reported outcomes in randomized clinical trials: development of ISOQOL reporting standards. Qual Life Res 2013; 22(6): 1161–1175.

19.

Murad

Wang

. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med 2017; 22(4): 139–142.

20.

Mercieca-Bebber

Perreca

King

, et al. Patient-reported outcomes in head and neck and thyroid cancer randomised controlled trials: a systematic review of completeness of reporting and impact on interpretation. Eur J Cancer 2016; 56: 144–161.

21.

Dirven

Taphoorn

MJB

Reijneveld

, et al. The level of patient-reported outcome reporting in randomised controlled trials of brain tumour patients: a systematic review. Eur J Cancer 2014; 50(14): 2432–2448.

22.

Efficace

Feuerstein

Fayers

, et al. Patient-reported outcomes in randomised controlled trials of prostate cancer: methodological quality and impact on clinical decision making. Eur Urol 2014; 66(3): 416–427.

23.

Rees

Whale

Fish

, et al. Patient-reported outcomes in randomised controlled trials of colorectal cancer: an analysis determining the availability of robust data to inform clinical decision-making. J Cancer Res Clin Oncol 2015; 141(12): 2181–2192.

24.

Ouzzani

Hammady

Fedorowicz

, et al. Rayyan—a web and mobile app for systematic reviews. Syst Rev 2016; 5(1): 210.

25.

Efficace

Osoba

Gotay

, et al. Has the quality of health-related quality of life reporting in cancer clinical trials improved over time? Towards bridging the gap with clinical decision making. Ann Oncol 2007; 18(4): 775–781.

26.

Diamond

. Randomized trials, observational registries, and the foundations of evidence-based medicine. Am J Cardiol 2014; 113(8): 1436–1441.

27.

Glasziou

Altman

Bossuyt

, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014; 383(9913): 267–276.

28.

Chalmers

Glasziou

. Avoidable waste in the production and reporting of research evidence. Lancet 2009; 374(9683): 86–89.

29.

Butcher

Monsour

Mew

, et al. Improving outcome reporting in clinical trial reports and protocols: study protocol for the Instrument for reporting Planned Endpoints in Clinical Trials (InsPECT). Trials 2019; 20(1): 161.

30.

Kyte

Retzer

Ahmed

, et al. Systematic evaluation of patient-reported outcome protocol content and reporting in cancer trials. J Natl Cancer Inst 2019; 111(11): 1170–1178.

31.

Efficace

Jacobs

Pusic

, et al. Patient-reported outcomes in randomised controlled trials of gynaecological cancers: investigating methodological quality and impact on clinical decision-making. Eur J Cancer 2014; 50(11): 1925–1941.

32.

Brundage

Bass

Davidson

, et al. Patterns of reporting health-related quality of life outcomes in randomized clinical trials: implications for clinicians and quality of life researchers. Qual Life Res 2011; 20(5): 653–664.

33.

Vodicka

Kim

Devine

, et al. Inclusion of patient-reported outcome measures in registered clinical trials: evidence from Clinicaltrials.Gov (2007–2013). Value Health 2015; 18(3): A209–A210.

34.

Bombard

Baker

Orlando

, et al. Engaging patients to improve quality of care: a systematic review. Implement Sci 2018; 13(1): 98.

35.

European Medicines Agency. Reflection paper on regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products. EMA 2005: 1–11.

36.

U.S. Department of Health and Human Services FDA Center for Drug Evaluation and Research; U.S. Department of Health and Human Services FDA Center for Biologics Evaluation and Research and U.S. Department of Health and Human Services FDA Center for Devices and Radiological Health. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance. Health Qual Life Outcomes 2006; 4: 79.

37.

Detmar

Muller

Schornagel

, et al. Health-related quality-of-life assessments and patient-physician communication: a randomized controlled trial. JAMA 2002; 288(23): 3027–3034.

38.

Greenhalgh

Long

Flynn

. The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc Sci Med 2005; 60(4): 833–843.

39.

Calvert

Blazeby

Altman

, et al. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA 2013; 309(8): 814–822.

40.

Morrison

Polisena

Husereau

, et al. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int J Technol Assess Health Care 2012; 28(2): 138–144.

Assessing the completeness of patient-reported outcomes reporting in congestive heart failure clinical trials

Abstract

Objective:

Design:

Setting:

Participants:

Main outcome measures:

Results:

Conclusion:

Trial registration:

Keywords

Introduction

Methods

Study objectives

PubMed search

Eligibility

Screening

Data extraction

Data analysis

Results

RCT characteristics

PRO measures

Adherence to ISOQOL PRO reporting standards

Influence on adherence

Discussion

Strengths and limitations

Conclusion

Footnotes

Acknowledgements

Declarations

ORCID iD

References