Abstract
Patient-reported outcomes are important measures to include in pulmonary arterial hypertension clinical trials but are not widely utilized in clinical practice. Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire (PAH-SYMPACT) is the only pulmonary arterial hypertension-specific patient-reported outcomes instrument developed and validated in accordance with the US Food and Drug Administration guidance on patient-reported outcomes development. The PAH-SYMPACT tool measures pulmonary arterial hypertension-related symptoms and impact of pulmonary arterial hypertension on daily life. Symptoms are reported each day for seven consecutive days, and the impact of pulmonary arterial hypertension over one week is recalled and reported on day 7; however, daily symptom reporting may overburden patients and healthcare resources, limiting the practicality of PAH-SYMPACT outside of clinical trials. To determine the practicability of an abridged version of PAH-SYMPACT for which all reporting is completed on one day, symptom data from the SYMPHONY trial (NCT01841762; PAH-SYMPACT validation study) were retrospectively analyzed to assess whether symptoms reported on each day correlated with the weekly average and whether one-day symptom scores were sensitive to disease severity. Correlation coefficients comparing the weekly average and individual day symptom scores were mostly high or very high regardless of the day they were measured. Findings were similar when using either Spearman's rank correlation or weighted kappa method. One-day symptom scores differentiated well between World Health Organization functional classes II and III/IV pulmonary arterial hypertension and were sensitive to change in disease severity as measured by the Patient Global Assessment of Disease Severity. These data suggest that the one-day PAH-SYMPACT is feasible and appropriate for routine implementation in clinical practice.
Introduction
Pulmonary arterial hypertension (PAH) is a debilitating condition that can severely impact patients' abilities to carry out daily activities, with symptoms becoming more severe—and even fatal—as the disease progresses.1,2 A key goal of PAH therapy is to improve symptoms and thereby patient quality of life. 3 The utilization of validated PAH-specific instruments for clinical use is increasingly considered essential to evaluate the effect of PAH and treatments on symptoms and quality of life and for the holistic management of patients with PAH.4–7 There is evidence to suggest that quality of life may be associated with outcomes in patients with PAH 8 and that PAH treatment can improve health-related quality of life. 9 PAH-specific patient-reported outcomes (PROs) instruments, including Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire (PAH-SYMPACT) 10 and emPHasis-10, 11 have been developed and are being validated to provide a comprehensive evaluation of the health status and symptoms of patients with PAH.
PROs are recommended as end points in clinical trials, providing subjective information on the impact of disease symptoms and therapeutic interventions on patient lives that are not captured by objective clinical end points.12–16 Instruments that measure PROs are used to inform treatment decisions in many disease settings, such as urologic diseases 17 and left-sided heart failure16,18; however, PROs are not used routinely in the PAH patient population. PAH-specific PRO instruments have been developed to provide a comprehensive evaluation of the health status and symptoms of patients with PAH that cannot be captured by generic quality-of-life instruments.10,11,19,20
PAH-SYMPACT is a new PAH-specific instrument developed and validated in accordance with guidance on PRO development from the US Food and Drug Administration (FDA). 10 In its original form, the 11 symptom items are reported on seven consecutive days and averaged, while the 11 impact items are reported on day 7 with a one-week recall period. 10 Although daily reporting of symptoms may be both beneficial and practical in the clinical trial setting, it may be impractical to incorporate into routine clinical practice. The objective of this analysis was to determine the practicability of a one-day version of the PAH-SYMPACT that enables patients to report their symptoms on one day instead of daily for seven days. We conducted a retrospective analysis of the PAH-SYMPACT symptom data from the SYMPHONY trial to assess (1) whether symptom scores reported on a single day correlated with the weekly average of symptom scores collected daily over seven days and (2) the sensitivity of one-day symptom scores to differentiate between patients with different disease severity and their ability to detect improvement or worsening in the severity of disease.
Methods
This is a retrospective analysis of the previously described SYMPHONY trial, a prospective, open-label, single-arm, multicenter, phase 3b study in patients with PAH who received the endothelin receptor antagonist macitentan 10 mg once daily (NCT01841762).
21
Eligible patients were 18–80 years of age with PAH as classified by the fourth World Symposium on Pulmonary Hypertension (idiopathic; heritable; drug- or toxin-induced; or associated with connective tissue disease, congenital heart disease with simple systemic-to-pulmonary shunt at least one year after surgical repair, or HIV infection),
22
had World Health Organization (WHO) functional class (FC) II to IV PAH, a six-minute walk distance of 150 m or more, and could be receiving stable doses of inhaled prostacyclin analogs or phosphodiesterase type 5 inhibitors. SYMPHONY comprised a 20-week study period: two weeks for screening, a two-week baseline period, and a 16-week treatment period. An electronic version of the PAH-SYMPACT instrument (ePRO)
10
was completed by screened patients twice during the baseline period (week –2 to week –1 and from week –1 to week 0) and twice in the treatment period at weeks 8 and 16 using a tablet (ePRO 1–4; Fig. 1). The draft PAH-SYMPACT completed by patients in SYMPHONY included a yes/no question about oxygen use assessed daily, 16 symptom items with a 24-h recall period assessed daily, and 25 impact items with a seven-day recall period assessed at the end of the week. Item reduction, content validation, and psychometric validation were performed to produce a final validated PAH-SYMPACT that comprises 11 impact questions (reported on day 7 with a one-week recall period) and 11 symptom questions (reported daily for seven days with a 24-h recall period) (Figs 1 and 2). The question regarding oxygen use was recorded but not used as part of the domain score.
21
Each PAH-SYMPACT symptom item was scored using a five-point Likert scale, with a score of 0 for no effect, 1 for mild, 2 for moderate, 3 for severe, and 4 for very severe.
PAH-SYMPACT symptom assessment in SYMPHONY (adapted from). PAH-SYMPACT domains.

SYMPHONY was conducted in accordance with the amended Declaration of Helsinki. Local institutional review boards or independent ethics committees at each participating institution approved the protocol, and all patients provided written informed consent.
Retrospective analysis
Using data for the 11 symptom questions over the four study periods (ePRO 1–4), the symptom domain scores collected each day (days 1–7) were compared with the weekly average using descriptive statistics. Any correlation between the weekly average and the one-day scores was assessed using Spearman's rank correlation (which allows determination of the correlation between two variables/assessments that have a non-normal distribution 23 ), using the mean score for each symptom domain per day. A correlation coefficient of 0.9 to 1.0 (−0.9 to −1.0) indicates a very high positive (negative) correlation, 0.7 to 0.9 (−0.7 to −0.9) a high positive (negative) correlation, 0.5 to 0.7 (−0.5 to −0.7) a moderate positive (negative) correlation, 0.3 to 0.5 (−0.3 to −0.5) a low positive (negative) correlation, and 0.0 to 0.3 (0.0 to −0.3) a negligible correlation. 23 The sensitivity of the Spearman's rank correlation was assessed using the weighted kappa method. 24 The correlation of scores between two days of the weekly average collection was also evaluated using Spearman's rank correlation.
The sensitivity of one-day scores to changes in disease severity was assessed using a t-test to compare scores for patients with FC II vs FC III/IV symptoms. Patient Global Assessment of Disease Severity (PGA-S, with response none = 1, very mild = 2, mild = 3, moderate = 4, severe = 5, and very severe = 6) was used to assess change in severity of disease (based on changes from baseline to week 8 and week 16); the sensitivity of one-day scores to this change was evaluated using an analysis of variance (ANOVA) model. PGA-S groups were included as the independent variable, and PAH-SYMPACT domain scores were included as the dependent variable. Pairwise comparisons between least squares means were performed using the Scheffe test and adjusted for multiple comparisons.
Results
As reported previously, 284 patients were enrolled in SYMPHONY between April 2013 and October 2015 at 71 centers in the United States. 21 Final analysis included 278 patients; six were excluded due to protocol deviations. Most patients had idiopathic (48.9%) or connective tissue disease-associated (39.9%) PAH. The majority of patients were female (78.8%), the median age was 59.7 years, most had WHO FC II (39.6%) or FC III (59.0%) symptoms, and 46.4% were receiving background PAH therapy. 21
Correlation between the weekly average and one-day symptom scores
Spearman correlation coefficient ranges by symptom domains on days 1 through 7 for each period.
ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire.
Spearman correlation coefficients were also calculated between days of the week for each symptom during each period. In general, a high correlation was observed between any of the days within the seven-day period, with higher correlations between days that were closer together (data not shown). For example, for stomach area swelling during period 1, the correlation coefficients between days 1 and 2 (0.78) and days 6 and 7 (0.81) were higher than the correlation coefficient between days 1 and 7 (0.67).
Mean score in symptom domains. Spearman's correlation coefficients between weekly average and day 7.
ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire; SD: standard deviation.
Correlation between day 7 and weekly average, by symptom, using Spearman's correlation coefficients.
ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire.
Correlation between day 7 and weekly average, by symptom, using the weighted kappa method. a
ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire.
Source: reproduced with permission from Landis and Koch, 1977. 24
Kappa value of 0.61–0.80 indicates substantial correlation and 0.81–1.00 indicates almost perfect correlation.
In order to calculate the weighted kappa value, weekly average and daily scores that were in the same range were included.
Sensitivity of one-day symptom scores to severity of disease and ability to detect change in disease severity
One-day symptom scores differentiated well between WHO FC II and FC III/IV PAH, particularly for cardiopulmonary domain symptoms (Table 4).
Mean score in symptom domains on day 7 by ePRO period (WHO FC II versus WHO FC III/IV). a
ePRO: electronic version of the Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire; SD: standard deviation; WHO FC: World Health Organization functional class.
WHO FC and symptoms are collected at baseline and during the same period.
Discussion
In this study, correlation coefficients comparing the weekly average and individual day symptom scores were mostly high (i.e. 0.70–0.90) or very high (i.e. > 0.90). One-day symptom scores also differentiated well between WHO FC II and III/IV PAH, and were sensitive to change in disease severity, as measured by the PGA-S. These results suggest that the PRO SYMPACT tool, when taken on a single day, may be a clinically useful and simpler alternative compared with completion on a daily basis for seven days.
There is widespread and increasing focus on the use of PROs in routine clinical practice to guide, individualize, and improve patient management in diverse disease settings.25–27 In addition, there is a move away from generic PRO tools to those that are disease-specific to capture the most relevant information for the patient population. 28 However, there are challenges to implementation of PRO instruments in clinical practice, which include the need to devise user-friendly PRO tools that can be introduced seamlessly into daily workflow without presenting an additional burden to routine patient management; training healthcare teams on the use of PRO tools and the interpretation of PRO data; and overcoming potential skepticism on the value or benefit that incorporation of PROs may achieve in clinical practice.26,29–31
Similarities and differences between the 1-day and 7-day PAH-SYMPACT versions.
PAH: pulmonary arterial hypertension; PAH-SYMPACT: Pulmonary Arterial Hypertension—Symptoms and Impact Questionnaire.
Study results show that this could be any day (1 through 7); however, using day 7 means the symptom and impact data will all be collected on the same day.
Source: reproduced with permission from Chin et al., 2018. 21
The validated PAH-SYMPACT tool can differentiate between patients according to disease severity (measured by WHO FC II vs FC III/IV and by PGA-S), and evaluation of the one-day version demonstrated that one-day symptom scores were also able to differentiate between these patient subgroups, particularly within cardiopulmonary domain symptoms.
PRO instruments require robust validation, and the original PAH-SYMPACT tool was developed according to the guidance of the FDA and validated in the phase 3b SYMPHONY study (NCT01841762). While this study demonstrates strong correlation between the one-day version of the instrument and the original PAH-SYMPACT tool, the former has not been formally and independently validated. However, since it is based on the validated PAH-SYMPACT instrument and shows good correlation with that instrument, the one-day PAH-SYMPACT may be more practical for use in clinical practice. Taken together, PAH-SYMPACT and the modified one-day version provide PAH-specific instruments applicable to the research and clinical practice environments, respectively.
Implementation of the one-day version of PAH-SYMPACT in clinical practice, including serial use following a change in therapy, holds the promise of providing quantifiable information about patient quality of life and the impact of changing therapy. A pilot project of administering the one-day PAH-SYMPACT tool when patients check in for their appointments at the Mayo Pulmonary Hypertension Clinic was recently initiated. The goal of this project is to assess in clinical practice how the instrument performs relative to other measures of disease severity, the impact of change in therapy, and relationship to outcome. In addition, flagging of particularly problematic symptoms and impacts may facilitate directed intervention. The initial reaction of patients has been positive, including ease of completing the instrument and sense of gratitude that quality of life is being taken seriously.
In conclusion, we have confirmed the feasibility of a one-day version of the PAH-SYMPACT instrument for reporting and monitoring patient outcomes. We encourage further evaluation of the one-day PAH-SYMPACT in both clinical and research settings to confirm the suitability and usefulness of the modified instrument in both scenarios.
Footnotes
Acknowledgments
Medical writing support was provided by Twist Medical, LLC.
Conflict of interest
R.P.F. reports steering committee and advisory board relationships with Actelion and an advisory board relationship with United Therapeutics. K.M.C. has served as a steering committee member for Actelion Pharmaceuticals Ltd; has received research grants from Actelion Pharmaceuticals Ltd, National Institutes of Health, Ironwood Pharmaceuticals, National Institutes of Health, and SoniVie; has served on an advisory board for Bayer Healthcare (through UCSD) and Flowonix; has served as an adjudication committee member for Arena Pharmaceuticals; is Circulation Associate Editor for the American Heart Association; and has received consultancy fees from Actelion Pharmaceuticals Ltd. C.Z. and M.F. are employees of Actelion Pharmaceuticals US, Inc. D.B. has received research support from Actelion Pharmaceuticals, Arena, Bellerophon, Gilead, Lung Rx, Reata, and United Therapeutics. He holds stock in Johnson and Johnson and has served as a steering committee member or advisory board member for Acceleron, Actelion Pharmaceuticals, Arena, Bellerophon, and Gilead.
Contributorship
Authors developed the first draft of the manuscript with assistance from a professional medical writer. All authors had access to the data; contributed equally to data interpretation, manuscript development, and critical analysis of the manuscript; and provided final approval for manuscript submission.
Funding
This SYMPHONY study was sponsored by Actelion Pharmaceuticals Ltd, Allschwil, Switzerland. The current analysis was sponsored by Actelion Pharmaceuticals Ltd, South San Francisco, CA.
Guarantor
Robert P. Frantz
