Validation of the Crohn’s and Ulcerative Colitis questionnaire in patients with acute severe ulcerative colitis

Abstract

Background

The Crohn’s and Ulcerative Colitis questionnaire (CUCQ) has previously been validated in patients with mild to moderate Crohn’s and ulcerative colitis (UC). The aim of this study was to validate the tool in patients with acute severe UC.

Methods

We undertook a validation of the CUCQ in patients recruited to the COmparison of iNfliximab and ciclosporin in STeroid Resistant Ulcerative Colitis Trial (CONSTRUCT). We carried out psychometric analysis to examine the underlying dimensions of the scale, internal consistency and construct validity. We carried out stepwise regression to examine which items accounted for the greatest variance in the scale.

Results

We obtained complete data for 270 patients. The internal consistency of the CUCQ was excellent (Cronbach’s alpha > 0.8). The CUCQ scores achieved significant correlations with two generic quality of life scales (SF-12 and EQ-5D), demonstrating good construct validity. Stepwise regression identified 16 items that accounted for greater than 95% of the variance of the CUCQ. Only three of the eight items selected for a short form in mild to moderate patients were selected for patients with acute severe UC.

Conclusions

The CUCQ demonstrated good validity in our sample of acute severe UC patients. Stepwise regression identified potential to shorten the tool, but that different items would be selected compared with less severe patients. If the tool is to be applied across the spectrum of disease it would be more appropriate to use the full 32 items in the scale. Further work to explore test-retest is required in acute patients.

Introduction

Ulcerative colitis (UC) is a chronic incurable relapsing inflammatory disorder which presents with a multitude of symptoms impairing patient quality of life (QoL).^1,2 The symptoms associated with UC are unpredictable and there can be significant variation in symptoms both over time within the same patient, and with different patients.

Measuring QoL in UC patients is important in order to assess changes in the patients’ condition over time and following treatment, and also to gain an insight into patient perception of their condition and how this compares with clinical or objective outcomes.

The aim of the COmparison of iNfliximab and ciclosporin in STeroid Resistant Ulcerative Colitis Trial (CONSTRUCT) was to compare the clinical and cost effectiveness of ciclosporin and infliximab in treating steroid-resistant acute severe UC.^3,4 Recent findings from other investigators illustrated no difference between cicolpsorin and infliximab in their primary outcome of treatment failures, but they did not report QoL of patients following treatment. ⁵ As the National Institute for Health and Care Excellence (NICE) have described that the ultimate criteria for interventions in healthcare are effectiveness and cost effectiveness in improving the survival and QoL of patients over extended periods, in undertaking the CONSTRUCT trial we chose a patient-reported outcome measure as the basis for our primary outcome measure.

Preliminary validation of the Crohn’s and Ulcerative Colitis questionnaire (CUCQ) in patients with mild to moderate disease confirmed that it met essential psychometric criteria. ⁶ This paper reports on the validation of the CUCQ within the context of the CONSTRUCT trial in patients with acute severe UC.^3,4

Methods

We used the standard psychometric approaches for validation of the CUCQ as outlined by Streiner et al. ⁷ For this validation we undertook: principal components analysis to examine the underlying dimensions of the scale; calculation of Cronbach’s alpha to test the internal consistency of the scale; correlations with two generic QoL scales (SF-12 and EQ-5D) to evaluate the construct validity of the scale; and stepwise regression to explore which items contributed most to the scale. Details of the item-generation process, piloting and initial validation of the CUCQ in a sample of mild to moderate patients have previously been reported. ⁶

Details of CUCQ scoring

We calculated scores for the CUCQ as follows.

We scored questions with four responses as 0, 1, 2 or 3 in ascending severity.

We scored questions with responses between 0 and 14 days as the actual value.

We reversed the scoring of questions with wording in the reverse direction (Q7, Q22 and Q32) to code all questions in the same direction.

We rescaled questions between 0 and 1 by dividing actual responses by their maximum score (3 or 14).

We calculated total CUCQ scores by summing all valid responses and dividing by the number of completed questions.

The lower the CUCQ the better the respondent’s health.

We calculated the CUCQ scores only when the patient had responded to at least 75% of the questions, i.e. 24/32. If participants had completed fewer than 75% of the questions, we treated the total CUCQ score as missing. We gave equal weight to each question of the 32 questions.

Validation of the CUCQ in the CONSTRUCT randomised controlled trial (RCT)

We undertook validation of the CUCQ on the CONSTRUCT^3,4 trial sample. We conducted psychometric analysis of the CUCQ in the following way.

We examined the 32 sets of response frequencies for floor or ceiling effects.

We calculated the Kaiser-Meyer-Olkin measure of adequacy (KMO) and Bartlett’s test to judge whether principal component analysis was appropriate.

We calculated Cronbach’s alpha (which should exceed 0.7 for good internal consistency).

We calculated item-total correlations for each question (which should exceed 0.2 for good homogeneity).

We undertook principal components analysis to assess the underlying structure; we considered factors important if their eigenvalues were clearly greater than 1, and individual questions as useful if their factor loadings exceeded 0.4.

We assessed the construct validity of the scale by examining the Pearson correlation between the CUCQ and two generic quality-of-life questionnaires – EQ-5D and SF-12.

Ethics

We received ethical approval from the Research Ethics Committee for Wales (Ref 08/MRE09/42); National Health Service (NHS) Research & Development (R&D) approval from each participating Trust or Health Board; and Medicine and Healthcare Regulatory Agency (MHRA) approval to undertake the CONSTRUCT trial.

Results

Patient sample

We had a total of 270 patients in our RCT validation sample. Table 1 gives the demographic characteristics of the CONSTRUCT RCT sample.

Table 1.

Clinical and demographic characteristics of the CONSTRUCT RCT participants.

Patient characteristic	RCT sample Maximum N = 270
Gender
Male	170
Female	100
Mean age (years (SD)) at recruitment	40.06 (15.31)
Ethnicity
White	250
Asian or Asian British	12
Black or Black British	3
Other ethnic groups	2
Mixed	0
Missing	3
Truelove & Witt Classification
Severe	251
Not severe	16
Mayo score
0	4
1	4
2	68
3	183
Co-morbidity
Yes	44
No	225
Mean EQ-5D (SD)	0.51 (0.30)
Mean CUCQ (SD)	0.36 (0.13)
Mean SF-6D (SD)	0.56 (0.11)

CONSTRUCT: COmparison of iNfliximab and ciclosporin in STeroid Resistant Ulcerative Colitis Trial; RCT: Randomised Controlled Trial; CUCQ: Crohn’s and Ulcerative Colitis questionnaire.

Validation of the CUCQ in the CONSTRUCT RCT

We examined the data prior to undertaking principal components analysis. The KMO was 0.844; and the Bartlett’s test of Sphericity was 0.000, indicating that the data were suitable principal components analysis.

The internal consistency of the CUCQ in the RCT sample was excellent with a Cronbach’s alpha of 0.845. Table 2 illustrates the item-total correlations and the maximum response rates for each of the 32 questions. All but two of the questions (Q1 and Q9) had an item-total correlation of more than 0.2. In addition, only two of the questions had a response rate greater than 80% (Q1 and Q6). Q1 was the only question that showed a slight ceiling effect, based on using the recommended value of 15% for the percentage of patients scoring the highest or lowest scores. ⁸

Table 2.

The CUCQ questions, their item total correlations and their maximum response rate (CONSTRUCT RCT sample).

Question	Item-total correlation	Maximum response rate
1. On how many days over the last two weeks have you had loose or runny bowel movements?	0.131	88.1%
2. On how many days in the last two weeks have you noticed blood in your stools?	0.230	70.4%
3. On how many days over the last two weeks have you felt tired?	0.506	73.3%
4. In the last two weeks have you felt frustrated?	0.493	41.5%
5. In the last two weeks, has your bowel condition prevented you from carrying out your work or other normal activities?	0.481	41.9%
6. On how many days over the last two weeks have you opened your bowels more than three times a day?	0.286	86.3%
7. On how many days over the last two weeks have you felt full of energy?	0.318	74.4%
8. In the last two weeks did your bowel condition prevent you from going out socially?	0.505	44.4%
9. On how many days over the last two weeks have your bowels opened accidentally?	0.194	46.7%
10. On how many days over the last two weeks have you felt generally unwell?	0.590	63.0%
11. In the last two weeks have you felt the need to keep close to a toilet?	0.500	56.3%
12. In the last two weeks, has your bowel condition affected your leisure or sports activities?	0.564	56.7%
13. On how many days over the last two weeks have you felt pain in your abdomen?	0.396	57.8%
14. On how many nights over the last two weeks have you been unable to sleep well (days if you are a shift worker)?	0.453	61.5%
15. On how many nights in the last two weeks have you had to get up to use the toilet because of your bowel condition after you have gone to bed?	0.436	70.0%
16. In the last two weeks have you felt depressed?	0.534	56.3%
17. In the last two weeks have you had to avoid attending events where there was no toilet close at hand?	0.529	36.7%
18. On how many days over the last two weeks, have you had a problem with large amounts of wind?	0.270	40.0%
19. On how many days over the last two weeks have you felt off your food?	0.294	27.0%
20. Many patients with bowel problems have worries about their illness. How often during the last two weeks have you felt worried?	0.508	34.8%
21. On how many days over the last two weeks has your abdomen felt bloated?	0.396	31.5%
22. In the last two weeks have you felt relaxed?	0.478	48.5%
23. In the last two weeks have you been embarrassed by your bowel problem?	0.466	38.1%
24. On how many days over the last two weeks have you wanted to go back to the toilet immediately after you thought you had emptied your bowels?	0.462	34.8%
25. In the last two weeks have you felt upset?	0.568	60.0%
26. On how many days over the last two weeks have you had to rush to the toilet?	0.409	61.9%
27. In the last two weeks have you felt angry as a result of your bowel problem?	0.423	50.4%
28. In the last two weeks, has your sex life been affected by your bowel problem?	0.391	45.2%
29. On how many days over the last two weeks have you felt sick?	0.466	22.6%
30. In the last two weeks have you felt irritable?	0.572	53.3%
31. In the last two weeks have you felt lack of sympathy from others?	0.266	64.1%
32. In the last two weeks have you felt happy?	0.377	54.8%

CONSTRUCT: COmparison of iNfliximab and ciclosporin in STeroid Resistant Ulcerative Colitis Trial; RCT: Randomised Controlled Trial; CUCQ: Crohn’s and Ulcerative Colitis questionnaire.

The principal components analysis indicated that there were four main factors with an eigenvalue of greater than 1 (Table 3), and which explained approximately 42% of the variance in the data. Where questions loaded onto more than one factor, we attributed the question to the factor with the greater factor loading. ⁷ The principal components analysis identified that the first factor covered emotional symptoms; the second bowel symptoms; the third social activities and the fourth general symptoms, with most questions exceeding the required factor loading of 0.4.

Table 3.

Principal component analysis of the CONSTRUCT CUCQ questions from the CONSTRUCT RCT sample based on a four-factor solution.

	Factors
	1 Social	2 Psychological	3 General	4 Bowel
Percentage of factor’s contribution	22.13%	7.72%	6.64%	5.20%
Eigenvalue	7.08	2.47	2.12	1.67
Q5. In the last two weeks, has your bowel condition prevented you from carrying out your work or other normal activities?	0.709
Q8. In the last two weeks did your bowel condition prevent you from going out socially?	0.872
Q11. In the last two weeks have you felt the need to keep close to a toilet?	0.621
Q12. In the last two weeks, has your bowel condition affected your leisure or sports activities?	0.803
Q17. In the last two weeks have you had to avoid attending events where there was no toilet close at hand?	0.637
Q28. In the last two weeks, has your sex life been affected by your bowel problem?	0.452
Q4. In the last two weeks have you felt frustrated?		–0.714
Q16. In the last two weeks have you felt depressed?		–0.696
Q20. Many patients with bowel problems have worries about their illness. How often during the last two weeks have you felt worried?		–0.615
Q22. In the last two weeks have you felt relaxed?		–0.450
Q23. In the last two weeks have you been embarrassed by your bowel problem?		–0.592
Q25. In the last two weeks have you felt upset?		–0.725
Q27. In the last two weeks have you felt angry as a result of your bowel problem?		–0.754
Q30. In the last two weeks have you felt irritable?		–0.533
Q31. In the last two weeks have you felt lack of sympathy from others?		–0.364
Q32. In the last two weeks have you felt happy?		–0.368
Q1. On how many days over the last two weeks have you had loose or runny bowel movements?			0.455
Q2. On how many days in the last two weeks have you noticed blood in your stools?			0.351
Q3. On how many days over the last two weeks have you felt tired?			0.514
Q6. On how many days over the last two weeks have you opened your bowels more than three times a day?			0.494
Q7. On how many days over the last two weeks have you felt full of energy?			0.483
Q10. On how many days over the last two weeks have you felt generally unwell?			0.485
Q13. On how many days over the last two weeks have you felt pain in your abdomen?			0.613
Q14. On how many nights over the last two weeks have you been unable to sleep well (days if you are a shift worker)?			0.411
Q15. On how many nights in the last two weeks have you had to get up to use the toilet because of your bowel condition after you have gone to bed?			0.445
Q19. On how many days over the last two weeks have you felt off your food?			0.509
Q29. On how many days over the last two weeks have you felt sick?			0.355
Q9. On how many days over the last two weeks have your bowels opened accidentally?				0.316
Q18. On how many days over the last two weeks, have you had a problem with large amounts of wind?				0.684
Q21. On how many days over the last two weeks has your abdomen felt bloated?				0.586
Q24. On how many days over the last two weeks have you wanted to go back to the toilet immediately after you thought you had emptied your bowels?				0.547
Q26. On how many days over the last two weeks have you had to rush to the toilet?				0.534

Extraction method: Principal component analysis Rotation method: Oblimin with Kaiser normalization

a

. Rotation converged in 15 iterations.

CONSTRUCT: COmparison of iNfliximab and ciclosporin in STeroid Resistant Ulcerative Colitis Trial; RCT: Randomised Controlled Trial; CUCQ: Crohn’s and Ulcerative Colitis questionnaire.

Table 4 illustrates that the CUCQ scores from the CONSTRUCT RCT sample achieved significant correlations (p < 0.001) with the two generic health-related QoL scales (SF-12 Mental Component summary scores –0.588; Physical Component summary scores –0.442 and the EQ-5D –0.459)) scores demonstrating good construct validity. ⁷

Table 4.

Correlations between the CUCQ and the SF-12 MCS and PCS and EQ-5D in the CONSTRUCT RCT sample.

Scale	CUCU
SF-12 MCS	–0.588 a
SF-12 PCS	–0.452 a
EQ-5D	–0.459 a

a

p < 0.001 Pearson’s correlation.

MCS: Mental Health Composite; PCS: Physical Health Composite; CUCQ: Crohn’s and Ulcerative Colitis questionnaire; CONSTRUCT: COmparison of iNfliximab and ciclosporin in STeroid Resistant Ulcerative Colitis Trial; RCT: Randomised Controlled Trial.

We carried out a stepwise regression to identify the potential to shorten the CUCQ and the smallest number of questions required to do so (Table 5). Stepwise regression identified 16 items that accounted for greater than 95% of the variance of the CUCQ. The questions identified as being major contributors to a shortened scale in the CONSTRUCT RCT sample of acute patients were, however, different to those identified in mild to moderate patients, with only three of the eight items selected for a short form in mild to moderate patients ⁶ being selected for patients with acute severe UC.

Table 5.

Model summary of the CUCQ questions in the CONSTRUCT RCT sample.

Questions	Cumulative % of variance
Q10. On how many days over the last two weeks have you felt generally unwell?	34.5
Q30. In the last two weeks have you felt irritable?	58.8
Q12. In the last two weeks, has your bowel condition affected your leisure or sports activities?	67.4
Q3. On how many days over the last two weeks have you felt tired?	74.0
Q23. In the last two weeks have you been embarrassed by your bowel problem?	79.1
Q24. On how many days over the last two weeks have you wanted to go back to the toilet immediately after you thought you had emptied your bowels?	81.8
Q16. In the last two weeks have you felt depressed?	84.0
Q29. On how many days over the last two weeks have you felt sick?	86.2
Q5. In the last two weeks, has your bowel condition prevented you from carrying out your work or other normal activities?	87.8
Q18. On how many days over the last two weeks, have you had a problem with large amounts of wind?	89.5
Q32. In the last two weeks have you felt happy?	90.8
Q28. In the last two weeks, has your sex life been affected by your bowel problem?	91.7
Q14. On how many nights over the last two weeks have you been unable to sleep well (days if you are a shift worker)?	92.7
Q20. Many patients with bowel problems have worries about their illness. How often during the last two weeks have you felt worried?	93.7
Q17. In the last two weeks have you had to avoid attending events where there was no toilet close at hand?	94.5
Q2. On how many days in the last two weeks have you noticed blood in your stools?	95.2
Q26. On how many days over the last two weeks have you had to rush to the toilet?	95.8
Q31. In the last two weeks have you felt lack of sympathy from others?	96.3
Q21. On how many days over the last two weeks has your abdomen felt bloated?	96.7
Q27. In the last two weeks have you felt angry as a result of your bowel problem?	97.1
Q22. In the last two weeks have you felt relaxed?	97.5
Q19. On how many days over the last two weeks have you felt off your food?	97.8
Q13. On how many days over the last two weeks have you felt pain in your abdomen?	98.1
Q11. In the last two weeks have you felt the need to keep close to a toilet?	98.4
Q9. On how many days over the last two weeks have your bowels opened accidentally?	98.7
Q4. In the last two weeks have you felt frustrated?	98.9
Q15. On how many nights in the last two weeks have you had to get up to use the toilet because of your bowel condition after you have gone to bed?	99.2
Q7. On how many days over the last two weeks have you felt full of energy?	99.4
Q8. In the last two weeks did your bowel condition prevent you from going out socially?	99.6
Q25. In the last two weeks have you felt upset?	99.7
Q1. On how many days over the last two weeks have you had loose or runny bowel movements?	99.9
Q6. On how many days over the last two weeks have you opened your bowels more than three times a day?	100

CUCQ: Crohn’s and Ulcerative Colitis questionnaire; CONSTRUCT: COmparison of iNfliximab and ciclosporin in STeroid Resistant Ulcerative Colitis Trial; RCT: Randomised Controlled Trial.

Discussion

We previously validated the CUCQ in patients with stable or moderately active inflammatory bowel disease (IBD) ⁶ where it demonstrated good psychometric properties. It had not previously been applied to acute severe patients, however, so we needed to validate it on patients with severe disease. We therefore tested the validity of the CUCQ within the CONSTRUCT trial ³ on patients diagnosed with acute severe UC.

Psychometric analysis of the CUCQ in the CONSTRUCT sample of acute severe patients demonstrated excellent internal consistency (Cronbach’s alpha > 0.8). There were only a handful of questions which demonstrated item-total correlations less than 0.2, response rates of greater than 80% or ceiling effects. Principal components analysis indicated that there were four main factors with an eigenvalue of greater than 1 and which explained 42% of the variance in the data. The CUCQ scores also achieved significant correlations with both the SF-12 mental and physical component summary scores and the EQ-5D demonstrating good construct validity. Stepwise regression analysis identified that 16 questions explained greater than 95% of the variance in the CUCQ when applied to the acute severe UC patients in the CONSTRUCT RCT sample. Interestingly the questions that were identified as being the major contributors to a shorter scale were different in the acute sample to those identified in a mild to moderate sample ⁶ (only three of the eight questions identified for the CUCQ ⁸ in the mild to moderate sample were identified in the acute severe sample). This would suggest that the full 32 questions in the CUCQ would be more appropriate to use across the spectrum of disease, to ensure that the changing picture of the disease on QoL is captured.

Although initial psychometric testing of the CUCQ demonstrated good validity, more work is needed to explore the responsiveness and test-retest validity of the CUCQ in an acute severe sample. Although the CONSTRUCT RCT patients completed follow-up questionnaires, there were insufficient data to satisfactorily undertake these analyses. Further work is also needed to explore the usefulness of the individual factor scores in monitoring patient QoL.

There have been no questionnaires that have been suitable for patients who have undergone colectomy surgery. We developed an extension to the CUCQ (CUCQ+), which included 10 additional questions specifically for stoma patients, and which excluded six non-relevant questions from the CUCQ. We undertook initial piloting and testing of the CUCQ+ within CONSTRUCT but need to gather additional data to undertake a full psychometric analysis (including principal components analysis, consistency, construct validity, responsiveness and test-rest validity).

In order to be clinically applicable we aim to produce a questionnaire which could be applied across the spectrum of disease from mild to severe (including one that is applicable to patients who undergo colectomy surgery), to longitudinally assess patient disease-specific QoL over time. In order to be practical in clinical practice this questionnaire should be short, so we would aim to produce a questionnaire that captures the most important symptoms across the spectrum of disease severity. Work is already underway to recruit more patients to undertake this further validation work and development of a short combined severity form.