Sage Journals: Discover world-class research

Abstract

Objective

This study aimed to examine the test–retest reliability and responsiveness of the Self-Regulation Assessment, a recently developed patient-reported outcome measure designed to evaluate self-regulation in a multi-diagnostic rehabilitation population.

Design

A prospective cohort study following COSMIN guidelines. Participants completed the Self-Regulation Assessment and other measurements at start of rehabilitation (T0), 6 months later (T1), and 2 weeks after T1 (T2). Test–retest reliability (T1-T2) was quantified by intraclass correlation coefficient values, Bland-Altman plots, and the smallest detectable change. Responsiveness (T0-T1) was quantified by hypothesis testing, effect size, area under the curve, and minimal important change based on the anchor Global Rating of Change scale of self-regulation.

Setting and participants

Inpatients and outpatients of 14 Dutch rehabilitation institutions with various diagnosis.

Main measure

The Self-Regulation Assessment.

Results

In total, 555 patients completed the Self-Regulation Assessment at T0 and T1 and 167 patients at T1 and T2. In inpatients and outpatients, the Self-Regulation Assessment showed adequate to good reliability, with smallest detectable changes ranging from 16.0 to 22.9 points at individual level and 1.5 to 3.3 at group level for outpatients and inpatients, respectively. Significant changes (T0-T1) were observed in both groups, with a small effect size for inpatients and large for outpatients. Hypothesis testing indicated near sufficient responsiveness in both groups (67% confirmed). Minimal important change values ranged from 6.25 to 9.8 points for outpatients.

Conclusion

The Self-Regulation Assessment demonstrated sufficient reliability and detected changes at group level, but was not suitable for detecting changes at individual level in outpatient rehabilitation.

Keywords

Self-regulation PROM measurement properties responsiveness rehabilitation

Introduction

Self-regulation involves an individual's ability to manage their thoughts, emotions, and behaviour to achieve long-term goals.¹ In the context of rehabilitation, self-regulation enables patients to actively engage in their recovery, adhere to treatment procedures, and adapt to new challenges associated with living with a chronic disability.² Effective self-regulation has been associated with better functional outcomes, higher quality of life, and enhanced autonomy, highlighting the importance of self-regulation in the rehabilitation process.^3,4 Accurate measurement of self-regulation allows healthcare professionals to identify patients who may require additional support, tailor interventions to individual needs, and track progress over time.^5,6

According to previous research, former rehabilitation patients defined self-regulation in the context of rehabilitation medicine as both the restoration of self-regulatory abilities and their practical application across multiple domains of life.⁷ This led to the identification of four key aspects: insight into one's health condition, awareness of own capabilities and possibilities, self-trust and application of self-regulation, and organisation of help.⁷ However, a patient-reported outcome measure to assess these aspects was lacking.⁸ Therefore, Mol et al. developed the Self-Regulation Assessment to provide a comprehensive assessment of an individual's self-regulatory capacity during rehabilitation.⁹ To ensure its applicability in routine rehabilitation care, it is essential to establish its psychometric properties.¹⁰ While previous studies have focused on the content and construct validity of the Self-Regulation Assessment, its test–retest reliability and responsiveness remain unknown – both of which are necessary for classifying the Self-Regulation Assessment as an evaluative measure.^4,9,11 Test–retest reliability reflects the instrument's consistency, while responsiveness assesses its ability to detect clinically meaningful changes over time.^12,13 According to the COSMIN guidelines, responsiveness is considered as longitudinal validity and is assessed by hypothesis testing.¹⁴ Hypotheses must be predefined and can be based on criterion validity and construct validity. Criterion-based hypotheses compare change scores with a global rating of change measure, while construct-based hypotheses compare change scores with expected changes in comparator patient-reported outcome measures or between known groups.

Responsiveness may differ across care settings due to variations in treatment goals, interventions, and onset of diagnosis.^10,15 Inpatient rehabilitation primarily focuses on regaining independence and mobility in patients with acute onset conditions, whereas outpatient rehabilitation mainly focuses on social participation and coping with their condition in patients with more chronic conditions. Therefore, it is essential to examine whether changes in self-regulation differ between inpatient and outpatient rehabilitation settings. In this study, we aim to examine the test–retest reliability and responsiveness of the Self-Regulation Assessment in inpatient and outpatient rehabilitation populations, in accordance with the COSMIN guidelines. A priori, we hypothesised that the Self-Regulation Assessment would show adequate responsiveness, indicated by moderate correlations with the global rating of change scale and sufficient area under the curve values, weak correlations with unrelated comparator patient-reported outcome measures, and larger effects in outpatients than inpatients.

Methods

Study design

This multicentre prospective cohort study is part of the Measurement of Outcomes of Rehabilitation in the Netherlands (MUREVAN) research project. Participants completed a questionnaire including the Self-Regulation Assessment at three measurement time points: at the start of rehabilitation treatment (T0), 6 months later (T1), and 2 weeks after T1 (T2). This study did not fall under the scope of the Dutch Medical Research Involving Human Subjects Act and therefore did not require approval from an accredited Medical Ethics Committee in the Netherlands. However, an independent quality check was conducted at the University Medical Centre Utrecht to ensure compliance with legal and regulatory requirements, including informed consent procedures, data management, privacy and ethical considerations (20-01-2023, reference number 212217652). Additionally, all participating institutions gave approval to conduct the study at their sites. The study was conducted in compliance with the Declaration of Helsinki and reported according to the COSMIN guidelines.¹³

Setting and participants

A multi-diagnostic group of adult inpatients and outpatients was recruited from 14 Dutch rehabilitation institutions, including three university hospitals and two general hospitals with outpatient facilities, eight medical rehabilitation centres with both in- and outpatient facilities, and one outpatient rehabilitation clinic. Participants were eligible for inclusion if they were at least 18 years old and were diagnosed with either acquired brain injury (e.g. stroke, traumatic brain injury and brain tumour), chronic musculoskeletal pain disorder, spinal cord injury, neurological disorder (e.g. multiple sclerosis and neuromuscular diseases), or oncological condition. Additionally, they had to be scheduled for multidisciplinary treatment lasting at least 4 weeks. Exclusion criteria included the inability to complete a questionnaire and having a rapidly progressive condition (life expectancy < 1 year).

To ensure sufficient statistical power, sample size estimates for this study were based on established quality criteria of the COSMIN guidelines, which consider sample sizes of >50 participants as adequate and >100 participants as very good for both test–retest reliability analyses and hypothesis testing of responsiveness.¹² For the calculation of the area under the curve, COSMIN recommends a sample size of >30 as adequate in the smallest group and >50 as very good. To ensure practical feasibility while accounting for a potential dropout rate of approximately 30%, we aimed to recruit 100 inpatients each for the diagnostic groups acquired brain injury and spinal cord injury, and 125 outpatients each for the following diagnostic groups: acquired brain injury, chronic musculoskeletal pain disorders, neurological and neuromuscular disorders, and oncological conditions. This resulted in a total intended sample size of 700 participants at baseline (T0): 200 inpatients and 500 outpatients. For T2, we aimed to include a total of 200 participants, 50 inpatients and 150 outpatients, to assess test–retest reliability. Participants were selected through consecutive sampling of those who first completed the follow-up assessment.

Procedures

At participating institutions, rehabilitation physicians and physician-assistants invited eligible patients during intake (outpatients) or at the start of rehabilitation (inpatients). The physician provided interested patients an information letter and forwarded their contact information to the researcher, who confirmed willingness of the patient to participate via email. The researcher sent a personalised link to the questionnaire via the Castor Electronic Data Capture system or a paper version by postal mail, according to the participant's preference. The questionnaire included informed consent, demographic questions, and patient-reported outcome measures. After completion of the baseline questionnaire, the researcher sent a personalised Castor Electronic Data Capture link to the participant's physician to collect disease-related information. To ensure participants remain stable between T1 and T2, we used the recommended interval of 2 weeks between these measurements, which is considered long enough to prevent recall bias, while short enough to ensure that patients remain stable.¹⁶ Moreover, follow-up questionnaires were sent in the same manner with the same instructions as the previous measurement. The researcher sent up to three reminders via email or phone within a three-week period after each invitation. Patients were recruited between March 2023 and October 2024, with the final follow-up measurements completed in April 2025.

Outcome measures

Self-regulation assessment

The Self-Regulation Assessment is a 22-item measure designed to assess self-regulation in the context of rehabilitation.^9,11 It measures self-regulation with four subscales: self-insight in own health condition, awareness of own capabilities, trust on self and application of self-regulation in one's own life, and organisation of help. All items are scored on a five-point scale, ranging from ‘totally disagree’ up to ‘totally agree’. The total score (all items) and four subscale scores (four to eight items) are calculated by converting the sum of the item scores into a scale ranging from zero up to 100, with higher scores indicating better self-regulation.

Global rating of change scale

The global rating of change scale is a single-item measure used as an external criterion (anchor) to assess self-reported change in an outcome domain at follow-up. Participants were asked to indicate how much their self-regulation had changed between the start of the rehabilitation treatment (T0) and follow-up (T1), with the question: ‘How has your self-regulation changed compared to the start of your rehabilitation treatment?’. This item is scored on a seven-point scale, ranging from ‘much worse’ up to ‘much improved’. For its use as an anchor, the response options were dichotomised into an improved group ('slightly improved’ up to ‘much improved’) and an unimproved group (‘much worse’ up to ‘no change’). Although the reliability and validity of this single-item measure is uncertain, the global rating of change scale is considered a realistic reference standard to calculate the minimal important change of a patient-reported outcome measure.¹⁰

Patient-reported outcomes measurement information system emotional distress short forms

We used the four-item short form applications of the patient-reported outcomes measurement information system (PROMIS) Anxiety and the PROMIS Depression v1.0 item banks to evaluate symptoms of anxiety and depression.^17,18 These measures assess the frequency of anxiety and depression symptoms over the past seven days. All items are scored on a five-point scale, ranging from ‘never’ up to ‘always’. A total score is calculated by converting the sum of the item scores into a T-score metric (mean = 50, standard deviation = 10) using a concordance table.^19,20 The T-score ranges from 40.3 to 81.6 for the PROMIS Anxiety short form and from 41.0 to 79.4 for PROMIS Depression short form, with higher scores indicating higher levels of emotional distress. The items of the PROMIS Anxiety and Depression item banks have demonstrated good validity and moderate to good responsiveness across various rehabilitation populations.^21–24

PROMIS Ability to Participate in Social Roles and Activities short form

We used the four-item short form application of the PROMIS Ability to Participate in Social Roles and Activities v2.0 item bank to evaluate participation.²⁵ This short form assesses perceived ability to perform usual social roles and activities. The PROMIS Ability to Participate in Social Roles and Activities short form items rate the level of limitation on a five-point scale, ranging from ‘never’ up to ‘always’. The sum score of the items is calculated, which is then transformed into a T-score metric by using a concordance table.²⁶ The T-score ranges from 27.5 (low ability) up to 64.2 (high ability). Across various rehabilitation populations, PROMIS Ability to Participate in Social Roles and Activities short form items have showed an adequate internal consistency, good construct validity, and moderate to good responsiveness.^24,27–29

PROMIS Physical Function short form

To assess physical functioning, we used a 23-item short form application which combines the seven-item short form Upper Extremity of the PROMIS Bank v2.0 Physical Function with the 16-item Mobility Custom short form for Rehabilitation of the PROMIS Bank v2.1 Mobility.^17,30,31 This custom short form was developed primarily to evaluate the effectiveness of multidisciplinary rehabilitation treatment, based on a pilot study of 146 rehabilitation patients. The 23 items of the Physical Function short form measures self-reported competence of physical mobility activities, such as getting out of a chair or running, and physical activities that require use of shoulder, arm, and hand. The seven upper extremity items and 10 mobility items rate the level of difficulty on a five-point scale, ranging from ‘no difficulty’ up to ‘not possible’, and six mobility items rate the level of limitation on a five-point scale, ranging from ‘not at all’ up to ‘not possible’. By using a concordance table, the sum score of the items is transformed into a T-score metric.³² T-scores range from 14.1 up to 63.5, with higher scores indicating better physical functioning. Overall, the PROMIS Bank Physical Function has demonstrated adequate psychometric properties for assessing physical functioning across various rehabilitation populations.^28,33–35

The 5-level EQ-5D version (EQ-5D-5L)

The Dutch version of the EQ-5D-5L was used to assess health-related quality of life.³⁶ This measure consists of five items, each covering a health-related quality of life domain: mobility, self-care, activities of daily living, pain, and depression or anxiety. Items are scored on a five-point scale ranging from ‘no problems at all’ up to ‘not possible at all’. We used the EQ-5D-5L crosswalk index score calculator to convert the item scores into a total score.³⁷ A total score of one represents perfect health and a negative score represents health worse than death, while a score of zero represents death. The EQ-5D-5L is a reliable, valid, and responsive measure for assessing health-related quality of life across a wide range of diagnostic groups.^38–41

Patient and disease-related characteristics

Collected demographic variables included sex, age, living situation (alone or with partner, kids or others), education level, ongoing rehabilitation treatment (either the initial treatment or follow-up treatment) and cognitive and physical functioning (self-reported outcomes). Disease-related variables included diagnosis, inpatient or outpatient treatment, and the presence of comorbidities (yes or no). To assess whether participants remained stable between T1 and T2, they were asked at T2 to what extent their general health and/or daily functioning had changed over the last 2 weeks.

Statistical analysis

Data were analysed using IBM SPSS Statistics 27.0 (Windows; Armonk, NY). Analyses were performed separately for the inpatient and outpatient group.

To describe the study sample at T1 (responsiveness sample) and T2 (test–retest sample), we summarised categorical variables as frequencies and proportions, and continuous variables as means and standard deviations or medians and interquartile range. We calculated the distributional statistics and internal consistency of the Self-Regulation Assessment at T0 and T1 (responsiveness sample) and T1 and T2 (test–retest sample), with mean scores (± standard deviation), score range, ceiling effect (>15% scored maximal), and Cronbach's alpha (>0.70 = adequate consistency).^42,43

Test–retest reliability (T1-T2)

We analysed test–retest reliability and measurement error in participants reporting ‘no’ or ‘little’ health change between T1 and T2. In addition, we performed a stricter sensitivity analysis by evaluating test–retest reliability in the subgroup of participants who reported ‘no change’ in their health status. Test–retest reliability was determined using the intraclass correlation coefficient with 95% confidence interval (two-way mixed-effects model for absolute agreement). An intraclass correlation coefficient of >0.70 indicates adequate and >0.80 good reliability.^12,44 We assessed the measurement error with a paired t-test and Bland-Altman plots.⁴⁵ We calculated the limits of agreement using the formula: mean difference score ±1.96 standard deviation of the mean difference score. The smallest detectable change at individual level was calculated as 1.96*√2*standard error of measurement (standard deviationPooled* (√1−intraclass correlation coefficient)) and at group level by smallest detectable change individual level/√N.⁴⁶

Responsiveness (T0-T1)

We calculated the effect size by dividing the mean change by the standard deviation at baseline, and the standardised response mean by dividing the mean change by the standard deviation of the change score (0.2 = small, 0.5 = moderate, 0.8 = large).⁴⁷ We also performed paired t-tests, with a statistical significance set at p < 0.05.

For hypothesis testing, we formulated a priori 12 hypotheses to evaluate responsiveness (Table 1), using Spearman correlations and the area under the curve derived from the receiver operating characteristic curve. We expected a moderate correlation between the Self-Regulation Assessment change score and the global rating of change scale, a sufficient area under the curve, weak correlations with unrelated comparator measures, and a larger effect in outpatients than in inpatients.^4,48 We considered an adequate responsiveness when at least 75% of the results were in accordance with the hypotheses.¹²

Table 1.

Hypotheses of responsiveness for the Self-Regulation Assessment.

Number	Hypothesis	Expected direction / threshold
1	The SeRA change score correlates moderately with the GRC scale.	ρ ≥ 0.50
2	The SeRA change score sufficiently discriminates between improved and non-improved patients.	AUC ≥ 0.70
3	The SeRA change score does not strongly correlate with the change scores of comparator PROMs.	ρ ≤ 0.50
4–6	The SeRA change score shows stronger correlations with PROMIS-A-SF than with PROMIS-PF-SF (4), PROMIS-APS-SF (5) and/or EQ-5D (6).	Higher correlations with mental health than other outcomes
7–9	The SeRA change score shows stronger correlations with PROMIS-D-SF than with PROMIS-PF-SF (7), PROMIS-APS-SF (8) and/or EQ-5D (9).	Higher correlations with mental health than other outcomes
10	The SeRA change score shows stronger correlations with PROMIS-APS-SF than with PROMIS-PF-SF.	Higher correlations with participation than physical functioning
11	The SeRA change score shows stronger correlations with EQ-5D than with PROMIS-PF-SF.	Higher correlations with quality of life than physical functioning
12	The SeRA change score demonstrates a larger effect size in outpatients compared with inpatients.	> effect size in outpatients than inpatients

SeRA: Self-Regulation Assessment; GRC: global rating of change; ρ: Spearman correlation; AUC: area under the curve; PROMs: patient-reported outcome measurements; PROMIS-A-SF: PROMIS Anxiety short form; PROMIS-D-SF: PROMIS Depression short form; PROMIS-APS-SF: PROMIS Ability to Participate in Social Roles and Activities short form; EQ-5D: EuroQoL-5D-5L; PROMIS-PF-SF: PROMIS Physical Function short form.

Minimal important change

A correlation of >0.30 between the Self-Regulation Assessment change score and the global rating of change scale is required to calculate a minimal important change value.⁴⁹ To determine the minimal important change, we used three anchor-based methods: mean change method,⁵⁰ receiver operating characteristic curve method,⁵¹ and predictive modelling method.⁵² The minimal important change mean is determined as the mean change score in participants reporting ‘slightly improved’ on the anchor. Based on the dichotomised global rating of change scale, the minimal important change based on the receiver operating characteristic method was determined as the optimal receiver operating characteristic cut-off point and the minimal important change based on predictive modelling was calculated using logistic regression analysis with 95% confidence intervals.⁵² A minimal important change greater than the smallest detectable change indicates that the Self-Regulation Assessment can reliably measure important change.¹⁰

Results

Descriptive analysis

Approximately 70% of the 1094 patients recruited by their rehabilitation physician provided informed consent to participate in the study (Figure 1). Of these patients, 733 met the inclusion criteria and completed the baseline questionnaire (T0). Almost 80% of these participants completed the 6-month follow-up questionnaire (T1). In total, 14 participants were excluded from the responsiveness analysis due to no completed SeRA at T0 or T1. Two weeks after T1, 44% of the participants were invited to complete the test–retest questionnaire, of which nearly 70% responded (T2). In total, 11 participants were excluded from the test–retest analysis: 10 participants due to substantial changes in their health status between T1 and T2, and one due to being identified as an outlier. This outlier showed an atypical difference of almost 50 points between T1 and T2 on the Self-Regulation Assessment total score without indicating any change in health status, which was inconsistent with the overall data pattern. Characteristics of the responsiveness sample (T1) and test–retest sample (T2) are summarised in Table 2.

Figure 1.

Flowchart inclusion of participants.

Table 2.

Baseline patient and disease-related characteristics.

	Responsiveness sample (n = 555)				Test–retest sample (n = 167)
	Inpatient (n = 141)		Outpatient (n = 414)		Inpatient (n = 47)		Outpatient (n = 120)
Demographic variables	n	n (%) or mean ± SD	n	n (%) or mean ± SD	n	n (%) or mean ± SD	n	n (%) or mean ± SD
Gender (female)	141	50 (35.5)	414	281 (67.9)	47	23 (48.9)	120	82 (68.3)
Age (years)	141	61 ± 13.6	414	54 ± 12.9	47	64 ± 11.3	120	55 ± 12.1
Education (high) ^a	140	94 (67.2)	414	312 (75.3)	47	31 (65.9)	120	87 (72.5)
Living situation	140		414		47		120
Alone		35 (25.0)		66 (15.9)		14 (29.8)		21 (17.5)
Together ^b		105 (75.0)		348 (84.1)		33 (70.2)		99 (82.5)
Days between T0-T1	141	193 (18)^c	414	190 (16)^c		N/A		N/A
Days between T1-T2		N/A		N/A	47	16 (11)^c	120	16 (9)^c
Disease-related variables
Diagnostic group	141		414		47		120
Acquired brain injury		88 (62.4)		125 (30.2)		34 (72.3)		29 (24.2)
Chronic pain disorder		N/A		97 (23.4)		N/A		34 (28.3)
Spinal cord injury		53 (37.6)		N/A		13 (27.7)		N/A
Neurological disorder		N/A		82 (19.8)		N/A		22 (18.3)
Oncological conditions		N/A		110 (26.6)		N/A		35 (29.2)
Rehabilitation completed at T1 (yes)	140	86 (61.4)	384	290 (75.5)	46	29 (63)	100	75 (75)
Cognitive functioning ^d	140		414		47		120
Memory (no problems)		82 (58.6)		130 (31.4)		28 (59.6)		51 (42.5)
Attention (no problems)		80 (57.1)		95 (22.9)		28 (59.6)		39 (32.5)
Planning (no problems)		86 (61.4)		111 (26.8)		35 (74.5)		48 (40.0)
Physical functioning ^d	140		414		47		120
Self-care (no problems)		42 (30.0)		258 (62.3)		29 (61.7)		88 (73.3)
Toileting (no problems)		59 (42.1)		343 (82.9)		37 (78.7)		104 (86.7)
Walking (no problems)		26 (18.6)		139 (33.6)		12 (25.5)		60 (50)
Comorbidity (yes)	141	59 (41.8)	414	85 (20.5)	47	15 (31.9)	120	26 (21.7)

≥ secondary education; ^bliving together with partner, family or friends; ^creported as median (IQR); ^dmeasured with a self-reported questionnaire.

Table 3 presents the distribution and the internal consistency of the Self-Regulation Assessment total score at T0 and T1 for the responsiveness sample and at T1 and T2 for the test–retest sample. The Self-Regulation Assessment demonstrated high internal consistency (α>0.90) and no ceiling effects (maximum scores <15% of the participants) at all measurement points and across all groups.

Table 3.

Distribution and internal consistency of the Self-Regulation Assessment total score.

	Responsiveness sample (N = 555)				Test–retest sample* (N = 167)
	Inpatient (N = 141)		Outpatient (N = 414)		Inpatient (N = 47)		Outpatient (N = 120)
	T0	T1	T0	T1	T1	T2	T1	T2
Mean ± SD	72.5 ± 14.1	76.3 ± 15.2	58.9 ± 15.9	71.2 ± 13.9	79 ± 17	80.2 ± 15.1	72.6 ± 13.6	73 ± 14.7
Range (min-max)	30–100	25–100	5–100	19–100	25–100	45–100	34–100	34–100
% max scores	2.8%	6.3%	0.5%	2.7%	6.4%	12.8%	1.7%	5.0%
Cronbach's α	0.93	0.96	0.94	0.95	0.97	0.97	0.94	0.95

*: participants who reported ‘no’ or ‘little’ health change between T1 and T2; SD: standard deviation; min: minimal; max: maximum.

Test–retest reliability

The intraclass correlation coefficient (95% confidence interval), paired sample t-test results, limits of agreement values, and smallest detectable change values of the test–retest sample (inpatient and outpatient group) are presented in Table 4. The Self-Regulation Assessment demonstrated adequate test–retest reliability in the inpatient group (intraclass correlation coefficient >0.70), and good test–retest reliability in the outpatient group (intraclass correlation coefficient >0.80).

Table 4.

Test–retest reliability, paired t-test and smallest detectable change of the Self-Regulation Assessment (T1 versus T2).

	Test–retest sample (N = 167)
	Inpatient (N = 47)	Outpatient (N = 120)
ICC (95%CI)	0.74 (0.58–0.85)	0.83 (0.77–0.88)
Range of diff score	−26.14–29.55	−25.00–21.59
Mean diff score ± SD	−1.2 ± 11.7	−0.5 ± 8.2
95%CI diff (p-value)	−4.61–2.24 (0.49)	−1.92–1.03 (0.55)
LoA	−24.05–21.68	−16.48–15.59
SDC individual	22.9	16.0
SDC group	3.3	1.5

ICC: intraclass correlation coefficient; CI: confidence interval; diff: difference; SD: standard deviation; LoA: limits of agreement; SDC: smallest detectable change.

The results of the paired t-tests indicated no significant systematic bias between the Self-Regulation Assessment total scores at T1 and T2 (p > 0.05) in both the inpatient and outpatient group. The Bland-Altman plots for the inpatient and outpatient group (Figure 2) revealed wide limits of agreement around the mean difference score, suggesting substantial measurement variability. Visual inspection of the plots indicated a slight increase in measurement variability as the mean difference increased. At individual level, smallest detectable change values for both the inpatient (22.9) and outpatient group (16.0) were relatively large in relation to the total range of the Self-Regulation Assessment (range 0–100). Additional sensitivity analyses of the test–retest reliability showed comparable results; however, the sample size of the inpatient group was too small to yield reliable estimates (Supplementary material 1).

Figure 2.

Bland-Altman plot of the Self-Regulation Assessment total score at T1 and T2 for the inpatient group (left) and outpatient group (right). Red solid line depicts the mean difference score and green dotted lines the limits of agreement.

Responsiveness

The mean changes in Self-Regulation Assessment scores between T0 and T1 were statistically significant in both the inpatient (p < 0.05) and the outpatient group (p < 0.01) (Table 5). The effect size and standardised response mean values indicated small changes in Self-Regulation Assessment scores between T0 and T1 in the inpatient group (0.25–0.23), and large in the outpatient group (0.77–0.81).

Table 5.

Responsiveness outcomes of the Self-Regulation Assessment: effect size, standardised response mean, area under the curve and minimal important change.

	Responsiveness sample (N = 555)
	Inpatient (N = 141)	Outpatient (N = 414)
Mean change ± SD	3.6 ± 15.6*	12.2 ± 15.1**
Effect size	0.25	0.77
SRM	0.23	0.81
AUC (95%CI)	0.56 (0.45–0.67)	0.65 (0.59–0.71)
MIC mean ± SD	N/A^a	9.0 ± 11.3
MIC ROC (sensitivity; specificity)	N/A^a	6.25 (0.69;0.55)
MIC predictive (95%CI)	N/A^a	9.8 (4.3–16.3)

SD: standard deviation; SRM: standardised response mean; AUC: area under the curve; CI: confidence interval; MIC: minimal important change; ROC: receiver operating characteristic; *p-value<0.05; **p-value<0.001.

Not applicable due to correlation <0.30 between Self-Regulation Assessment change score and global rating of change scale.

Table 6 presents the mean Self-Regulation Assessment change scores across the global rating of change scale response categories. The number of participants classified as ‘unimproved’ was small in both the inpatient group and outpatient group. Correlations between changes in Self-Regulation Assessment scores and in comparator PROMs scores and the global rating of change scale were generally weak to moderate, with slightly lower correlations observed in the inpatient group compared to the outpatient group (Table 7). Figure 3 displays the receiver operating characteristic curves for the Self-Regulation Assessment change scores in both the inpatient and outpatient group, with area under the curve values below 0.70 (Table 5).

Figure 3.

ROC curves of the Self-Regulation Assessment change score (T0 and T1) of the inpatient group and outpatient group. Blue dotted lines depicts the reference line and red solid lines the change score.

Table 6.

Self-Regulation Assessment change scores (T0 vs T1) for categories of improvement on the global rating of change scale.

	Responsiveness sample (N = 555)
	Inpatient (N = 141)		Outpatient (N = 414)
	N	Mean ± SD	N	Mean ± SD
Much worse	4	−9.9 ± 21.4	1	N/A
Moderately worse	5	5.7 ± 19.1	4	−2.6 ± 15.3
Slightly worse	8	−9.4 ± 19.1	16	1.4 ± 11.0
No change	28	5.6 ± 16.7	86	7.4 ± 13.4
Slightly improved	15	2.5 ± 8.9	103	9.0 ± 11.3
Moderately improved	34	3.6 ± 11.5	128	13.5 ± 14.6
Much improved	46	6.7 ± 16.2	76	23.1 ± 16.5

SD: standard deviation.

Table 7.

Correlations between Self-Regulation Assessment change scores and change scores in comparator PROMs and global rating of change scale.

	Responsiveness sample (N = 555)
	Inpatient (N = 141)	Outpatient (N = 414)
	Self-Regulation Assessment	Self-Regulation Assessment
GRC scale	0.15	0.35**
PROMIS-A-SF	−0.33**	−0.36**
PROMIS-D-SF	−0.31**	−0.42**
PROMIS-APS-SF	0.25**	0.29**
EQ-5D-5L	0.28**	0.33**
PROMIS-PF-SF	0.29**	0.34**

PROMs: patient-reported outcome measures; GRC: global rating of change; PROMIS-A-SF: PROMIS Anxiety short form; PROMIS-D-SF: PROMIS Depression short form; PROMIS-APS-SF: PROMIS Ability to Participate in Social Roles and Activities short form; EQ-5D-5L: 5-level EQ-5D version; PROMIS-PF-SF: PROMIS Physical Function short form; ** p-value<0.01.

Table 8 provides an overview of the hypothesis testing results. In both the inpatient and outpatient group, 67% of predefined hypotheses were confirmed, suggesting that the responsiveness of the Self-Regulation Assessment was insufficient according to hypothesis testing criteria.

Table 8.

Number of responsiveness hypotheses confirmed.

	Number of hypotheses confirmed
	Inpatient	Outpatient
Criterion
H1. GRC scale correlation >0.5	0/1	0/1
H2. AUC of ROC curve >0.70	0/1	0/1
Construct
H3. No correlations >0.5	1/1	1/1
H4/5/6. PROMIS-A-SF > correlation than with other comparator PROMs (except PROMIS-D-SF)	3/3	3/3
H7/8/9. PROMIS-D-SF > correlation than with other comparator PROMs (except PROMIS-A-SF)	3/3	3/3
H10. PROMIS-APS-SF > correlation than PROMIS-PF-SF	0/1	0/1
H11. EQ-5D > correlation than PROMIS-PF-SF	0/1	0/1
H12. Effect size Self-Regulation Assessment in outpatients > than in inpatients	1/1	1/1
Total number of hypotheses met	8 / 12 (67%)	8 / 12 (67%)

H: hypothesis; GRC: global rating of change; AUC: area under the curve; ROC: receiver operating characteristic; PROMIS-A-SF: PROMIS Anxiety short form; PROMIS-D-SF: PROMIS Depression short form; PROMIS-APS-SF: PROMIS Ability to Participate in Social Roles and Activities short form; EQ-5D: EuroQoL-5D-5L; PROMIS-PF-SF: PROMIS Physical Function short form.

Minimal important change

The minimal important change value was calculated only for the outpatient group (Table 5), given that the correlation between the Self-Regulation Assessment change score and the global rating of change scale exceeded 0.30 in this group. Depending on the method used, values for outpatients ranged from 6.25 (receiver operating characteristic method) to 9.8 (predictive modelling method).

All calculated minimal important change values exceeded the smallest detectable change value at group level, suggesting that the Self-Regulation Assessment can reliably detect meaningful change beyond measurement error in group-level analyses. However, all minimal important change values were lower than the smallest detectable change at individual level, indicating that the Self-Regulation Assessment is not capable of reliably detecting important change at individual level.

Discussion

This study aimed to examine the test–retest reliability and responsiveness of the Self-Regulation Assessment in both inpatient and outpatient rehabilitation populations. Test–retest reliability was adequate in inpatients and good in outpatients. Inpatients showed small changes in scores, whereas outpatients demonstrated large changes between baseline and follow-up. Hypothesis testing showed insufficient responsiveness in both groups, though both approached the 75% threshold. The Self-Regulation Assessment is suitable for detecting important change at group level; limited for monitoring important change at individual level.

The test–retest reliability of the Self-Regulation Assessment was adequate in both groups, which align with the findings from a systematic review on the reliability of other self-regulation measures.⁵³ While the lower bound of the intraclass correlation coefficient confidence interval was adequate in outpatients, it was below the acceptable threshold in inpatients. This difference is partly due to the relatively small sample size of the inpatient group, limiting the precision of the reliability estimates.⁵⁴

The smallest detectable change and limits of agreement values were relatively large in relation to the Self-Regulation Assessment's scale range. Large smallest detectable change and limits of agreement values are commonly observed in patient-reported outcome measures, as these instruments often assess complex subjective constructs that are susceptible to fluctuations in mood, contextual factors, and daily variability.^10,49 Our findings showed that the group-level smallest detectable change value was lower than the minimal important change values, supporting its use in outpatient rehabilitation settings to identify the proportion of patients who demonstrate meaningful improvements in self-regulation at group level.¹⁰ However, the flat receiver operating characteristic curves, the varying minimal important change values across methods, and the wide confidence intervals indicate considerable uncertainty in determining a single minimal important change value. This has important implications for clinical practice, where clinicians should interpret changes in Self-Regulation Assessment scores with caution. Clinicians could use the Self-Regulation Assessment change score as a starting point for dialogue with the patient, given that the importance of change is patient-specific, shaped by factors such as baseline health, perceived risks and benefits, and personal values.⁵⁵ Further research on the minimal important change is needed, preferably using alternative methods⁵⁶ or more robust anchors than the current global rating of change scale.⁵⁷

We observed significant changes in scores over time, particularly among outpatients. These findings are not unexpected since outpatient rehabilitation goals focus on supporting patients to adapt to their condition and learn to manage it in daily life, rather than on physical recovery.² The smaller changes in the inpatient group may be partly explained by the higher baseline scores in this group. Inpatients usually begin their rehabilitation soon after onset of their condition, often in a setting with intensive support from healthcare professionals and family members, which may temporarily mask self-regulation challenges. Similar findings of high baseline self-regulation scores among inpatients have been reported in previous research in inpatients with spinal cord injury.⁵⁸ Additionally, at the time of follow-up, 40% of the inpatient group still received some kind of rehabilitation treatment, which may have contributed to the smaller changes in scores between baseline and follow-up.

Fewer than the required 75% of our hypotheses were confirmed, mainly because of the weaker than expected correlations between Self-Regulation Assessment change scores and the global rating of change scale scores, and corresponding lower area under the curve values. Although commonly used as criterion, prior research has raised concerns regarding the validity and reliability of the global rating of change scale as an external anchor, primarily due to its single-item format and susceptibility to recall bias and present-state bias.^59–62

In addition, response shift could have occurred in our study, whereby participants internally redefine the construct being assessed over time.⁶³ In rehabilitation populations, both recall bias and response shift have been identified as threats to the validity of patient-reported outcome measures, including the global rating of change scale.^64–66 Further research should address potential biases in interpreting self-regulation changes using appropriate designs, such as the Then Test or Oort's Structural Equation Models.^67,68

The hypotheses based on the comparator measures were primarily informed by cross-sectional correlations observed in a former rehabilitation population.⁴ Nevertheless, the current findings align with previous literature which also suggest that self-regulation is more strongly associated with subjective constructs such as anxiety and depression, rather than with measures that assess more objective or functional domains.^4,58,69

A strength of this research is its multicentre design, which included participants across multiple diagnostic groups and from various institutions in the Netherlands. This enhances the generalizability of the findings to a broad rehabilitation population. However, several limitations should be acknowledged. First, the absence of triangulation with another self-regulation measure limited the ability to assess responsiveness according to hypothesis testing. Secondly, we defined minimal important change as ‘slightly improved’ on the anchor, whereas other studies sometimes applied higher thresholds of improvement (e.g. ‘moderately’ or ‘much improved’).¹⁰ Nonetheless, additional sensitivity analyses in our study showed minimal differences in area under the curve values between these cut-off points. Lastly, we limited our analysis to the Self-Regulation Assessment total score, without evaluating its subscales separately. Future studies could examine the responsiveness of the subscales by using aspect-specific global rating of change scales that align more closely with the multidimensional structure of the measure.

Overall, the Self-Regulation Assessment showed substantial test–retest reliability and was sensitive to meaningful change at group level in outpatient rehabilitation, as both observed changes and minimal important change values exceeded the smallest detectable change at group level. However, in both inpatient and outpatient rehabilitation, responsiveness was insufficient according to hypothesis testing, and the Self-Regulation Assessment showed limited ability to detect meaningful change at individual level. Consequently, the Self-Regulation Assessment appears suitable for group-level evaluation of self-regulation in outpatient rehabilitation. Further refinement and evaluation of responsiveness are recommended before using the measure for individual patient monitoring.

Clinical messages

The Self-Regulation Assessment is a reliable patient-reported outcome measure in rehabilitation.

The Self-Regulation Assessment is sensitive to meaningful change at group level in outpatient rehabilitation (minimal important change 6.25–9.8). However, its responsiveness based on hypothesis testing was insufficient.

The Self-Regulation Assessment appears to have limited ability to detect meaningful change in individual patients.

Supplemental Material

sj-pdf-1-cre-10.1177_02692155251413773 - Supplemental material for Test–retest reliability and responsiveness of the Self-Regulation Assessment in a rehabilitation population: A prospective multicentre validation study

Supplemental material, sj-pdf-1-cre-10.1177_02692155251413773 for Test–retest reliability and responsiveness of the Self-Regulation Assessment in a rehabilitation population: A prospective multicentre validation study by B.M.P. Mourits, E.W.M. Scholten, J.A. de Graaf, S. Oberink, R.J.E.M. Smeets, P. Stegeman, P.E.C.A. Passier, A.M. de Rooij, M.M.R. Riemens, J. Stolwijk, J.M.A. Visser-Meily and M.W.M. Post in Clinical Rehabilitation

Footnotes

Acknowledgements

We would like to thank all participating healthcare professionals and patients of the following Dutch institutions for their valuable contributions to this study: Amsterdam University Medical Center, Basalt Rehabilitation Centre, Clinics In Revalidatie (CIR), De Hoogstraat Rehabilitation Centre, Heliomare Rehabilitation Centre, Libra Rehabilitation & Audiology, Maasstad Hospital, MEREM Rehabilitation Centre, Revant Rehabilitation Centre, Sint Antonius Hospital, Sint Maartenskliniek, Tolbrug Rehabilitation Centre, University Medical Center Groningen, and University Medical Center Utrecht.

ORCID iDs

B.M.P. Mourits

E.W.M. Scholten

J.A. de Graaf

S. Oberink

R.J.E.M. Smeets

P. Stegeman

A.M. de Rooij

J. Stolwijk

J.M.A. Visser-Meily

M.W.M. Post

Ethical considerations

This study did not fall under the scope of the Dutch Medical Research Involving Human Subjects Act (WMO) and therefore did not require approval from an accredited Medical Ethics Committee (METC) in the Netherlands. However, an independent quality check was conducted at the University Medical Centre Utrecht to ensure compliance with legal and regulatory requirements, including informed consent procedures, data management, privacy and ethical considerations (20-01-2023, reference number 212217652).

The study was conducted in compliance with the Declaration of Helsinki.

Consent to participate

All participants and institutions provided written or online informed consent to participate in the study.

Consent for publication

Not applicable

Author contributions

ES, JA, JV and MP contributed to the study conception and design. Material preparation, data collection and analysis were performed by BM, ES, JA, SO, RS, PS, PP, AR and MR. The first draft of the manuscript was written by BM and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported and funded by the Netherlands Organisation for Health Research and Development (ZonMw) (project number 630000004).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability

The dataset used and analysed during the current study are available from the corresponding author on reasonable request via the public data repository DataverseNL (https://doi.org/10.34894/F5MIGX).

Supplemental material

Supplemental material for this article is available online.

References

Bandura

. Social cognitive theory of self-regulation. Organ Behav Hum Decis Process 1991; 50: 248–287.

Wade

. A general theory of rehabilitation: rehabilitation catalyses and assists adaptation to illness. Clin Rehabil 2024; 38: 429–442.

Allegrante

Wells

Peterson

. Interventions to support behavioral self-management of chronic diseases. Physiol Behav 2020; 176: 127–146.

Mol

Scholten

EWM

van Bennekom

CAM

, et al. Using self-regulation assessment to explore associations between self-regulation, participation and health-related quality of life in a rehabilitation population. J Rehabil Med 2023; 55: 1–8.

Howell

Molloy

Wilkinson

, et al. Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol 2015; 26: 1846–1858.

Foster

Croot

Brazier

, et al. The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews. J Patient-Reported Outcomes 2018; 2: 1–16.

Mol

van Bennekom

CAM

Scholten

EWM

, et al.

Self-regulation as rehabilitation outcome: what is important according to former patients?

Disabil Rehabil 2022; 44: 7484–7490.

Mol

van Bennekom

CAM

Scholten

EWM

, et al. Measures of self-regulation used in adult rehabilitation populations: a systematic review and content screening. Clin Rehabil 2022; 36: 1120–1138.

Mol

Van Bennekom

CAM

Scholten

EWM

, et al. The self-regulation assessment (SeRA) questionnaire: development and exploratory analyses of a new patient-reported outcome measure for rehabilitation. Disabil Rehabil 2023; 45: 2038–2045.

10.

de Vet

HCW

Terwee

Mokkink

, et al. Measurement in Medicine. First. Cambridge, England: Cambridge University Press, 2011.

11.

Mol

Scholten

van Bennekom

, et al. Development of the self-regulation assessment and content validation using cognitive interviews in a multicultural post-rehabilitation population. Front Rehabil Sci 2023; 4: 1–9.

12.

Terwee

Bot

SDM

de Boer

, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60: 34–42.

13.

Mokkink

Terwee

Knol

, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: A clarification of its content. BMC Med Res Methodol 2010; 10: 1–8.

14.

Mokkink

Terwee

de Vet

. Key concepts in clinical epidemiology: responsiveness, the longitudinal aspect of validity. J Clin Epidemiol 2021; 140: 159–162.

15.

Revicki

Cella

Hays

, et al. Responsiveness and minimal important differences for patient reported outcomes. Health Qual Life Outcomes 2006; 4: 1–5.

16.

Streiner

Norman

Cairney

. Health Measurement Scales–A Practical Guide to Their Development and Use. 5th ed. Oxford: Oxford University Press, 2015.

17.

Terwee

Roorda

. Dutch–Flemish translation of 17 item banks from the PROMIS.pdf. Qual Life Res 2014; 23: 1733–1741.

18.

Pilkonis

Choi

Reise

, et al. Item banks for measuring emotional distress from the patient-reported outcomes measurement information system (PROMIS®): depression, anxiety, and anger. Assessment 2011; 18: 263–283.

19.

PROMIS Anxiety_Scoring Manual. https://www.healthmeasures.net/images/PROMIS/manuals/Scoring_Manual_Only/PROMIS_Anxiety_Scoring_Manual_08Sept2023.pdf (2023, accessed 5 May 2025).

20.

PROMIS Depression_Scoring Manual. https://www.healthmeasures.net/images/PROMIS/manuals/Scoring_Manual_Only/PROMIS_Depression_Scoring_Manual_05Dec2023.pdf (2023, accessed 5 May 2025).

21.

Clover

Lambert

Oldmeadow

, et al. Convergent and criterion validity of PROMIS anxiety measures relative to six legacy measures and a structured diagnostic interview for anxiety in cancer patients. J Patient-Reported Outcomes 2022; 6: 1–11.

22.

Schalet

Pilkonis

, et al. Clinical validity of PROMIS depression, anxiety, and anger across diverse clinical samples. J Clin Epidemiol 2016; 73: 119–127.

23.

Lee

Driban

Price

, et al. Responsiveness and minimally important differences for 4 patient-reported outcomes measurement information system short forms: physical function, pain interference, depression, and anxiety in knee osteoarthritis. J Pain 2017; 18: 1096–1110.

24.

Oosterveer

van Meijeren-Pont

Arwert

, et al. The psychometric properties of the PROMIS® profile CAT in people with stroke. Top Stroke Rehabil 2024; 31: 738–744.

25.

Terwee

Crins

MHP

Boers

, et al. Validation of two PROMIS item banks for measuring social participation in the Dutch general population. Qual Life Res 2019; 28: 211–220.

26.

PROMIS. PROMIS Ability to Participate in Social Roles and Activities_Scoring Manual, https://www.healthmeasures.net/images/PROMIS/manuals/Scoring_Manual_Only/PROMIS_Ability_to_Participate_in_Social_Roles_and_Activities_Scoring_Manual_05Dec2023.pdf (2023, accessed 27 August 2025).

27.

Mourits

BMP

Scholten

EWM

de Graaf

, et al. Evaluation of participation in a former rehabilitation population: a comparison between USER-participation restriction and satisfaction subscales and PROMIS ability to participate and PROMIS satisfaction with participation 8-item short forms in a cross-se. Disabil Rehabil 2024; 47: 3742–3752.

28.

Jensen

Moinpour

Potosky

, et al. Responsiveness of 8 PROMIS® measures in a large, community-based cancer study cohort. Cancer 2017; 123: 327–335.

29.

Pecorelli

Guarneri

Vallorani

, et al. Validation of the PROMIS-29 questionnaire as a measure of recovery after pancreatic surgery. Ann Surg 2023; 278: 732–739.

30.

Rose

Bjorner

Becker

, et al. Evaluation of a preliminary physical function item bank supported the expected advantages of the patient-reported outcomes measurement information system (PROMIS). J Clin Epidemiol 2008; 61: 17–33.

31.

Hays

Spritzer

Amtmann

, et al. Upper-extremity and mobility subdomains from the patient-reported outcomes measurement information system (PROMIS) adult physical functioning item bank. Arch Phys Med Rehabil 2013; 94: 2291–2296.

32.

PROMIS. PROMIS Physical Functioning_Scoring Manual, https://www.healthmeasures.net/images/PROMIS/manuals/Scoring_Manual_Only/PROMIS_Physical_Function_User_Manual_and_Scoring_Instructions_12July2024.pdf (2024, accessed 5 May 2025).

33.

Crins

MHP

van der Wees

Klausch

, et al. Psychometric properties of the PROMIS physical function item bank in patients receiving physical therapy. PLoS One 2018; 13: 1–14.

34.

Deutscher

Kallen

Hayes

, et al. The stroke upper and lower extremity physical function measures were supported for score reliability, validity, and administration efficiency for patients poststroke. Phys Ther 2023; 103: 1–13.

35.

Castleberry

England

Fylstra

, et al. PROMIS-9 UE physical function demonstrates moderate responsiveness for patients following upper limb prosthesis intervention. J Patient-Reported Outcomes 2025; 9: 1–6.

36.

Group

. EuroQol - a new facility for the measurement of health-related quality of life. Health Policy (New York) 1990; 16: 199–208.

37.

Van Hout

Janssen

Feng

, et al. Interim scoring for the EQ-5D-5L: mapping the EQ-5D-5L to EQ-5D-3L value sets. Value Heal 2012; 15: 708–715.

38.

Chen

Lin

Liing

, et al. Validity, responsiveness, and minimal clinically important difference of EQ-5D-5L in stroke patients undergoing rehabilitation. Qual Life Res 2016; 25: 1585–1596.

39.

Feng

Kohlmann

Janssen

, et al. Psychometric properties of the EQ-5D-5L: a systematic review of the literature. Qual Life Res 2021; 30: 647–673.

40.

Soer

Reneman

Speijer

BLGN

, et al. Clinimetric properties of the EuroQol-5D in patients with chronic low back pain. Spine J 2012; 12: 1035–1039.

41.

Garratt

Engen

Kjeldberg

, et al. Use of EQ-5D-5L for assessing patient-reported outcomes in a national register for specialized rehabilitation. Arch Phys Med Rehabil 2024; 105: 40–48.

42.

Sharma

. A focus on reliability in developmental research through Cronbach’s Alpha among medical, dental and paramedical professionals. Asian Pacific J Heal Sci 2016; 3: 271–278.

43.

McHorney

Tarlov

. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 1995; 4: 293–307.

44.

Koo

. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155–163.

45.

Altman

Bland

. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 327: 307–310.

46.

Mokkink

Eekhout

Boers

, et al. Studies on reliability and measurement error of measurements in medicine – from design to statistics explained for medical researchers. Patient Relat Outcome Meas 2023; 14: 193–212.

47.

Cohen

. Statistical power analysis for behavioral science. Second Edi. New York: Lawrence Erlbaum Associates, 1988.

48.

Prinsen

CAC

Mokkink

Bouter

, et al. COSMIN Guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 2018; 27: 1147–1157.

49.

Revicki

Hays

Cella

, et al. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008; 61: 102–109.

50.

Jaeschke

Singer

Guyatt

. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials 1989; 10: 407–415.

51.

Deyo

Centor

. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis 1986; 39: 897–906.

52.

Terluin

Eekhout

Terwee

, et al. Minimal important change (MIC) based on a predictive modeling approach was more precise than MIC based on ROC analysis. J Clin Epidemiol 2015; 68: 1388–1396.

53.

Zeynep Enkavi

Eisenberg

Bissett

, et al. Large-scale analysis of test–retest reliabilities of self-regulation measures. Proc Natl Acad Sci U S A 2019; 116: 5472–5477.

54.

Walter

Eliasziw

Donner

. Sample size and optimal designs for reliability studies. Stat Med 1998; 17: 101–110.

55.

Abdel Shaheed

Mathieson

Wilson

, et al.

Who should judge treatment effects as unimportant?

J Physiother 2023; 69: 133–135.

56.

Terluin

Fromy

Trigg

, et al. Effect of present state bias on minimal important change estimates: a simulation study. Qual Life Res 2024; 33: 2963–2973.

57.

Zhang

Huang

. The anchor design of anchor-based method to determine the minimal clinically important difference: a systematic review. Health Qual Life Outcomes 2023; 21: 1–10.

58.

van Diemen

Tran

Stolwijk-Swuste

, et al. Trajectories of self-efficacy, depressed mood, and anxiety from admission to spinal cord injury rehabilitation to 1 year after discharge. Arch Phys Med Rehabil 2021; 102: 1939–1946.

59.

Norman

Stratford

Regehr

. Methodological problems in the retrospective computation of responsiveness to change: the lesson of Cronbach. J Clin Epidemiol 1997; 50: 869–879.

60.

Guyatt

Norman

Juniper

, et al. A critical look at transition ratings. J Clin Epidemiol 2002; 55: 900–908.

61.

Deyo

Diehr

Patrick

. Reproducibility and responsiveness of health status measures statistics and strategies for evaluation. Control Clin Trials 1991; 12: 142–158.

62.

Schmitt

Di Fabio

. The validity of prospective and retrospective global change criterion measures. Arch Phys Med Rehabil 2005; 86: 2270–2276.

63.

Schwartz

Sprangers

MAG

. Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Soc Sci Med 1999; 48: 1531–1548.

64.

Friedrich

Karoff

Hinz

. Response shift effects in patients’ assessments of their quality of life after cardiac rehabilitation. Qual Life Res 2019; 28: 2609–2620.

65.

Haagsma

Spronk

De Jongh

MAC

, et al. Conventional and retrospective change in health-related quality of life of trauma patients: an explorative observational follow-up study. Health Qual Life Outcomes 2020; 18: 1–13.

66.

Schwartz

Stucky

Rivers

, et al. Quality of life and adaptation in people with spinal cord injury: response shift effects from 1 to 5 years postinjury. Arch Phys Med Rehabil 2018; 99: 1599–1608.e1.

67.

Sébille

Lix

Ayilara

, et al. Critical examination of current response shift methods and proposal for advancing new methods. Qual Life Res 2021; 30: 3325–3342.

68.

Ortega-Gómez

Vicente-Galindo

Martín-Rodero

, et al. Detection of response shift in health-related quality of life studies: a systematic review. Health Qual Life Outcomes 2022; 20: 1–10.

69.

Craig

Nicholson Perry

Guest

, et al. Prospective study of the occurrence of psychological disorders and comorbidities after spinal cord injury. Arch Phys Med Rehabil 2015; 96: 1426–1434.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.19 MB