Instruments to measure outcomes of post-intensive care syndrome in outpatient care settings – Results of an expert consensus and feasibility field test

Abstract

Background

There is no consensus on the instruments for diagnosis of post-intensive care syndrome (PICS). We present a proposal for a set of outcome measurement instruments of PICS in outpatient care.

Methods

We conducted a three-round, semi-structured consensus-seeking process with medical experts, followed each by exploratory feasibility investigations with intensive care unit survivors (n₁ = 5; n₂ = 5; n₃ = 7). Fourteen participants from nine stakeholder groups participated in the first and second consensus meeting. In the third consensus meeting, a core group of six clinical researchers refined the final outcome measurement instrument set proposal.

Results

We suggest an outcome measurement instrument set used in a two-step process. First step: Screening with brief tests covering PICS domains of (1) mental health (Patient Health Questionnaire-4 (PHQ-4)), (2) cognition (MiniCog, Animal Naming), (3) physical function (Timed Up-and-Go (TUG), handgrip strength), and (4) health-related quality of life (HRQoL) (EQ-5D-5L). Single items measure subjective health before and after the intensive care unit stay. If patients report new or worsened health problems after intensive care unit discharge and show relevant impairment in at least one of the screening tests, a second extended assessment follows: (1) Mental health (Patient Health Questionnaire-8 (PHQ-8), Generalized Anxiety Disorder Scale-7 (GAD-7), Impact of Event Scale – revised (IES-R)); (2) cognition (Repeatable Battery for the Assessment of Neuropsychological Status (RBANS), Trail Making Test (TMT) A and B); (3) physical function (2-Minute Walk Test (2-MWT), handgrip strength, Short Physical Performance Battery (SPPB)); and (4) HRQoL (EQ-5D-5L, 12-Item WHO Disability Assessment Schedule (WHODAS 2.0)).

Conclusions

We propose an outcome measurement instrument set used in a two-step measurement of PICS, combining performance-based and patient-reported outcome measures. First-step screening is brief, free-of-charge, and easily applicable by health care professionals across different sectors. If indicated, specialized healthcare providers can perform the extended, second-step assessment. Usage of the first-step screening of our suggested outcome measurement instrument set in outpatient clinics with subsequent transfer to specialists is recommended for all intensive care unit survivors. This may increase awareness and reduce the burden of PICS.

Trial registration

This study was registered at ClinicalTrials.gov (Identifier: NCT04175236; first posted 22 November 2019).

Keywords

Post-intensive care syndrome critical care intensive care medicine survivors mental health cognition physical function intensive care unit

Background

Chances of survival after critical illness have dramatically increased in the last decades, generating a new cohort of intensive care unit (ICU) survivors who suffer from medium- and long-term impairments as a result of ICU care.^1–4 Research into these outcomes has become a major target of critical care, as ICU survivors commonly face problems in the three domains of mental health, cognition, and physical function.^1,3,5–10 Regarding mental health, ICU survivors frequently suffer from anxiety, post-traumatic stress disorder, and depression.^11–13 As to cognition, ICU survivors show impairments in major domains, including memory, attention, verbal fluency, and executive functioning.^14–17 ICU survivors are also at higher risk for dementia.¹⁸ Impairments in physical function include exercise limitations and limitations in activities of daily living.^19–21 Participants of a pioneering stakeholders' conference in 2010 suggested the umbrella term ‘post-intensive care syndrome’ (PICS) to cover persistent impairments in a variety of outcomes.⁶

There is heterogeneity and lack of consensus about appropriate, feasible outcome measures for PICS.^{8,9,11–14,16,17,19,22–25} A recent systematic review of Robinson et al. concluded that none of the measures of the PICS domains had an adequate evidence base for sufficient measurement quality in survivors of critical illness.² Nevertheless, they found promising psychometric results for the Hospital Anxiety and Depression Scale (HADS),²⁶ the Impact of Event Scale – revised (IES-R),²⁷ and the EQ-5D.^28–31 Needham et al. developed a set of outcome measurement instruments (OMIs) for survivors of acute respiratory failure, consisting of the HADS and IES-R for mental health, and the EQ-5D and/or the Short Form-36, version 2, for HRQoL, but did not achieve a consensus regarding measures of cognition or physical function.³²

Given the dynamic state of evolving PICS research, it seems premature to define PICS core outcome sets (COS) with sophisticated and rigorous methodological COS standards.^33–35 Rather, clinical studies should first establish potential OMIs for PICS as a basis for future COS development.^33–38 Further, OMIs should fulfill some basic requirement apart from general considerations of validity. Specifically, they should include patient reported as well as objective measurements, their application should be feasible in the vulnerable target population and they should be free of charge. In addition, questions of their implementation across different health care settings need to be explored. For example, Needham et al. discussed a stepwise approach for PICS diagnosis.⁶ Screenings might detect potential PICS patients in primary care settings, and an extended assessment by specialized clinicians could confirm screening results for therapy initiation.³⁹ In this work, we propose a set of OMIs to measure PICS in settings of outpatient healthcare service as well as a feasibility assessment of these OMIs.

Methods

General approach

We conducted a three-round, semi-structured consensus-seeking process, informed by the results of recent systematic reviews and consensus statements on outcomes in ICU survivors. After each consensus meeting, we conducted small pilot studies with samples of survivors from critical illness. The process is illustrated in Figure 1. All patients consented to participate in feasibility testings, and we complied to standards of the Declaration of Helsinki. This work was approved by the Institutional Review Board of the Medizinische Hochschule Brandenburg (record: E-01-20190619) and the Ethics Committee of Charité – Universitätsmedizin Berlin (record: EA4/122/19). The set of OMIs was developed in the context of the study ‘Enhanced Recovery after Intensive Care’ (ERIC) that investigates the impact of a telemedical critical-care intervention on quality of care and functional outcomes (CTN-identifier: NCT03671447). Within ERIC, functional outcomes are assessed using the OMI set proposed in the work at hand.

Figure 1.

Illustration of the three-round, semi-structured consensus-seeking process.

Expert participants

Members of the interdisciplinary, semi-structured consensus meetings comprised nine health care professionals from critical care and anesthesiology, including nursing staff and physicians, a respiratory therapist, a psychotherapist, a sepsis researcher, and a physical therapist. Further, two researchers on health care management, two telehealth researchers, and a clinical psychologist with a focus on outcome and process research participated in the consensus meetings. Half of the participating experts were women. The decision for the final OMI set was made by a core group of six clinical researchers (CDS, BW, CD, JKi, HK and KSt).

Consensus meetings

In preparation of the consensus meetings, two senior physicians (BW, CH) and two senior psychologists (CD, HK) reviewed recent evidence on outcomes in ICU survivors.^{2,6,8,11–14,32,39} Based on this literature, a preliminary OMI set was chosen and discussed in the consensus meetings. Predefined selection criteria were: (1) free-of-charge, non-commercial application; (2) time for completion of the set not exceeding 20–30 min in total; (3) potentially administrable by a variety of clinical practitioners; (4) validated measurement properties in clinical patient populations, including adults of all age groups; application in and psychometric data from ICU survivorship research were considered beneficial but not necessary; (5) existence of a validated German version.

Feasibility test and data analysis

After each consensus conference, a convenience sample of survivors from critical illness was chosen to assess feasibility and to qualitatively analyze the experience of survivors and examiners. Patients suffered from critical illness with main diagnoses from surgery, neurology, or internal medicine. They were either treated at Charité – Universitätsmedizin Berlin or Brandenburg Clinic in Bad Belzig. Feasibility testing was performed with five patients after the first consensus conference, five patients after the second consensus conference, and seven patients after the third consensus conference. After each feasibility testing, interviewers and patients reported on their experiences during the assessment with emphasis on completion time, strains the tests imposed on patients, and subjective operability of the set. OMI results were evaluated according to test manuals and cut-off values, and descriptive statistics (median, range, rate of patients exceeding cut-offs) were calculated. Upon completion of the final feasibility testing, four experts (BW, NP, HK, CD) subjectively rated the feasibility of each OMI set based on practicability, applicability, and ability to identify PICS on a scale from 0% (not feasible) to 100% (perfectly feasible).

Results

Consensus meeting outcomes: Preliminary Set 1

The preliminary OMI set included Patient Health Questionnaire-9 (PHQ-9),^40,41 Generalized Anxiety Disorder Scale-7 (GAD-7),⁴⁰ Impact of Event Scale (IES),⁴² MiniCog,^43,44 Animal Naming,⁴⁵ Timed Up-and-Go (TUG),⁴⁶ and EQ-5D-3L.^47,48 Two single items assessed current subjective physical and mental health concerns in terms of current distressing problems that are frequent in ICU survivors. Four visual analogue scales (VAS) were presented as health thermometers and measured self-reported subjective physical and mental health on a scale from 0 to 100, each both prior to ICU treatment and in the week prior to the assessment.^49–53

Feasibility testing: Preliminary Set 1

In the first feasibility testing, the preliminary OMI set was piloted with five survivors of critical illness. Three of those patients did not consent to publishing their results and one did not complete the test set; therefore, they were excluded from analysis. PHQ-9, GAD-7, MiniCog, Animal Naming, and EQ-5D-3L were easily completed. For the TUG, the patient was unable to walk. The IES as well as the VAS health thermometers were difficult to understand for both patient and health care professional who administered the tests and questionnaires. Completion of the assessment took approximately 40 min.

Consensus meeting outcomes: Preliminary Set 2

Based on results of the first consensus meeting and feasibility testing, the IES was dropped, and the VAS thermometers were replaced by numerical rating scales (NRS) from 0 to 10.

Feasibility testing: Preliminary Set 2

In the second feasibility testing, the revised OMI set was piloted with five survivors of critical illness. One patient did not consent to publishing results and was therefore excluded from analysis. Instruments were completed by patients without any problems. Overall, the assessment took between 30 and 40 min per participant and was deemed too long and extensive by both patients and examining clinical health professionals.

Consensus meeting outcomes: Final set

The core group agreed that the OMI set was still too time-consuming to be used in outpatient, primary health care settings. In addition, the outcome measures were considered not granular enough to differentiate between varying intensities of PICS. Furthermore, there were reliable and valid updates of the EQ-5D^47,48 and IES,⁶⁵ as well as ultra-short versions of the PHQ and the GAD.^40,41 Finally, handgrip strength, as measured with hand-held dynamometry, was considered a valid indicator of muscle strength and physical health. Taking together the abovementioned reasons, members of the expert group decided to divide the OMI set in two steps, with a short screening and an extended assessment. Patients showing new or worsening health problems after ICU and exceeding at least one of the cut-off points in the screening tests participate in a second, more comprehensive PICS assessment. Figure 2 and Table 2 show the questionnaires, tests, and cut-off points of both steps of the final OMI set.

Figure 2.

Proposal of a set of outcome measurement instruments (OMIs) of post-intensive care syndrome (PICS).

Screening

The screening consists of a combination of brief tests covering PICS domains. After a short instruction, they can be administered by a variety of health care professionals:

Mental health: Patient Health Questionnaire-4 (PHQ-4).^40,41

Cognition: MiniCog,^43,44 Animal Naming.⁴⁵

Physical function: Timed Up-and-Go (TUG),⁴⁶ handgrip strength.⁶⁶

Health-related quality of life: EQ-5D-5L.^47,48

New or worsening health problems: Two single items assess whether a patient currently has subjective mental, cognitive, or physical health concerns. Comparing the NRS ratings of subjective physical and mental health indicates whether current health impairments are new problems or worsening of conditions that have already existed prior to ICU admission. The screening takes about 20 min per patient.

Extended assessment

The extended assessment includes more elaborate and comprehensive OMIs and should be administered by experienced staff familiar with PICS patients. The standardized questionnaires and tests are:

Mental health: Patient Health Questionnaire-8 (PHQ-8),^40,54 Generalized Anxiety Disorder Scale-7 (GAD-7),⁴⁰ Impact of Event Scale – revised (IES-R)⁶⁵

Cognition: Repeatable Battery for the Assessment of Neuropsychological Status (RBANS),^62,67 Trail Making Test (TMT) A and B⁶⁸

Physical function: 2-Minute Walk Test (2-MWT),⁵⁷ handgrip strength,⁶⁶ Short Physical Performance Battery (SPPB)⁶⁹

Health-related quality of life: EQ-5D-5L,^47,48 12-Item WHO Disability Assessment Schedule (WHODAS 2.0); versions for self-rating and proxy-rating exist for measuring both patient and caretaker evaluation, respectively.^70,71

New or worsening health problems: The interviewer can assess the single items and NRS of subjective mental and physical health for a second time to include the resulting scores in a reassessment of the extent of patients' subjective health concerns. The extended assessment takes between 85 and 110 min per patient.

Feasibility testing: Final set

In the third feasibility testing, the screening and the more challenging extended assessment was piloted with seven survivors of critical illness. Two patients did not complete the testing and one did not attend the appointment. Four patients completed the extended assessment of the final OMI set. Patients were able to complete the instruments without any trouble, except one patient who was unable to walk and therefore could not conduct the TUG, the 2-MWT, and the SPPB. Table 1 displays detailed results. In the subjective feasibility rating of each OMI set by four experts (two medical doctors (BW, NP) and two senior psychologists (CD, HK)) on a scale from 0% (not feasible) to 100% (perfectly feasible), median feasibility scores increased from initially 60% for OMI set 1 to 97.5% for the final OMI set (Figure 3).

Figure 3.

Expert feasibility rating of OMI sets.

Table 1.

Results of nine pilot patients who completed the different versions of the OMI sets.

	Preliminary OMI sets 1 and 2, n = 5			Final screening and extended OMI set, n = 4
	Clinical characteristics
	Md, n	Range, %	Cut-off^a	Md, n	Range, %	Cut-off^a
Age (years)	77	53–82	–	45.5	23–56	–
Men	3/5	60%	–	2/4	50%	–
Main diagnosis of critical illness	Internal medicine (n = 4); surgery (n = 1)			Neurology (n = 2); surgery (n = 2)
	Subjective health
	Md, n	Range, %	Cut-off^a	Md, n	Range, %	Cut-off^a
Number of current subjective physical health problems^b	2	0–7	At least 1: 4/5	2	0–4	At least 1: 3/4
Subjective physical health before ICU^c	7.0	5.0–9.7		6.5	2.0–10.0
Subjective physical in the last week^c	4.0	3.0–7.0		7.0	4.0–9.0
Worse subjective physical health after ICU^d	3/5	60%		2/4	50%
Number of current subjective mental health problems^e	2	0–10	At least 1: 4/5	1	1–4	At least 1: 4/4
Subjective mental health before ICU^c	5.0	2.0–7.0		6.5	0.0–10.0
Subjective mental health in the last week^c	5.2	3.0–8.0		8.0	3.0–10.0
Worse subjective mental health after ICU^d	2/5	40%		3/4	75%
	Mental health
	Md	Range	Cut-off^a	Md	Range	Cut-off^a
PHQ-4, cut-off: ≥6^40,41 f	–	–	–	3.0	0–6	1/4
PHQ-2, cut-off: ≥3^f	–	–	–	1.5	0–3	1/4
GAD-2, cut-off: ≥3^f	–	–	–	1.5	0–3	1/4
PHQ-9, cut-off: >9^40,54 f	10.0	3.0–14.0	4/5	–	–	–
PHQ-8, cut-off: >9^f	10.0	3.0–14.0	4/5	5.4	3–10	1/4
GAD-7, cut-off: >9^40 f	10.0	4.7–10.0	3/5	4.5	2–7	0/4
IES, cut-off: >29; IES-R, cut-off: >1.4^27,55 f,g	38.0	–	1	0.96	0.45–1.36	0/4
	Cognition
	Md	Range	Cut-off^a	Md	Range	Cut-off^a
MiniCog, cut-off: ≤2^44 h	2	0–5	3/5	5	3–5	0/4
Animal naming, cut-off: <14^45 h	20	16–29	0/5	25	16–28	0/4
TMT A, s, cut-off: Percentile ≤PR 10^56 f	–	–	–	32.0	22.0–66.3	1/4
TMT B, s, cut-off: Percentile ≤PR 10^56 f	–	–	–	69.2	37.5–150.3	1/4
RBANS	–	–	–
Total score ⁱ				103.0	52–116	1/4
Index scores ⁱ
Immediate memory				123.0	49.0–129.0	1/4
Visuospatial/constructional				76.5	62.0–126.0	2/4
Language				106.0	84.0–113.0	1/4
Attention				84.5	60.0–118.0	2/4
Delayed memory				97.0	48.0–124.0	1/4
	Physical health
	Md	Range	Cut-off ^a	Md	Range	Cut-off ^a
TUG, s^f,j	15.0	10.0–19.0	2/5	7.6	6.1–10.44	2/4
Handgrip strength, kg^h,k	–	–	–	33.0	18.0–46.00	0/4
2-MWT, m, cut-off: <80 m (87 yards or 262 ft.)^57,58 h,l	–	–	–	176.0	90.0–184.0	1/4
SPPB, cut-off: <10^59 h,m	–	–	–	10.0	9.0–12.0	2/4
	Health-related quality of life
	Md, n	Range, %		Md, n	Range, %
EQ-5D-3L descriptive systemⁿ
At least some/moderate problems
Mobility	4/5	80		1/4	25
Self-care	1/5	20		0/4	0
Usual activities	4/5	80		0/4	0
Pain/discomfort	4/5	80		2/4	50
Anxiety/depression	5/5	100		0/4	0
EQ-5D subjective health, 0 to 100, higher scores indicate better HRQoL^47,48	50.0	11.0–70.0		75.0	35.0–95.0
WHODAS 2.0, total mean score^f	–	–		75.0	35.0–95
	PICS
	n	%		n	%
PICS according to self-report^o	3/5	60		2/4	50
Subjective health concern^o
At least one physical or mental health problem	5/5	100		4/4	100
Worse physical or mental health after ICU	3/5	60		3/4	75
Outcome scores indicating clinically sign. impairment in at least one PICS domain	5/5	100		2/4	50
Impairments in PICS domains^o
Physical health	2/5	40		2/4	50
Mental health	4/5	80		2/4	50
Cognition	3/5	60		1/4	25
Impairments in PICS domains^o
Only one domain	1/5	20		0/4	0
Two domains	4/5	80		1/4	25
Three domains	0/5	0		1/4	25

EQ-5D: European Quality of Life 5 Dimensions; GAD: Generalized Anxiety Disorder Scale; HRQoL: Health-related Quality of Life; ICU: Intensive Care Unit; IES: Impact of Event Scale; IES-R: Impact of Event Scale – revised; Md: Median; 2-MWT: 2-Minute Walk Test; OMI: Outcome Measurement Instrument; PICS: Post-intensive Care Syndrome; PHQ: Patient Health Questionnaire; RBANS: Repeatable Battery for the Assessment of Neuropsychological Status; SPPB: Short Physical Performance Battery; TMT: Trail Making Test; TUG: Timed Up-and-Go; WHODAS: WHO Disability Assessment Schedule.

Rate of patients exceeding cut-off.

OMI set 1: 7 problems; OMI set 2 and final OMI set: 8 problems.

OMI set 1: VAS rating (0–100) divided by 10.

Subtracting current VAS/NRS rating from VAS/NRS rating of time before ICU.

OMI set 1: 11 problems; OMI set 2 and final OMI set: 12 problems.

Higher scores indicate higher impairment.

Cut-offs for the IES reach from 19 to 35.^60,61 One patient of OMI sets 1 and 2 completed IES.

Lower scores indicate higher impairment.

Age-based standard scores (M = 100; SD = 15); cut-off: total/index score ≤85.⁶²

Cut-offs: ≤60 years: >10 s, 61–79 years: >15 s, ≥80 years: >20 s⁶³; n = 4 (OMI sets 1 and 2) and n = 3 (final OMI set) for Md, range; patients unable to walk considered as exceeding cut-off.

Cut-offs: men: ≤60 years: <25 kg; 61–79 years: <23 kg; ≥ 80 years: <19 kg; women: ≤60 years: <15 kg; 61–79 years: <14 kg; ≥80 years: <13 kg.^63,64

n = 3 (final OMI set) for Md, range; patient unable to walk considered as exceeding cut-off.

n = 3 (final OMI set) for Md, range; patient too weak for SPPB considered as exceeding cut-off.

Per subdomain: EQ-5D-3L: No; some/moderate; extreme problems. EQ-5D-5L: No; slight; moderate; severe; extreme problems.

PICS: (1) Subjective health concern: At least one subjective physical or mental health problem; (2) Worse subjective mental and/or physical health after ICU; (3) Scores indicating significant impairment in at least one of the standardized screening measures (mental health, cognition, physical health).

Table 2.

A two-step set of outcome measurement instruments (OMIs) of post-intensive care syndrome (PICS).

STEP 1: PICS screening
OMI	Description	Cut-off score
Patient Health Questionnaire-4 (PHQ-4)^40,41	PICS domain: Mental health Outcomes/subdomains: Depression, anxiety Instrument: Ultra-brief, self-report screening tool to measure frequency of symptoms of depression and anxiety; subscales for depression (PHQ-2) and anxiety (GAD-2), total scale score PHQ-4 for general mental distress; time frame: past 14 days; four items, 4-point scale from 0 to 3; for PHQ-2 and GAD-2 two items each, ranges from 0 to 6; total score from 0 to 12. Higher scores indicate worse mental health.	PHQ-2 sum score: ≥3 GAD-2 sum score: ≥3 PHQ-4 sum score: ≥6^40,41 Higher scores indicate higher impairment
MiniCog^43,44	PICS domain: Cognition Outcomes/subdomains: Memory, visuospatial, and visuoconstructional skills Instrument: Brief cognitive screening test; two tasks: (1) three-item recall task, scoring from 0 to 3, one point for each correctly recalled item; (2) clock drawing task, 0 point for incorrectly drawn clock, 2 points for correct clock. Sum score from 0 to 5. Higher scores indicate better cognitive functioning.	MiniCog sum score: ≤2⁴⁴ Lower scores indicate higher impairment
Animal Naming⁴⁵	PICS domain: Cognition Outcome/subdomain: Verbal fluency Instrument: Brief cognitive screening test; one task: Naming as many animals as possible in 60 s. One point for each correct animal name. Higher scores indicate better cognitive functioning.	Number of correct animal names <14⁴⁵ Lower scores indicate higher impairment
Timed Up-and-Go (TUG)⁴⁶	PICS domain: Physical function Outcome/subdomain: Mobility Instrument: Brief mobility test; one task: Subject is sitting on a chair with armrests, with the back on the backrest. When the investigator says ‘Go', the subject stands up, walks 3 m in a straight line, turns around, walks back to the chair again, and sits down with the back at the backrest. The time to complete the task is measured in seconds.	Age ≤60: >9 s Age 61–79: >14 s Age ≥80: >19 s ⁶³ Higher scores indicate higher impairment
Handgrip Strength⁶⁶ ^a	PICS domain: Physical function Outcome/subdomain: Muscle strength Instrument: Hand-held dynamometry; one task: Subject is sitting on a chair, the arm of the dominant hand at right angles. Subject holds hand dynamometer in a vertical position and squeezes it with maximum strength without changing position of arm and hand; three trials; standard assessment in geriatric care, can also be performed while lying in bed.	Men: Age ≤60: <25 kg Age 61–79: <23 kg Age ≥80: <19 kg Women: Age ≤60: <15 kg Age 61–79: <14 kg Age ≥80: <13 kg^63,64 Lower scores indicate higher impairment
EQ-5D-5L^47,48 ^a	PICS domain: Health-related quality of life (HRQoL) Outcomes/subdomains: Mobility, self-care, usual activities, pain/discomfort, anxiety/depression, general subjective health Instrument: Brief self-report questionnaire of generic HRQoL; six items; time frame: today. EQ-5D-5L descriptive system has 5 items (1 item per subdomain), with 5-point scales from 1 to 5; generic subjective health rating with 1 additional single item, in form of a thermometer scale from 0 to 100; single index value for general health state can be calculated from the five 5-point items. EQ-5D descriptive system: Higher scores indicate worse HRQoL; EQ-5D subjective health: Higher scores indicate better HRQoL.	Cut-off score not available EQ-5D descriptive system: Higher scores indicate higher impairment EQ-5D subjective health rating: Lower scores indicate higher impairment


Step 2: PICS extended assessment
OMI	Description	Cut-off score
Patient Health Questionnaire-8 (PHQ-8)^40,54	PICS domain: Mental Health Outcome/subdomain: Depression Instrument: Self-report questionnaire to measure frequency of depressive symptoms; time frame: past 14 days; 8 items, 4-point scale from 0 to 3; sum score from 0 to 24. Higher scores indicate worse depression.	PHQ-8 sum score: >9^40,54 Higher scores indicate higher impairment
Generalized Anxiety Disorder Scale-7 (GAD-7)⁴⁰	PICS domain: Mental Health Outcome/subdomain: Generalized anxiety Instrument: Self-report questionnaire to measure frequency of anxiety symptoms; time frame: past 14 days; seven items, 4-point scale from 0 to 3; sum score from 0 to 21. Higher scores indicate worse anxiety.	GAD-7 sum score: >9⁴⁰ Higher scores indicate higher impairment
Impact of Event Scale – revised (IES-R)⁶⁵	PICS domain: Mental Health Outcome/subdomain: Symptoms of post-traumatic stress disorder (PTSD) Instrument: Self-report questionnaire measuring severity of PTSD symptoms; subscales for avoidance, intrusion, and hyperarousal; time frame: past 7 days; 22 items, 5-point scale from 0 to 4; for avoidance and intrusion each 8 items, for hyperarousal 6 items, with mean total and subscale scores ranging from 0 to 4. Higher scores indicate worse PTSD symptoms.	Community sample: IES-R mean total score: >1.4⁵⁵ Sample of ICU survivors: IES-R mean total score: >1.5²⁷ Higher scores indicate higher impairment
Repeatable Battery for the Assessment of Neuropsychological Status (RBANS)^62,67	PICS domain: Cognition Outcomes/subdomains: Immediate memory, visuospatial/visuoconstructional skills, language, attention, delayed memory Instrument: Brief standardized test battery measuring cognitive functioning; equivalent alternate forms exist for retesting to measure cognitive change over time; 12 subtests: List Learning, story memory, figure copy, line orientation, picture naming, semantic fluency, digit span, coding, list recall, list recognition, story recall, figure recall. Total score and index scores for the five cognitive subdomains. Higher scores indicate better cognitive functioning.	Age-based standard scores (M = 100; SD = 15)⁶² Lower scores indicate higher impairment
Trail Making Test (TMT) A and B⁶⁸	PICS domain: Cognition Outcomes/subdomains: Processing speed (TMT A) and working memory/executive functioning (TMT B). Instrument: Test of scanning, visuomotor tracking, divided attention, and cognitive flexibility; two tasks: (1) TMT A, draw lines to connect numbered circles; (2) TMT B, draw lines to connect alternately numbered and lettered circles. Scoring based on time to complete the task. Higher scores indicate worse cognitive functioning.	Cut-off score not available, but country-specific normative values exist^{e.g., 56} Higher scores indicate higher impairment
2-Minute Walk Test (2-MWT)⁵⁷	PICS domain: Physical function Outcomes/subdomains: Walking ability, endurance Instrument: 1 task: Subject walks for two minutes along a marked route a distance as long as possible.	Walking distance: <80 m (87 yards or 262 ft.)^57,58 Lower scores indicate higher impairment
Short Physical Performance Battery (SPPB)^59,69	PICS domain: Physical function Outcome/subdomain: Lower extremity physical performance Instrument: Three timed tasks: (1) triad of standing balance tests, (2) walking speed, (3) sit-to-stands from a chair. Each task is rated with 0 to 4 points, resulting in a total score of 0 to 12.	SPPB total score <10 points is associated with increased risk of mortality.⁵⁹ Lower scores indicate higher impairment
WHO Disability Assessment Schedule (WHODAS 2.0)^70,71	PICS domain: Health-related quality of life (HRQoL) Outcome/subdomain: Functioning and disability due to health conditions Instrument: Self-report questionnaire of functioning and disability concerning understanding and communication, getting around, self-care, getting along with people, life activities, and participation in society; time frame: past 30 days. 12-item version or 36-item version; 5-point scale from 0 to 4; total mean score from 0 to 4. Higher scores indicate worse functioning and disability.	Cut-off score not available. Higher scores indicating higher impairment

Handgrip strength and EQ-5D-5L and should be administered within both screening and extended assessment.

Discussion

Our suggested two-step OMI set presents a combination of performance-based and patient-reported outcome measures for PICS. The standardized screening tools of the first measurement step are brief, applicable by clinical practitioners in outpatient healthcare, and free-of-charge. The instruments are commonly used in clinical research and have validated measurement properties in patient populations. While the screening tests and questionnaires fulfill our defined selection criteria, the instruments of the extended assessment demand staff familiar with PICS patients and more time for completion.

Feasibility and measurement properties of the proposed OMIs

The different scales of the PHQ have proven psychometric quality, and they are frequently used as mental health outcome measures in clinical research.^40,41,54,72 Previous or current ICU survivorship research has applied several of the PHQ scales.^8,13,73–76 While the ultra-short depression and anxiety screening tool PHQ-4 is part of the proposed PICS screening, we suggest the 7-item Generalized Anxiety Disorder Scale (GAD-7) and the 8-item depression scale (PHQ-8) for the extended assessment. The measurement quality of the PHQ-8 is equivalent to that of the PHQ-9, which has an additional item for thoughts of death and self-harm.^40,54 Since this item has not shown to be an accurate screen for suicide ideation,⁷⁷ we propose using the PHQ-8.

For the cognitive part of the screening process, we suggest the MiniCog and the verbal fluency test Animal Naming. Both tests are established and psychometrically sound screens for dementia.^{43–45,78–81} The MiniCog has shown feasibility in the context of preoperative frailty assessment in the elderly⁸² and was used in two critical care studies.^83,84

Remaining mental health and cognition tests of both screening and extended assessment are well validated in general clinical research and commonly used in ICU survivorship studies. There are investigations of measurement quality in ICU survivors for the IES-R^{2,8,12,23,32,39} and the EQ-5D,^2,8,32,39 which should be used in its current revision EQ-5D-5L.^47,48 The psychometric properties of the scales of the WHODAS 2.0 have been investigated in the general population and diverse clinical settings,^70,71,85 and results suggest feasibility in critical care settings.^86–88 Pivotal ICU survivorship research has applied the RBANS as well as TMT A and B. Unfortunately, to our knowledge, there are no studies elucidating measurement quality of these tools in ICU survivors.^{9,15,75,89,90} Users should also consider the licensing fees of the RBANS.

The single items and NRS of subjective health play a pivotal role to answer if health problems have occurred or worsened after the ICU treatment. Single items, VAS and NRS are frequently used in clinical research and practice, and sufficient measurement properties have been reported for the measurement of pain and general subjective health.^49–53 Interestingly, previous data suggest that NRS have better compliance and usability than VAS and verbal rating scales.^91,92 There is also established validity of the EQ-5D, which includes a blend of VAS and NRS for the rating of subjective health.^47,48 However, the psychometric properties of the single items and NRS for measuring aspects of PICS-related subjective health still have to be investigated in ICU survivors.

For the screening of physical function, we suggest the TUG⁴⁶ and handgrip strength dynamometry.⁶⁶ The TUG was developed for and used on geriatric patients.⁹³ Recently, it has also been used in a perioperative setting as a measure of physical decline and recovery.⁹⁴ Handgrip strength is one of the basic methods for measuring muscle strength,⁹⁵ with normative values for all age groups.^63,64 Several studies have assessed both TUG^96–98 and handgrip strength dynamometry in ICU survivors.^98–103 Some of these investigations also reported promising data on measurement quality of both tests in ICU survivors.^96,100–103

For the extended assessment of the physical function, we recommend handgrip strength and two additional physical tests. Handgrip strength, EQ-5D-5L, the NRS, and single items regarding new and worsening health problems are already part of the screening, but a repetition in synopsis with the additional tests allows for a reassessment of the initial impairment. The 2-MWT was developed as a short version of the 6-Minute Walk Test (6-MWT) to account for primarily geriatric patients who are unable to walk for 6 min.⁵⁷ Bohannon et al. provide normative values based on a large population-based sample for all age groups above 18 years, but the authors do not report information on health status of the sample.¹⁰⁴ Two studies independently found a mean distance of around 80 m in participants of geriatric inpatient rehabilitation centers⁵⁷ and patients with COPD,⁵⁸ similar to findings from Connelly et al.¹⁰⁵ In comparison, Chan and Pin reported a mean distance of 75 m in frail elderly suffering from dementia without using walking aids, whereas shorter distances were observed in participants using walking aids.¹⁰⁶ Thus, we conclude that a cut-off of 80 m is feasible for ICU survivors. While the 2-MWT has, to our knowledge, not yet been investigated in ICU survivors, the 6-MWT has been applied in several ICU survivor studies.^{19,76,96,97,99,102,103} The SPPB consists of three tests on standing balance, walking speed, and standing up from a chair, with a sum score ranging from 0 to 12 points. In a recent meta-analysis, scores below 10 were associated with an increased risk for mortality in elderly.⁵⁹ In non-geriatric settings, the SPPB was used in cancer survivors, where it was predictive of survival- and treatment-related complications.¹⁰⁷ Recent ICU survivor studies utilizing the SPPB report favorable results concerning validity.^{100,101,108,109} Finally, our proposed OMIs for the physical function domain are also supported by feasibility of TUG,^110,111 handgrip strength,^111–113 6-MWT,¹¹⁰ and SPPB¹¹⁴ in patients of acute ICU settings, as well as currently running clinical trials in critical care patients with TUG,¹¹⁵ handgrip strength^115,116 and 2-MWT¹⁰⁵ as outcomes.

Implications for clinical research and practice

It is yet unknown which of the outcomes and measurement instruments of PICS domains will prove relevant, reliable, responsive, and valid in future projects to develop a COS for PICS.^2,8,32,39 Furthermore, variations in COS, OMIs, as well as definitions and quantification approaches might be appropriate, depending on diverse objectives, scopes, and settings. Different COS and OMIs may be necessary for observational studies versus clinical efficacy trials, and clinical research versus practice. Binary outcomes are needed to determine incidence rates, whereas measurement of change over time and treatment effects may be more successful with several specific continuous outcomes and/or elaborate composite scores.^33–38

Here, we present an OMI proposal that is suitable to determine incidence and extent of PICS in settings ranging from primary outpatient healthcare to health centers experienced in PICS. The suggested measures of PICS domains have well-established cut-off points in both the screening and the extended measurement, thereby making an operationalization and quantification of a binary PICS outcome possible. The results may define PICS of at least mild extent: (1) Subjective physical, mental and/or cognitive health concern; (2) lower current subjective mental and/or physical health, as compared with ratings prior to ICU treatment; (3) scores indicating clinically significant impairment in at least one of the screening measures. Yet, the characterization of different grades of PICS severity remains unknown.^7,9 In the suggested OMI set, severe PICS might be identified in the extended assessment if results indicate significant impairment in two or three PICS domains, or by substantial impairment in at least one of the more comprehensive PICS measures. In order to determine substantial impairment, more extreme cut-off points are needed, e.g. scores exceeding two standard deviations of a normative sample, as applied in the DSM-5 features of major neurocognitive disorder.¹¹⁷ The specific scores of EQ-5D-5L and WHODAS 2.0 may contribute to the evaluation of the severity of PICS by investigating to which extent physical, mental, and cognitive health outcomes are related to HRQoL outcomes. A structured approach for the diagnosis of PICS would allow for individually tailored treatments in the conspicuous domains. Tailored treatment concepts might include individual physical therapy, psychotherapy, memory training, or home nursing services.

Limitations and future directions

Although the number of participants in the consensus process was comparatively low, feedback was comprehensive and partially similar. Nevertheless, future studies should test the proposed OMI set on a larger number of patients and nourish further discussions. We covered only the first stages of an OMI set development project, and thereby established the foundation to apply rigorous methodological standards of COS research in subsequent endeavors.^33–38 The study has important strengths. The OMI set is a pragmatic set for daily clinical practice and can be applied across health sectors and in different medical fields, thus contributing to consistent and standardized patient care. For the final OMI set, we selected instruments that have shown robust evidence of measurement quality. Further, most of the selected instruments have demonstrated feasibility in samples of critical care survivors. Since all OMIs currently applied in ICU survivorship research lack elaborate data on measurement quality in this population,² further applications of OMIs in clinical PICS studies should contribute to research on reliability and validity.^118,119 Necessary data include repeated measurements and assessments of clinical outcomes to determine aspects of content, construct, and criterion validity, by using for example a detailed history and physical examination by clinicians, psychiatric interviews, or biological data.

Conclusions

To summarize our key message, we propose a two-step OMI set for PICS measurement, which combines performance-based and patient-reported outcome measures. The initial screening is brief, but assesses the relevant PICS domains cognition, mental health, physical function, and HRQoL. The subsequent extended assessment is more time consuming and utilizes more elaborate tests for an in-depth assessment of the PICS domains, allowing for tailored therapies. Our proposal fulfils essential criteria for the application in clinical settings ranging from primary outpatient healthcare to health centers experienced in PICS. Future studies should investigate the measurement properties of different OMI sets to identify PICS and examine ways to measure severity of PICS. Aside from the methodological issues, we are convinced that a PICS screening test in outpatient settings followed by a transfer to specialists would possibly increase awareness as well as accessibility and, thus, reduce the burden of PICS.

Footnotes

Acknowledgements

This article is endorsed by the working group ‘Intensive care and critical illness’ of the German Network Health Care Research (Deutsches Netzwerk Versorgungforschung e.V.). Further, the authors thank the ERIC study team for the excellent, perpetual support at all stages of this work. Particularly, Dr Christine Adrion and Professor Dr Ulrich Mansmann from the Institute for Medical Information Processing, Biometry, and Epidemiology of Ludwig-Maximilians-University Munich (LMU) have provided constant support.

Availability of data and materials

The datasets generated during the current study are available from the corresponding author on reasonable request.

Authors' contributions

CDS, BW, HK and CD designed the study. BW, CD, SR and NP collected the data. HK analyzed and interpreted the patient data. HK, BW and NP drafted the article. CDS, CD, JKi, SP, JKr, JJG, KSt, TG, KSc, SR, CH, JC, RB, UM, MG, SWC, and CA revised and commented on the article. All authors read and approved the final version of the article.

Consent for publication

All authors gave their consent for publication.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: CDS reports grants from Gemeinsamer Bundesausschuss (G-BA)/Federal Joint Committee, during the conduct of the study; grants from Aridis Pharmaceutical Inc., grants from B Braun Melsungen AG, grants from Drägerwerk AG & Co KGaA, grants from Grünenthal GmbH, grants from Infectopharm GmbH, grants from Sedana Medical Ltd, grants from Deutsche Forschungsgemeinschaft/German Research Society, grants from Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR)/German Aerospace Center, grants from Einstein Stiftung Berlin/Einstein Foundation Berlin, grants from the European Society of Anaesthesiology, grants from Gemeinsamer Bundesausschuss (G-BA)/Federal Joint Committee, grants from Inneruniversitäre Forschungsförderung/Inner University Grants, grants from Projektträger im DLR/Project Management Agency, grants from Stifterverband/Non-Profit Society Promoting Science and Education, grants from WHOCC, grants from Baxter Deutschland GmbH, grants from Biotest AG, grants from Cytosorbents Europe GmbH, grants from Edwards Lifesciences Germany GmbH, grants from Fresenius Medical Care, grants from Grünenthal GmbH, grants from Masimo Europe Ltd, grants from Medtronic GmbH, grants from Pfizer Pharma PFE GmbH, personal fees from Georg Thieme Verlag, grants from Dr. F. Köhler Chemie GmbH, grants from Sintetica GmbH, grants from the European Commission, grants from Stifterverband für die Deutsche Wissenschaft e.V./Philips, grants from Stiftung Charité, outside the submitted work; in addition, CDS has a patent 10 2014 215 211.9 pending, a patent Application No. PCT/EP2015/067730 pending to Graft Gesellschaft von Architekten mbH, and a patent Application No. PCT/EP2015/067731 pending to Graft Gesellschaft von Architekten mbH. HK, NP, TG, SR, JJG, KSc, KSt, JC, JKr, JKi, CD, SP, CA, RB, UM, SWC and MG declare that there is no conflict of interest. CH reports grants to her institution from the German Federal Ministry of Education and Research (BMBF) via the Center for Sepsis Control and Care Jena, the German Innovations Funds/Federal Joint Committee, and the European Society of Intensive Care Medicine. BW reports personal fees from Orion Pharma Ltd. and personal fees from Dr. F. Köhler Chemie GmbH, outside the submitted work.

Ethics approval and consent to participate

This work was approved by the Institutional Review Board of the Medizinische Hochschule Brandenburg (record: E-01-20190619) and the Ethics Committee of Charité – Universitätsmedizin Berlin (record: EA4/122/19). Informed consent was obtained from all individual participants included in the study.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was composed in the context of a project supported by the Gemeinsamer Bundesausschuss (G-BA)/Federal Joint Committee of the Federal Republic of Germany (Grant-ID: 01NVF16011). The funders had no role in in the design of the study as well as collection, analysis, and interpretation of data and in writing the article.

ORCID iDs

Claudia D Spies

Nicolas Paul

Sophie K Piper

References

Elliott

Davidson

Harvey

, et al. Exploring the scope of post-intensive care syndrome therapy and care: engagement of non-critical care providers and survivors in a second stakeholders meeting. Crit Care Med 2014; 42: 2518–2526.

Robinson

Davis

Dinglas

, et al. A systematic review finds limited data on measurement properties of instruments measuring outcomes in adult intensive care unit survivors. J Clin Epidemiol 2017; 82: 37–46.

Gajic

Ahmad

Wilson

, et al.

Outcomes of critical illness: what is meaningful?

Curr Opin Crit Care 2018; 24: 394–400.

Prescott

Iwashyna

Blackwood

, et al. Understanding and enhancing sepsis survivorship. Priorities for research and practice. Am J Respir Crit Care Med 2019; 200: 972–981.

Iwashyna

. Survivorship will be the defining challenge of critical care in the 21st century. Ann Intern Med 2010; 153: 204–205.

Needham

Davidson

Cohen

, et al. Improving long-term outcomes after discharge from intensive care unit: report from a stakeholders' conference. Crit Care Med 2012; 40: 502–509.

McPeake

Mikkelsen

. The evolution of post intensive care syndrome. Crit Care Med 2018; 46: 1551–1552.

Turnbull

Rabiee

Davis

, et al. Outcome measurement in ICU survivorship research from 1970 to 2013: a scoping review of 425 publications. Crit Care Med 2016; 44: 1267–1277.

Marra

Pandharipande

Girard

, et al. Co-occurrence of post-intensive care syndrome problems among 406 survivors of critical illness. Crit Care Med 2018; 46: 1393–1401.

10.

Inoue

Hatakeyama

Kondo

, et al. Post-intensive care syndrome: its pathophysiology, prevention, and future directions. Acute Med Surg 2019; 6: 233–246.

11.

Nikayin

Rabiee

Hashem

, et al. Anxiety symptoms in survivors of critical illness: a systematic review and meta-analysis. Gen Hosp Psychiatry 2016; 43: 23–29.

12.

Parker

Sricharoenchai

Raparla

, et al. Posttraumatic stress disorder in critical illness survivors: a metaanalysis. Crit Care Med 2015; 43: 1121–1129.

13.

Rabiee

Nikayin

Hashem

, et al. Depressive symptoms after critical illness: a systematic review and meta-analysis. Crit Care Med 2016; 44: 1744–1753.

14.

Wolters

Slooter

van der Kooi

, et al. Cognitive impairment after intensive care unit admission: a systematic review. Intensive Care Med 2013; 39: 376–386.

15.

Pandharipande

Girard

Jackson

, et al. Long-term cognitive impairment after critical illness. N Engl J Med 2013; 369: 1306–1316.

16.

Rengel

Hayhurst

Pandharipande

, et al. Long-term cognitive and functional impairments after critical illness. Anesth Analg 2019; 128: 772–780.

17.

Sakusic

Rabinstein

. Cognitive outcomes after critical illness. Curr Opin Crit Care 2018; 24: 410–414.

18.

Guerra

Hua

Wunsch

. Risk of a diagnosis of dementia for elderly medicare beneficiaries after intensive care. Anesthesiology 2015; 123: 1105–1112.

19.

Herridge

Tansey

Matte

, et al. Functional disability 5 years after acute respiratory distress syndrome. N Engl J Med 2011; 364: 1293–1304.

20.

Iwashyna

Cooke

Wunsch

, et al. Population burden of long-term survivorship after severe sepsis in older Americans. J Am Geriatr Soc 2012; 60: 1070–1077.

21.

Jackson

Pandharipande

Girard

, et al. Depression, post-traumatic stress disorder, and functional disability in survivors of critical illness in the BRAIN-ICU study: a longitudinal cohort study. Lancet Respir Med 2014; 2: 369–379.

22.

Herridge

Cheung

Tansey

, et al. One-year outcomes in survivors of the acute respiratory distress syndrome. N Engl J Med 2003; 348: 683–693.

23.

Bienvenu

Friedman

Colantuoni

, et al. Psychiatric symptoms after acute respiratory distress syndrome: a 5-year longitudinal study. Intensive Care Med 2018; 44: 38–47.

24.

Haines

Berney

Warrillow

, et al. Long-term recovery following critical illness in an Australian cohort. J Intensive Care 2018; 6: 8.

25.

Schefold

Bierbrauer

Weber-Carstens

. Intensive care unit-acquired weakness (ICUAW) and muscle wasting in critically ill patients with severe sepsis and septic shock. J Cachexia Sarcopenia Muscle 2010; 1: 147–157.

26.

Sukantarat

Williamson

Brett

. Psychological assessment of ICU survivors: a comparison between the Hospital Anxiety and Depression scale and the Depression, Anxiety and Stress scale. Anaesthesia 2007; 62: 239–243.

27.

Bienvenu

Williams

Yang

, et al. Posttraumatic stress disorder in survivors of acute lung injury: evaluating the Impact of Event Scale-Revised. Chest 2013; 144: 24–31.

28.

Kaarlola

Pettila

Kekki

. Performance of two measures of general health-related quality of life, the EQ-5D and the RAND-36 among critically ill patients. Intensive Care Med 2004; 30: 2245–2252.

29.

Khoudri

Ali Zeggwagh

Abidi

, et al. Measurement properties of the short form 36 and health-related quality of life after intensive care in Morocco. Acta Anaesthesiol Scand 2007; 51: 189–197.

30.

Khoudri

Belayachi

Dendane

, et al. Measuring quality of life after intensive care using the Arabic version for Morocco of the EuroQol 5 dimensions. BMC Res Notes 2012; 5: 56.

31.

Vainiola

Pettilä

Roine

, et al. Comparison of two utility instruments, the EQ-5D and the 15D, in the critical care setting. Intensive Care Med 2010; 36: 2090–2093.

32.

Needham

Sepulveda

Dinglas

, et al. Core outcome measures for clinical research in acute respiratory failure survivors. An international modified Delphi consensus study. Am J Respir Crit Care Med 2017; 196: 1122–1130.

33.

Williamson

Altman

Bagley

, et al. The COMET handbook: version 1.0. Trials 2017; 18: 280.

34.

Gargon

Gurung

Medley

, et al. Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One 2014; 9: e99111.

35.

Kirkham

Davis

Altman

, et al. Core outcome set-STAndards for DEVELOPMENT: the COS-STAD recommendations. PLoS Med 2017; 14: e1002447.

36.

Kirkham

Gorst

Altman

, et al. Core outcome set-STAndards for reporting: the COS-STAR statement. PLoS Med 2016; 13: e1002148.

37.

Boers

Kirwan

Wells

, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol 2014; 67: 745–753.

38.

Prinsen

Vohra

Rose

, et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set” – a practical guideline. Trials 2016; 17: 449.

39.

Major

Kwakman

Kho

, et al. Surviving critical illness: what is next? An expert consensus statement on physical rehabilitation after hospital discharge. Crit Care 2016; 20: 354.

40.

Kroenke

Spitzer

Williams

, et al. The Patient Health Questionnaire somatic, anxiety, and depressive symptom scales: a systematic review. Gen Hosp Psychiatry 2010; 32: 345–359.

41.

Loewe

Wahl

Rose

, et al. A 4-item measure for depression and anxiety: validation and standardization of the Patient Health Questionnaire-4 (PHQ-4) in the general population. J Affect Disord 2010; 122: 86–95.

42.

Horowitz

Wilner

Alverez

. Impact of Event Scale: a measure of subjective stress. Psychosom Med 1979; 41: 209–218.

43.

Borson

Scanlan

Chen

, et al. The Mini-Cog as a screen for dementia: validation in a population-based sample. J Am Geriatr Soc 2003; 51: 1451–1454.

44.

Borson

Scanlan

Watanabe

, et al. Improving identification of cognitive impairment in primary care. Int J Geriatr Psychiatry 2006; 21: 349–355.

45.

Sager

Hermann

La Rue

, et al. Screening for dementia in community-based memory clinics. Wis Med J 2006; 105: 25–29.

46.

Podsiadlo

Richardson

. The timed “Up & Go”: a test of basic functional mobility for frail elderly persons. J Am Geriatr Soc 1991; 39: 142–148.

47.

Herdman

Gudex

Lloyd

, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011; 20: 1727–1736.

48.

Janssen

Pickard

Golicki

, et al. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study. Qual Life Res 2013; 22: 1717–1727.

49.

Bergner

Rothman

. Health status measures: an overview and guide for selection. Annu Rev Public Health 1987; 8: 191–210.

50.

Wewers

Lowe

. A critical review of visual analogue scales in the measurement of clinical phenomena. Res Nurs Health 1990; 13: 227–236.

51.

Ngamaba

Panagioti

Armitage

. How strongly related are health status and subjective well-being? Systematic review and meta-analysis. Eur J Public Health 2017; 27: 879–885.

52.

McDowell

. Measuring health: a guide to rating scales and questionnaires, 3rd ed. New York, NY, US: Oxford University Press, 2006.

53.

Beaton

Boers

Tugwell

Assessment of health outcomes. In: G

Firestein

Budd

Gabriel

, et al.(eds). Kelley & Firestein's textbook of rheumatology, 10th ed. Philadelphia, PA, US: Elsevier, 2017, pp. 496–508.

54.

Kroenke

Spitzer

. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr Ann 2002; 32: 1–7.

55.

Creamer

Bell

Failla

. Psychometric properties of the Impact of Event Scale – revised. Behav Res Ther 2003; 41: 1489–1496.

56.

Tombaugh

. Trail making test A and B: normative data stratified by age and education. Arch Clin Neuropsychol 2004; 19: 203–214.

57.

Brooks

Davis

Naglie

. Validity of 3 physical performance measures in inpatient geriatric rehabilitation. Arch Phys Med Rehabil 2006; 87: 105–110.

58.

Neo

, et al. Prediction of poor short-term prognosis and unmet needs in advanced chronic obstructive pulmonary disease: use of the two-minute walking distance extracted from a six-minute walk test. J Palliat Med 2017; 20: 821–828.

59.

Pavasini

Guralnik

Brown

, et al. Short Physical Performance Battery and all-cause mortality: systematic review and meta-analysis. BMC Med 2016; 14: 215.

60.

Wohlfarth

van den Brink

Winkel

, et al. Screening for posttraumatic stress disorder: an evaluation of two self-report scales among crime victims. Psychol Assess 2003; 15: 101–109.

61.

Scragg

Jones

Fauvel

. Psychological problems following ICU treatment. Anaesthesia 2001; 56: 9–14.

62.

Randolph

. Repeatable Battery for the Assessment of Neuropsychological Status Update (RBANS Update), San Antonio, TX, US: The Psychological Corporation, 2012.

63.

Kenny

Coen

Frewen

, et al. Normative values of cognitive and physical function in older adults: findings from the Irish Longitudinal Study on Ageing. J Am Geriatr Soc 2013; 61: S279–S290.

64.

Bohannon

Wang

Yen

, et al. Handgrip strength: a comparison of values obtained from the NHANES and NIH toolbox studies. Am J Occup Ther 2019; 73: 7302205080.

65.

Weiss

The Impact of Event Scale: Revised. In: J

Wilson

So-Kum Tang

(eds). International and cultural psychology cross-cultural assessment of psychological trauma and PTSD, New York, NY, US: Springer Science + Business Media, 2007, pp. 219–238.

66.

Roberts

Denison

Martin

, et al. A review of the measurement of grip strength in clinical and epidemiological studies: towards a standardised approach. Age Ageing 2011; 40: 423–429.

67.

Randolph

Tierney

Mohr

, et al. The Repeatable Battery for the Assessment of Neuropsychological Status (RBANS): preliminary clinical validity. J Clin Exp Neuropsychol 1998; 20: 310–319.

68.

Reitan

. Validity of the Trail Making Test as an indicator of organic brain damage. Percept Mot Skills 1958; 8: 271–276.

69.

Guralnik

Ferrucci

Simonsick

, et al. Lower-extremity function in persons over the age of 70 years as a predictor of subsequent disability. N Engl J Med 1995; 332: 556–561.

70.

Üstün

Kostanjsek

Chatterji

, et al. Measuring health and disability. Manual for WHO Disability Assessment Schedule WHODAS 2.0, Geneva: World Health Organization, 2010.

71.

Andrews

Kemp

Sunderland

, et al. Normative data for the 12 item WHO Disability Assessment Schedule 2.0. PLoS One 2009; 4: e8343.

72.

Kerper

Spies

Tillinger

, et al. Screening for depression, anxiety and general psychological distress in preoperative surgical patients: a psychometric analysis of the Patient Health Questionnaire 4 (PHQ-4). Clin Health Promot 2014; 4: 5–14.

73.

Wang

Mosher

Perkins

, et al. Post-intensive care unit psychiatric comorbidity and quality of life. J Hosp Med 2017; 12: 831–835.

74.

Wang

Mosher

Gao

, et al. Antidepressant use and depressive symptoms in intensive care unit survivors. J Hosp Med 2017; 12: 731–734.

75.

Wang

Hammes

Khan

, et al. Improving Recovery and Outcomes Every Day after the ICU (IMPROVE): study protocol for a randomized controlled trial. Trials 2018; 19: 196.

76.

van Aartsen

van Aswegen

. Changes in biopsychosocial outcomes for a mixed cohort of ICU survivors. S Afr J Physiother 2018; 74: 427.

77.

Razykov

Ziegelstein

Whooley

, et al. The PHQ-9 versus the PHQ-8 – is item 9 useful for assessing suicide risk in coronary artery disease patients? Data from the Heart and Soul Study. J Psychosom Res 2012; 73: 163–168.

78.

Scanlan

Borson

. The Mini-Cog: receiver operating characteristics with expert and naive raters. Int J Geriatr Psychiatry 2001; 16: 216–222.

79.

Monsch

Bondi

Butters

, et al. Comparisons of verbal fluency tasks in the detection of dementia of the Alzheimer type. Arch Neurol 1992; 49: 1253–1258.

80.

Canning

Leach

Stuss

, et al. Diagnostic utility of abbreviated fluency measures in Alzheimer disease and vascular dementia. Neurology 2004; 62: 556–562.

81.

Woodard

Dorsett

ESW

Cooper

, et al. Development of a brief cognitive screen for mild cognitive impairment and neurocognitive disorder. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn 2005; 12: 299–315.

82.

Amini

Crowley

Hizel

, et al. Feasibility and rationale for incorporating frailty and cognitive screening protocols in a preoperative anesthesia clinic. Anesth Analg 2019; 129: 830–838.

83.

Chung

Yoo

Park

, et al. Cognitive impairment and psychological distress at discharge from intensive care unit. Psychiatry Investig 2017; 14: 376–379.

84.

Woon

Dunn

Hopkins

. Predicting cognitive sequelae in survivors of critical illness with cognitive screening tests. Am J Respir Crit Care Med 2012; 186: 333–340.

85.

Kirchberger

Braitmayer

Coenen

, et al. Feasibility and psychometric properties of the German 12-item WHO Disability Assessment Schedule (WHODAS 2.0) in a population-based sample of patients with myocardial infarction from the MONICA/KORA myocardial infarction registry. Popul Health Metr 2014; 12: 27.

86.

Papadakaki

Tsalkanis

Sarris

, et al. Physical, psychological and economic burden of two-wheel users after a road traffic injury: evidence from intensive care units of three EU countries. J Saf Res 2018; 67: 155–163.

87.

Papadakaki

Ferraro

Orsi

, et al. Psychological distress and physical disability in patients sustaining severe injuries in road traffic crashes: results from a one-year cohort study from three European countries. Injury 2017; 48: 297–306.

88.

Abedzadeh-Kalahroudi

Razi

Sehat

, et al. Measurement of disability and its predictors among trauma patients: a follow-up study. Arch Trauma Res 2015; 4: e29393.

89.

Wood

Maslove

Muscedere

, et al. Robotic technology provides objective and quantifiable metrics of neurocognitive functioning in survivors of critical illness: a feasibility study. J Crit Care 2018; 48: 228–236.

90.

Mitchell

Shum

DHK

Mihala

, et al. Long-term cognitive impairment and delirium in intensive care: a prospective cohort study. Aust Crit Care 2018; 31: 204–211.

91.

Hjermstad

Fayers

Haugen

, et al. Studies comparing Numerical Rating Scales, Verbal Rating Scales, and Visual Analogue Scales for assessment of pain intensity in adults: a systematic literature review. J Pain Symptom Manage 2011; 41: 1073–1093.

92.

Karcioglu

Topacoglu

Dikme

, et al.

A systematic review of the pain scales in adults: which to use?

Am J Emerg Med 2018; 36: 707–714.

93.

Bassett

Siu

Honaker

. Functional measures for fall risk in the acute care setting: a review. West J Nurs Res 2018; 40: 1469–1488.

94.

Lachmann G, Morgeli R, Kuenz S, et al. Perioperatively acquired weakness. Anesth Analg 2020; 130: 341–351.

95.

Bohannon

. Is it legitimate to characterize muscle strength using a limited number of measures? J Strength Cond Res 2008; 22: 166–173.

96.

Denehy

Nordon-Craft

Edbrooke

, et al. Outcome measures report different aspects of patient function three months following critical care. Intensive Care Med 2014; 40: 1862–1869.

97.

Veldema

Bosl

Kugler

, et al. Cycle ergometer training vs resistance training in ICU-acquired weakness. Acta Neurol Scand 2019; 140: 62–71.

98.

Walsh

Salisbury

Merriweather

, et al. Increased hospital-based physical rehabilitation and information provision after intensive care unit discharge: the RECOVER Randomized Clinical Trial. JAMA Int Med 2015; 175: 901–910.

99.

Ohtake

Lee

Scott

, et al. Physical impairments associated with post-intensive care syndrome: systematic review based on the World Health Organization's International Classification of Functioning, Disability and Health Framework. Phys Ther 2018; 98: 631–645.

100.

Gardner

Ghita

Wang

, et al. The development of chronic critical illness determines physical function, quality of life, and long-term survival among early survivors of sepsis in surgical ICUs. Crit Care Med 2019; 47: 566–573.

101.

Custodero

Ghita

, et al. Prognostic value of NT-proBNP levels in the acute phase of sepsis on lower long-term physical function and muscle strength in sepsis survivors. Crit Care 2019; 23: 230.

102.

Borges

Carvalho

Colombo

, et al. Physical activity, muscle strength, and exercise capacity 3 months after severe sepsis and septic shock. Intensive Care Med 2015; 41: 1433–1444.

103.

Fan

Dowdy

Colantuoni

, et al. Physical complications in acute lung injury survivors: a two-year longitudinal prospective study. Crit Care Med 2014; 42: 849–859.

104.

Bohannon

Wang

Gershon

. Two-minute walk test performance by adults 18 to 85 years: normative values, reliability, and responsiveness. Arch Phys Med Rehabil 2015; 96: 472–477.

105.

Connelly

Thomas

Cliffe

, et al. Clinical utility of the 2-minute walk test for older adults living in long-term care. Physiother Can 2009; 61: 78–87.

106.

Chan

WLS

. Reliability, validity and minimal detectable change of 2-minute walk test, 6-minute walk test and 10-meter walk test in frail older adults with dementia. Exp Gerontol 2019; 115: 9–18.

107.

Verweij

Schiphorst

Pronk

, et al. Physical performance measures for predicting outcome in cancer patients: a systematic review. Acta Oncol 2016; 55: 1386–1391.

108.

Berry

Love

Files

, et al. The relationship between self-report and performance-based measures of physical function following an ICU stay. J Crit Care 2019; 51: 19–23.

109.

Bakhru

Davidson

Bookstaver

, et al. Physical function impairment in survivors of critical illness in an ICU recovery clinic. J Crit Care 2018; 45: 163–169.

110.

Denehy

de Morton

Skinner

, et al. A physical function test for use in the intensive care unit: validity, responsiveness, and predictive utility of the physical function ICU test (scored). Phys Ther 2013; 93: 1636–1645.

111.

Fischer

Spiegl

Altmann

, et al. Muscle mass, strength and functional outcomes in critically ill patients after cardiothoracic surgery: does neuromuscular electrical stimulation help? The Catastim 2 randomized controlled trial. Crit Care 2016; 20: 30.

112.

Lasocki

Chudeau

Papet

, et al. Prevalence of iron deficiency on ICU discharge and its relation with fatigue: a multicenter prospective study. Crit Care 2014; 18: 542.

113.

Mohamed-Hussein

AAR

Makhlouf

Selim

, et al. Association between hand grip strength with weaning and intensive care outcomes in COPD patients: a pilot study. Clin Respir J 2018; 12: 2475–2479.

114.

Parry

Denehy

Beach

, et al. Functional outcomes in ICU – what should we be using? – An observational study. Crit Care 2015; 19: 127.

115.

Schujmann

Lunardi

. Progressive mobility program and technology to increase the level of physical activity and its benefits in respiratory, muscular system, and functionality of ICU patients: study protocol for a randomized controlled trial. Trials 2018; 19: 274.

116.

Heyland

Davidson

Skrobik

, et al. Improving partnerships with family members of ICU patients: study protocol for a randomized controlled trial. Trials 2018; 19: 3.

117.

American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5, Washington, DC: American Psychiatric Association, 2013.

118.

Terwee

Bot

de Boer

, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60: 34–42.

119.

Mokkink

Terwee

Patrick

, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010; 63: 737–745.