Sage Journals: Discover world-class research

Abstract

Objective

We tested whether intermixing mental health items with items addressing comfort and capability could limit the floor effects noted when mental health is measured in musculoskeletal specialty care.

Methods

One hundred and 31 people seeking care for upper and lower extremity musculoskeletal conditions were randomized to complete randomly ordered, unlabeled mental health items intermixed with comfort and capability items, or intact and labelled questionnaires. For the two approaches, we compared: (1) flooring and ceiling effects; (2) mean and median questionnaire scores; (3) internal consistency (Cronbach alpha); and (4) exploratory factor analysis. We sought correlations between mental health and levels of pain intensity and capability.

Results

We found slightly more flooring in the intermixed group for symptoms of depression (66% [41 of 62] vs 46% [32 of 69], p-value = .034), no differences in the mean and median scores for each questionnaire, lower internal consistency measured by Cronbach alpha, and lower factor loading coefficients in exploratory factor analysis for symptoms of depression and anxiety in the intermixed group. The mean level of symptoms of anxiety was significantly different between two groups (intermixed: 0.87 [95% CI 0.82 to 0.92], fixed: 0.96 [95% CI 0.93 to 0.98]). There were no differences in the association of the mental health measures gathered via the two different strategies with measures of pain intensity and magnitude of capability.

Conclusion

The finding that intermixing mental health questions with questions about comfort and capability did not diminish floor effects suggests no advantage to intermixing mental health items in questionnaires used in musculoskeletal care and research.

Keywords

Mental health patient reported outcome measures pain intensity incapability floor effects

Introduction

Background

Questionnaires used to quantify the subjective aspects of health such as comfort and capability are referred to as patient reported outcome measures (PROMs).¹ Questionnaires are also used to measure mental health, such as symptoms of depression or anxiety.² There is a notable correlation between measures of capability and comfort and mental health measures.^3–5 And the true correlations may be stronger given the evidence that people don’t report measures of symptoms of depression and anxiety forthrightly, creating a strong floor effect for measures of symptoms of depression and anxiety.^6,7

Rationale

The existing research questionnaires may pose challenges as they tend to be lengthy and burdensome for participants. These questionnaires may include mental health measures that encompass factors perceived as irrelevant by patients, which can potentially be offensive. This issue becomes evident through the quick completion of the questionnaires and the prevalence of high floor effects, particularly observed in measures related to symptoms of general anxiety and depression within musculoskeletal care settings.^8,9

Factors such as internal reliability (the degree to which the items in an instrument measure the same construct)¹⁰ and flooring and ceiling effects partially determine the usefulness of questionnaires in a medical setting. Ceiling or flooring effects occur when a large proportion of participants provide maximum or minimum scores to questionnaires, which results in lost information at the top or bottom of the scale.¹¹ Our thought was that intermixing mental health questions with questions regarding comfort and capability might make answering questions about mental health seem more appropriate in musculoskeletal specialty care, thereby limiting the observed flooring and ceiling effects. In other words, a set of questions about mental health together might seem like screening for depression or anxiety, which might feel inappropriate or unwelcomed. On the other hand, intermixing questionnaires with questions regarding comfort and capability might be percieved as a genuine curiosity about the impact of those symptoms on overall wellbeing. We found a study of 603 college students who completed a set of 3 12-item questionnaires evaluating a travel website, half randomly intermixed and half kept with the original questionnaire, that found that internal reliability (the degree to which the items in an instrument measure the same construct)¹⁰ was increased with grouping of questionnaires while correlation with other constructs was reduced.⁷ We did not find examples of intermixing questionnaire items to try to reduce flooring and ceiling effects.

Study questions

Among people seeking musculoskeletal specialty care, we asked: (1) Is there a difference in flooring and ceiling effects, mean and median questionnaire scores, internal consistency, and factor loading coefficients in exploratory factor analysis between randomly intermixed unlabeled questions compared to labeled fixed order complete questionnaires? (2) Is there a difference in the association with capability and pain intensity?

Materials and methods

Study design and setting

We obtained approval from our Institutional Review Board. For this cross-sectional questionnaire-based randomized study we enrolled participants seeking care regarding upper and lower extremity conditions at two regional orthopedic clinics, and one institution-based upper extremity clinic in an urban area in the United States between July and August 2021.

Participants

Consecutive new and returning patients were approached by a research assistant who was not directly involved in care before or after their visit. All patients who were not fluent in English or were unable to provide verbal informed consent were excluded. We perform extensive cross-sectional research and stopped tracking declines to participate because they are very infrequent. A total of 143 participants began filling out the survey on a tablet device or phone in a private exam room without a researcher present; however, 12 of them did not complete the survey. The most common reasons for noncompletion were a desire to bring a long visit to a close and logistical problems such as system errors or internet connectivity issues. Multiple imputation, the ideal method of addressing missing data at random,¹² cannot be used for factor analysis. We therefore omitted all 12 patients who did not complete all questionnaire, leaving 131 patients for analysis: 69 participants in the fixed order group and 62 participants in the intermixed group. Our Institutional Review Board approved the protocol and accepted completing the survey as a form of consent. The average completion time was less than 9 minutes. This ensures that respondents are unlikely to experience survey fatigue.

Randomization

Participants were randomized 1:1 into two groups: (1) Participants completed randomly ordered, intermixed, non-labeled questionnaires. We refer to this group as the intermixed group. (2) Participants completed each questionnaire with its questions in the same order as used in the questionnaire’s development and validation studies. In this group, each questionnaire was labelled with the questionnaire’s name (e.g., The Negative Pain Thoughts Questionnaire). The questionnaires were also in the same order for each participant. We refer to this group as the fixed-order group. Randomization was achieved through the survey software used SurveyMonkey (Palo Alto, CA, USA). We contacted SurveyMonkey on their exact method of randomization, but they wouldn’t comment. We assume randomization is based on a semi-random number generator.

Questionnaires

Patient-Reported Outcome Measurement Information System Physical Functioning Short Form (PROMIS PF SF), Negative Pain Thoughts Questionnaire-4 (NPTQ), and Pain Catastrophizing Scale-4 (PCS-4) were answered on 5-point Likert scales, while Patient Health Questionnaire-2 (PHQ-2) and Generalized Anxiety Disorder 2 (GAD-2) were answered on a 4-point Likert scale.

The validated PROMIS PF SF Version two consists of eight questions, and measures limitations of physical activity.¹³ Higher scores indicate better physical function (fewer limitations of physical activity).^14,15 Sample questions include, “Are you able to do chores such as vacuuming or yard work?” and “are you able to go for a walk of at least 15 min”. A score of 50 is the average for the United States general population with a standard deviation of 10. PROMIS PF SF has shown high internal reliability via Chronbach alpha in patients with lower extremity orthopedic trauma injuries.¹³

We measured distress and misconceptions about symptoms with (1) the validated Negative Pain Thoughts Questionnaire (NPTQ-4). Higher scores indicate greater distress and unhelpful thoughts.¹⁶ Sample statements include “my problem makes me feel awful and it overwhelms me” and “even though I can still do a lot of things, I can’t enjoy them because of my condition”. The scores range from 4 to 20. Previous studies have shown greater physical capability to be associated with fewer negative pain thoughts.¹⁷ (2) Validated PCS-4,¹⁸ total scores ranging from 0 to 16 with higher scores representing greater distress and misconceptions.¹⁹ A sample statement includes, “I worry all the time about whether the pain will end”. Magnitude of incapability is associated with catastrophic thinking as measured by the PCS-4.²⁰

We measured symptoms of anxiety using the validated GAD-2,²¹ a two-item screening questionnaire scored from 0 to 6 with a higher score indicating greater anxiety. Questions include: “over the last 2 weeks have you (1) felt nervous, anxious, or on edge and (2) been unable to stop control of worrying?”. Magnitude of incapability is associated with symptoms of depression and anxiety.²²

The validated PHQ-2 measures symptoms of depression.¹⁵ Its score ranges from 0 to six and higher score indicates more symptoms of depression.^15,23 Questions include: “over the last 2 weeks have you (1) had little interest or pleasure in doing things and (2) felt down, depressed, or hopeless?”. Pain intensity and incapability correlate with PHQ-2 scores in patients with upper extremity illness.²⁴

Pain intensity was measured on an ordinal scale from 0 to 10, with 0 indicating no pain and 10 the worst pain possible.

We did not collect patient demographics. Due to the randomization, we expect little influence of demographic variables. Our patients symptoms, diagnoses, and sociodemographics are representative of a typical musculoskeletal specialty practice.

Primary and secondary study outcomes

Our primary study goal was to determine if there was a difference in reliability between randomly intermixed and unlabeled questionnaires compared to labeled and fixed order questionnaires. We specifically tested reliability by comparing: (1) flooring and ceiling effects, using fisher exact test. We defined flooring as the lowest possible score and ceiling as the highest possible score; (2) mean or median questionnaire scores using t-test (parametric) or Mann-Whitney U test (non-parametric); (3) internal consistency, in other words, are the questions within each questionnaire measuring the same thing? We measured this in two ways: (A) Cronbach alpha and (B) exploratory factor analysis. Factor analysis was performed with a Promax rotation; 95% confidence intervals were created around factor loading coefficients through bootstrapping (n=1000). We defined a difference in internal consistency between embedded and intermixed groups as a non-overlapping 95% confidence interval.

Our secondary study goal was to compare the association with capability and pain intensity. We calculated spearman correlation with bootstrap (n=1000) 95% confidence intervals of each mental health measure with capability and pain intensity. We defined a difference in strength of association as non-overlapping 95% confidence intervals.

Power analysis

A priori power analysis indicated that to find a difference of five points on PROMIS PF short form, with an expected SD of 10 in both groups, we would need 63 participants in each group (total of 126) with alpha at 0.05 and power 80%. Based on previous study, our goal was to include 150 participants, in order to perform a reliable factor analysis. The study was inadvertently terminated upon reaching the sample size for our primary hypothesis, probably due to personnel change, but there was adequate power for the factors analysis with the number of patients available.

Results

Difference in flooring and ceiling effects, mean and median questionnaire scores, internal consistency, and factor loading coefficients in exploratory factor between unlabeled, intermixed questionnaires compared to labeled questionnaires in a fixed order

We found slightly more flooring in symptoms of depression in the intermixed questionnaire (66% [41/62] vs 46% [32/69], p-value = .034); (Table 1).

Table 1.

Difference in mean or median scores and the number of people with the lowest and highest score.

Mean or median score	PROMIS physical function	p-value	NPTQ-4	p-value	PCS-4	p-value	PHQ-2	p-value	GAD-2	p-value
Intermixed	44 ± 9.2		9.0 ± 4.4		4.0 (2.0–7.0)		0.0 (0.0–2.0)		1.0 (0.0–2.0)
Fixed order	43 ± 8.3	.61	9.2 ± 3.8	.75	4.0 (1.0–8.0)	.50	0.0 (0.0–1.0)	.066	0.0 (0.0–1.0)	.15
Number of people with the lowest score
Intermixed	0		5 (8%)		12 (19%)		41 (66%)		34 (55%)
Fixed order	0	n.a	4 (6%)	.74	7 (10%)	.74	32 (46%)	.034	28 (41%)	.12
Number of people with the highest score
Intermixed	0		0		0 (0%)		1 (2%)		0 (0%)
Fixed order	0	n.a	0	n.a	1 (1%)	1.0	3 (4%)	.62	2 (3%)	.50

We included 62 people in the intermixed group and 69 people in the fixed order group. PROMIS: patient-reported outcome measurement information system; NPTQ-4: negative pain thoughts questionnaire-4; PCS-4: pain catastrophizing scale-4; PHQ-2: patient health questionnaire-2 (depression); GAD-2: generalized anxiety disorder 2-item (anxiety); parametric data represented as mean ± SD, non-parametric data represented as median (IQR).

We found no differences in the mean and median scores among each of the questionnaires between groups (PROMIS PF SF, NPTQ-4, PCS-4, PHQ-2, GAD-2).

Although not significantly different, we found consistently lower internal consistency measured by Cronbach alpha in the intermixed questions groups compared to the fixed order question group; specifically for symptoms of anxiety (intermixed: α=0.64; 95% CI 0.35 to 0.93, fixed: α=0.86; 95% CI 0.73 to 0.99) and symptoms of depression (intermixed: α=0.67; 95% CI 0.48 to 0.87, fixed: α=0.92; 95% CI 0.86 to 0.98) (Table 2).

Table 2.

Difference in reliability measured by Cronbach alpha between fixed order and intermixed questionnaires.

Questionnaire	Fixed order	Intermixed
PROMIS physical function	0.95 (0.92–0.97)	0.93 (0.91–0.96)
NPTQ-4	0.84 (0.79–0.90)	0.77 (0.66–0.89)
PCS-4	0.89 (0.83–0.95)	0.88 (0.81–0.94)
PHQ-2	0.86 (0.73–0.99)	0.64 (0.35–0.93)
GAD-2	0.92 (0.86–0.98)	0.67 (0.48–0.87)

PROMIS: patient-reported outcome measurement information system; NPTQ-4: negative pain thoughts questionnaire-4; PCS-4: pain catastrophizing scale-4; PHQ-2: patient health questionnaire-2 (depression); GAD-2: generalized anxiety disorder 2-item (anxiety); alpha coefficient with 95% confidence interval.

In exploratory factor analysis the factor loading coefficients were lower for symptoms of depression and anxiety in the intermixed group. This difference was significant for symptoms of anxiety (GAD-2) (intermixed: 0.87 [95% CI 0.82 to 0.92], fixed: 0.96 [95% CI 0.93 to 0.98]). Question four of the NPTQ-4 had a lower coefficient in the intermixed group than the fixed order group (intermixed: 0.60 [95% CI 0.33 to 0.79], fixed: 0.88 [95% CI 0.82 to 0.93]); (Table 3).

Table 3.

Difference in questions groupings between intermixed and fixed order questionnaires.

Questionnaire	Questions	Factor loading coefficient (95% confidence interval) intermixed	Factor loading coefficient (95% confidence interval) fixed order
PROMIS physical function	1	0.88 (0.83 to 0.93)	0.80 (0.67 to 0.89)
	2	0.65 (0.48 to 0.79)	0.80 (0.69 to 0.89)
	3	0.65 (0.48 to 0.77)	0.79 (0.71 to 0.86)
	4	0.83 (0.75 to 0.89)	0.86 (0.79 to 0.92)
	5	0.89 (0.84 to 0.93)	0.87 (0.82 to 0.91)
	6	0.91 (0.87 to 0.94)	0.91 (0.87 to 0.94)
	7	0.88 (0.82 to 0.93)	0.90 (0.85 to 0.93)
	8	0.85 (0.79 to 0.90)	0.89 (0.85 to 0.92)
Pain catastrophizing scale	1	0.90 (0.86 to 0.94)	0.89 (0.84 to 0.93)
	2	0.84 (0.75 to 0.90)	0.90 (0.85 to 0.93)
	3	0.93 (0.90 to 0.96)	0.87 (0.80 to 0.92)
	4	0.78 (0.70 to 0.85)	0.83 (0.74 to 0.89)
NPTQ	1	0.80 (0.69 to 0.89)	0.75 (0.57 to 0.88)
	2	0.83 (0.77 to 0.88)	0.85 (0.79 to 0.90)
	3	0.84 (0.76 to 0.90)	0.82 (0.73 to 0.89)
	4	0.60 (0.33 to 0.79)	0.88 (0.82 to 0.93)
PHQ	1	0.86 (0.78 to 0.93)	0.94 (0.89 to 0.97)
PHQ	2	0.86 (0.78 to 0.93)	0.94 (0.89 to 0.97)
GAD	1	0.87 (0.82 to 0.92)	0.96 (0.93 to 0.98)
GAD	2	0.87 (0.82 to 0.92)	0.96 (0.93 to 0.98)

The factor loading coefficient indicates how much the question increases when the construct measured by the questionnaire (like physical function) increases by one; a coefficient of 1.0 would indicate perfect alignment. PROMIS: patient-reported outcome measurement information system; NPTQ-4: negative pain thoughts questionnaire-4; PCS-4: pain catastrophizing scale-4; PHQ-2: patient health questionnaire-2 (depression); GAD-2: generalized anxiety disorder 2-item (anxiety).

Difference in association with disability and pain intensity

There were no differences in association of the NPTQ-4, PCS-4, PHQ-2, and GAD-2 with measures of disability (PROMIS PF SF) and Pain Intensity scores between the intermixed and fixed-order groups (p < .05) (Table 4).

Table 4.

The difference in association of mental health with disability and pain intensity.

	Questions intermix		Questions fixed order
	Spearman rho (bootstrap 95% CI)	p value	Spearman rho (bootstrap 95% CI)	p value
Association with PROMIS PF
NPTQ-4	−0.53 (−0.71 to −0.35)	<.001	−0.54 (−0.73 to −0.34)	<.001
PCS-4	−0.45 (−0.64 to −0.26)	<.001	−0.49 (−0.71 to −0.27)	<.001
PHQ-2	−0.43 (−0.62 to −0.24)	<.001	−0.66 (−0.82 to −0.50)	<.001
GAD-2	−0.29 (−0.53 to −0.052)	<.001	−0.39 (−0.64 to −0.14)	.002
Association with pain intensity
NPTQ-4	0.51 (0.33 to 0.69)	<.001	0.55 (0.37 0.72)	<.001
PCS-4	0.72 (0.59 to 0.85)	<.001	0.65 (0.46 to 0.84)	<.001
PHQ-2	0.39 (0.17 to 0.61)	<.001	0.38 (0.15 to 0.60)	.001
GAD-2	0.35 (0.12 to 0.58)	.003	0.29 (0.033 to 0.55)	.027

Discussion

Patient-reported outcome measures quantify the subjective aspects of illness with the intention of capturing outcomes that matter to patients to help improve patient care. Current questionnaires are designed for research purposes and may be long and burdensome. Mental health measures may address factors patients deem irrelevant, and with a risk of offense, which seems manifest in rapid completion and high floor effects of measures of symptoms of general anxiety and symptoms of depression in musculoskeletal care.^6,7 We analyzed whether intermixing mental health items with items addressing physical symptoms and capability could limit flooring and ceiling effects, and if doing so would alter mean and median questionnaire scores, internal consistency, and factor loading coefficients in exploratory factor analysis. We conclude that intermixing mental health questions does not improve their performance.

Limitations

The findings of this study should be interpreted in light of some limitations. First, we had more incomplete questionnaires than typical and our cohorts were slightly unbalanced (69 and 62 participants). This was due to use of quick response (QR) codes to allow people to complete questionnaires on their phone rather than on our tablet during the COVID-19 pandemic. People using their phone are more likely to leave before the questionnaire is completed. Second, the generalizability might be limited because we only have English-speaking patients with musculoskeletal pain visiting orthopedic surgeons who are all White men. Our impression is that there is sufficient diversity for a viable experiment to measure associations and the concepts measured are unlikely to change with greater variation in language, socio-demographics, and specialist characteristics.

Difference in flooring and ceiling effects, mean and median questionnaire scores, internal consistency, and factor loading coefficients in exploratory factor between intermixed compared to fixed order questionnaires

The observation of comparable flooring, no differences in the mean and median scores, no differences in internal consistency, and lower factor loading coefficients for symptoms of anxiety in the intermixed group suggests there are no advantages to intermixing mental health and capability questions. These findings are similar to the prior study among college students completing questionnaires about travel websites, with the exception that their sample was larger, which may explain why their differences were statistically significant.⁷ For our purposes, small differences that might be significant in a larger sample are clinically irrelevant. Our primary aim was to reduce floor and ceiling effects to get better spread in mental health scores, and intermixing questionnaire items did not achieve this.

Difference in association with disability and pain intensity

The observation of no differences in association of the NPTQ-4, PCS-4, PHQ-2, and GAD-2 with measures of levels of incapability (PROMIS PF) and pain intensity between the intermixed and fixed-order group, further supports similar responses no matter the question presentation. Notably, this observation is in spite of the known, and reproduced, floor effects of questions about general despair and worry. Despite the imperfections of measuring mental health using questionnaires in musculoskeletal specialty care, we are still able to identify important relationships and opportunities for improved mental health.

Conclusion

The observation of no advantages to intermixing mental health and comfort and capability questionnaires suggests a need for alternative strategies to improve measurement of mental health in musculoskeletal specialty care. For instance, patients might answer questions addressing thoughts and feelings regarding physical symptoms more forthrightly than questions addressing general worry or despair.²⁵

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author’s Note

Work performed at the University of Louisville School of Medicine and Dell Medical School—University of Texas.

ORCID iD

Patrick Merkel

References

Meadows

. Patient-reported outcome measures: An overview. Br J Community Nurs 2011; 16: 146–151.

Kroenke

Spitzer

Williams

JBW

, et al. The patient health questionnaire somatic, anxiety, and depressive symptom scales: a systematic review. Gen Hosp Psychiatr 2010; 32: 345–359.

Crijns

Bernstein

Teunis

, et al. The Association between symptoms of depression and office visits in patients with nontraumatic upper-extremity illness. J Hand Surg 2020; 45: 159.e1–159.e8.

Overbeek

Nota

SPFT

Jayakumar

, et al. The PROMIS physical function correlates with the QuickDASH in patients with upper extremity illness. Clin Orthop 2015; 473: 311–317.

Drijkoningen

Menendez

, et al. The influence of psychological factors on the michigan hand questionnaire. Hand 2017; 12: 197–201.

Garbarski

Schaeffer

Dykema

. The effects of response option order and question order on self-rated health. Qual Life Res 2015; 24: 1443–1453. DOI:10.1007/s11136-014-0861-y.

Goodhue

Loiacono

. Randomizing survey question order vs. grouping questions by construct: an empirical test of the impact on apparent reliabilities and links to related constructs, 2002. 2002 Proceedings of the 35th Annual Hawaii International Conference on System Sciences HICSS. Big Island, Hawaii, Epub ahead of print January 2002. DOI:10.1109/HICSS.2002.994385.

Guattery

Dardas

Kelly

, et al. Floor Effect of PROMIS depression CAT associated with hasty completion in orthopaedic surgery patients. Clin Orthop Relat Res 2018; 476: 696–703.

Bernstein

Atkinson

Fear

, et al. Determining the generalizability of the PROMIS depression domain’s floor effect and completion time in patients undergoing orthopaedic surgery. Clin Orthop Relat Res 2019; 477: 2215–2225.

10.

Cronbach

. Coefficient alpha and the internal structure of tests, Psychometrika 1951; 16: 297–334.

11.

Andrade

. The ceiling effect, the floor effect, and the importance of active and placebo control arms in randomized controlled trials of an investigational drug. Indian J Psychol Med 2021; 43: 360–361.

12.

White

Royston

Wood

. Multiple imputation using chained equations: issues and guidance for practice. Stat Med 2011; 30: 377–399.

13.

Rothrock

Kaat

Vrahas

, et al. Validation of PROMIS physical function instruments in patients with an orthopaedic trauma to a lower extremity. J Orthop Trauma 2019; 33: 377–383.

14.

Bruce

Fries

Lingala

, et al. Development and assessment of floor and ceiling items for the PROMIS physical function item bank. Arthritis Res Ther 2013; 15: R144.

15.

Inagaki

Ohtsuki

Yonemoto

, et al. Validity of the patient health questionnaire (PHQ)-9 and PHQ-2 in general internal medicine primary care at a Japanese rural hospital: a cross-sectional study. Gen Hosp Psychiatr 2013; 35: 592–597.

16.

Vranceanu

A-M

Safren

Cowan

, et al. The development of the negative pain thoughts questionnaire. Pain Pract 2008; 8: 337–341.

17.

Rossano

Al Salman

Ring

, et al.

Do unhelpful thoughts or confidence in problem solving have stronger associations with musculoskeletal illness?

Clin Orthop Relat Res 2022; 480: 287–295.

18.

Sullivan

MJL

Bishop

Pivik

. The pain catastrophizing scale: development and validation. Psychol Assess 1995; 7: 524–532.

19.

Bot

AGJ

Becker

SJE

Bruijnzeel

, et al. Creation of the abbreviated measures of the pain catastrophizing scale and the short health anxiety inventory: the PCS-4 and SHAI-5. J Muscoskel Pain 2014; 22: 145–151.

20.

Kopp

Furlough

Goldberg

, et al. Factors associated with pain intensity and magnitude of limitations among people with hip and knee arthritis. J Orthop 2021; 25: 295–300.

21.

Sapra

Bhandari

Sharma

, et al. Using generalized anxiety disorder-2 (GAD-2) and GAD-7 in a primary care setting. Cureus 2020; 12: Article e8224. Epub ahead of print 21 May 2020. DOI:10.7759/cureus.8224.

22.

Al Salman

Khatiri

Cremers

, et al. Difficult life events affect lower extremity illness. Arch Orthop Trauma Surg 2022; 142: 599–605. Epub ahead of print November 2020. DOI:10.1007/s00402-020-03686-y.

23.

Spitzer

Kroenke

Williams

JBW

. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. Patient health questionnaire. JAMA 1999; 282: 1737–1744.

24.

Hageman

MGJS

Briet

Oosterhoff

TCH

, et al. The correlation of cognitive flexibility with pain intensity and magnitude of disability in upper extremity illness. J Hand Microsurg 2014; 6: 59–64.

25.

Teunis

Al Salman

Koenig

, et al. Unhelpful thoughts and distress regarding symptoms limit accommodation of musculoskeletal pain. Clin Orthop 2022; 480: 276–283.

Is there a difference in floor effects and reliability between intermixed and fixed-order items in a questionnaire?

Abstract

Objective

Methods

Results

Conclusion

Keywords

Introduction

Background

Rationale

Study questions

Materials and methods

Study design and setting

Participants

Randomization

Questionnaires

Primary and secondary study outcomes

Power analysis

Results

Difference in flooring and ceiling effects, mean and median questionnaire scores, internal consistency, and factor loading coefficients in exploratory factor between unlabeled, intermixed questionnaires compared to labeled questionnaires in a fixed order

Difference in association with disability and pain intensity

Discussion

Limitations

Difference in flooring and ceiling effects, mean and median questionnaire scores, internal consistency, and factor loading coefficients in exploratory factor between intermixed compared to fixed order questionnaires

Difference in association with disability and pain intensity

Conclusion

Footnotes

Declaration of conflicting interests

Funding

Author’s Note

ORCID iD

References