A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Abstract

Objectives. To derive a shorter version of the Motor Activity Log Quality-of-Movement Scale (MAL-28) with enhanced content and construct validity. Design. Validation cohort. Setting. Outpatient rehabilitation within an academic laboratory. Participants. Retrospective consecutive sample of 149 community-dwelling adults with chronic mild/moderate upper-extremity hemiparesis caused by stroke or multiple sclerosis (MS). Intervention. Not applicable. Methods. Participants received the MAL-28 at baseline and following upper-extremity rehabilitation. Rasch Measurement Theory informed threshold ordering of scoring categories, tests of fit, differential item functioning, targeting, response dependency, local dependency, and reliability (person separation index [PSI]). Seasoned examiners rated the content validity of each item. Test-retest reliability of the revised scale was calculated. Results. We established content and construct validity for 18 items. The resultant 18-item MAL fit the model (χ² = 77.93; df = 72; P = .30) and targeted the population—that is, minimal floor (12.08%) or ceiling effects (0%), with acceptable reliability (PSI = 0.84) and good test-retest reliability [ICC(1, 1) = 0.86]. The hierarchy of item difficulty was independent of sex, age, affected side, diagnosis, or intervention type used, and there was local dependency in 3 pairs of items. Responses from a subsequent testing session were dependent on the responses from prior testing, indicating response dependency, for which a correction was proposed. Once response dependency was neutralized, there was a 15% greater treatment response. Conclusions. Content and construct validity are established for Rasch-based MAL-18 for chronic stages of stroke and MS. A Rasch-based conversion table enables clinical use of the MAL-18.

Keywords

validity Motor Activity Log stroke multiple sclerosis Rasch upper extremity

Introduction

Use of the affected arm in daily life^1,2 is an important metric of upper-limb improvement in people with neurological impairment. The Motor Activity Log-28 (MAL-28) is a self-assessment of amount and quality of movement of the hemiparetic arm during daily activities, rated on a 0 (no use) to 5 (normal ability) scale, with increments of 0.5.³ However, this ordinal scale fails to account for different difficulty hierarchies between items, thereby reducing the accuracy of quantifying differences in arm use.

Rasch Measurement Theory achieves interval scale level of measurement⁴ by accounting for both the participant’s overall arm use and relative item difficulty. Two Rasch analyses have previously been performed on the MAL in stroke.^5,6 However, transformation tables were not given, hindering wide use of Rasch-based measures. Moreover, response dependency was not addressed. Response dependency happens when scoring at a later time point is statistically dependent on previous scoring (Supplement 1).⁷ This is especially relevant when evaluating treatment efficacy in clinical trials because the occurrence of response dependency can under- or overestimate the treatment response.

This brief report proposes a revised 18-item MAL that meets the standards for content and construct validity, unidimensionality, local dependency, and response dependency in people with stroke and multiple sclerosis (MS).

Methods

Participants

A retrospective consecutive sample of 149 participants (Supplement 2) provided informed consent between November 2012 and February 2018 to participate in 1 of 4 laboratory-based constraint-induced movement therapy studies. Participants had mild to moderate chronic hemiparesis of at least 6 months duration resulting from stroke or MS. Aside from diagnosis, inclusion criteria were consistent between studies. They included presence of nonuse (MAL-28 mean score ≤2.5/5 at screening), 45° active shoulder flexion/abduction, 45° elbow flexion/extension, 10° finger/wrist extension,⁸ and cognitive capacity to provide informed consent. The sample demographics are the following: age 57 ± 15 years, 88 women, 127 with diagnosis of stroke, and 75 with left hemiparesis.

Data Collection

The MAL-28 Quality-of-Movement Scale was administered before and after receiving constraint-induced movement therapy or comparator interventions⁸ (Supplement 3). The Amount-of-Use Scale was not included because it measures an overlapping construct (r = 0.92)³ and is less reliable.⁹

Content Validity

Five experienced examiners independently categorized each item on the MAL-28 as “essential” or “questionably useful” for measuring daily arm use. Only items rated as “essential” by all 5 testers were included in the revised MAL (Supplement 4).

Construct Validity

Data were analyzed with Rasch Unidimensional Measurement Model 2030 software¹⁰ using a rating scale, polytomous model. Rasch model estimation is based on pairwise conditional maximum likelihood. Indexes of adherence to the model are considered as a form of construct validity. Rasch Measurement Theory informed threshold ordering of scoring categories (Supplement 5), tests of fit, differential item functioning (DIF), targeting, local dependency, response dependency, and reliability (person separation index [PSI]). DIF analyses whether the order of the hierarchy is maintained across demographic characteristics. In this case, DIF was performed on sex (female/male), age (below/above 60 years), affected side (left/right), diagnosis (stroke/MS), and intervention type.

To analyze construct validity, data were “stacked,” producing 2 observations per participant. The data were then reentered in a “racked” way, simulating a 56-item assessment (28 pretreatment, 28 posttreatment scores) to analyze response dependency and apply a correction factor as needed.

Test-Retest Reliability

Test-retest reliability on Rasch-derived scores of the MAL was evaluated with intraclass correlation [ICC(1, 1); https://www.statstodo.com/IntraclassCorrelation_Pgm.php] for the subset of 19 participants (9 stroke) who had blinded double-baseline testing.

Results and Discussion

Content Validity

A total of 8 items were rated “questionably useful” by at least 1 rater and were thus flagged for removal. Six were not universally applicable: comb hair (short hair or baldness), turn a key (keypad/key fob used instead), carry an object (wheelchair users), button shirt, put shoes on, take shoes off (adaptations to wardrobe following disability). Getting up from a chair and getting out of a car did not necessitate upper-extremity use.

Construct Validity

All MAL-28 items had disordered thresholds. Scoring options were collapsed until there were no more disordered thresholds (Table 1, Supplement 5). Conceptual considerations (ie, the extent to which the weaker arm can participate independently in a task) guided which item categories were collapsed, whereas as many response categories as possible were preserved for each item. Three items (carry object; dry hands; turn light on/off) were deleted following poor model fit (χ² item fit residuals > 2.5; Bonferroni adjusted P < .0006).⁴

Table 1.

Rasch-Based MAL-18 With New Scoring Categories.^a

		Verbal Description and MAL-28 Numerical Ratings
		Not Used (0), Very Poor (1), or Required Some Help From the Stronger Arm (2)			Fair	Almost Normal	Normal
	Item (Easy to Hard)	0	0.5-1	1.5-2	2.5-3	3.5-4	4.5-5
1	Using fork or spoon for eating	0	0	0	1	2	3
2	Wash your hands	0	0	0	1	2	3
3	Take off your socks	0	0	0	1	1	2
4	Wipe off kitchen counter	0	0	0	1	2	3
5	Pull chair away from table	0	0	0	1	1	2
6	Turn water on/off with knob/lever on faucet	0	0	0	1	2	3
7	Open door by turning doorknob	0	0	0	1	2	3
8	Remove clothing from drawer	0	0	0	1	2	3
9	Open drawer	0	0	0	1	2	3
10	Open refrigerator	0	0	0	1	1	2
11	Put on your socks	0	0	0	1	1	2
12	Use a TV remote control	0	0	0	1	1	2
13	Pull chair toward table after sitting down	0	0	0	1	1	2
14	Pick up a glass, bottle, cup, or can	0	0	0	1	1	2
15	Pick up a cup by the handle	0	0	0	1	1	2
16	Brush your teeth	0	0	0	1	1	2
17	Eat half a sandwich or finger foods	0	0	0	1	1	2
18	Pick up phone	0	0	0	1	1	2

Abbreviation: MAL, Motor Activity Log.

Rasch-based MAL-18 scoring method. The original 11-point scale is converted to a 3-point or 4-point scale, depending on psychometric properties of the item. The MAL-28 scoring anchors are shown in the header, with the converted scores listed below for each item. A more detailed description of MAL-28 scoring can be found in the administration manual (https://www.uab.edu/citherapy/images/pdf_files/CIT_Training_MAL_manual.pdf).

After removing items with questionable content validity and/or poor fit, 18 items remained (Table 1; Supplementary Table 1; Supplementary Figure 1). Table 2 shows the conversion of the original scores to Rasch-based scores (logits and scaled logits). Supplement 6 illustrates the process of applying Rasch-based scoring.

Table 2.

Converted Scoring System for the Total Score of the Rasch-Based MAL-18.^a

Raw Score (Total Score)	Logit	Converted Logit 0-100
0	−4.617	0.00
1	−3.8	8.79
2	−3.241	14.80
3	−2.859	18.91
4	−2.564	22.09
5	−2.321	24.70
6	−2.113	26.94
7	−1.93	28.91
8	−1.766	30.67
9	−1.615	32.30
10	−1.476	33.79
11	−1.346	35.19
12	−1.222	36.53
13	−1.104	37.79
14	−0.99	39.02
15	−0.879	40.22
16	−0.771	41.38
17	−0.665	42.52
18	−0.56	43.65
19	−0.455	44.78
20	−0.351	45.90
21	−0.246	47.03
22	−0.14	48.17
23	−0.032	49.33
24	0.078	50.51
25	0.19	51.72
26	0.306	52.96
27	0.425	54.24
28	0.549	55.58
29	0.678	56.97
30	0.813	58.42
31	0.954	59.94
32	1.103	61.54
33	1.261	63.24
34	1.429	65.05
35	1.609	66.98
36	1.804	69.08
37	2.017	71.37
38	2.256	73.94
39	2.528	76.87
40	2.852	80.36
41	3.261	84.76
42	3.845	91.04
43	4.678	100.00

Abbreviation: MAL, Motor Activity Log.

Converted scoring system for Rasch-based MAL-18. This table can be used when all items were answered and were applicable to the individual. Prior to utilizing this table, rescore each item according to Table 1, then sum the scores across the 18 items to calculate the total score.

MAL-18 retained acceptable internal consistency reliability (PSI = 0.84), good test-retest reliability [ICC(1, 1) = 0.86 vs ICC(1, 1) = 0.87 for MAL-28], and established construct validity.⁴ The item-trait interaction demonstrated good model fit (χ² = 77.93; df = 72; P = .30), reflecting the property of invariance across the trait.⁴ The scale had acceptable floor and ceiling targeting (12% minimum score, 0% maximum score; Supplement 7). DIF analysis revealed that item hierarchy was stable across sex, age, affected side, diagnosis, and intervention type, and there was local dependency (r ≥ 0.3) in 3 pairs of items (Supplement 8).

The mean person location was −1.79 ± 1.52 logits compared with the mean item location set at 0, reflecting that the average item difficulty is somewhat higher than the average patient’s ability. For a well-targeted scale, the mean person location should be around 0.⁴ The scale is possibly sensitive to differences between diagnostic groups; Supplementary Figure 2 shows that people with stroke (blue histogram, mean person location −1.99 ± 1.46 logits) showed poorer quality of movement than people with MS (red histogram, mean person location −0.68 ± 1.41 logits). Treatment effect is appropriately observed when comparing arm use at posttreatment (red histogram) versus pretreatment (blue histogram; Supplementary Figure 3). The racked data showed response dependency, requiring correction. Before correction, the average posttreatment person location was −1.49 ± 1.49 logits. After correction, the average person location was −1.35 ± 1.47 logits. Given a pretreatment MAL-18 score of −2.28 ± 1.60 logits, failure to correct for response dependency would have underestimated the treatment response by 15.05%.

Limitations

The group with MS was smaller than the group with stroke; thus, item calibration is based primarily on people with stroke.

Conclusion

The Rasch-based 18-item MAL has good test-retest reliability, achieves content and construct validity, and has acceptable internal consistency among people with stroke and MS. A Rasch-based conversion table enables clinical use of the Rasch-based scoring system.

Supplemental Material

Supplement_1_response_dependency – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Supplement_1_response_dependency for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Supplement_2_sample – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Supplement_2_sample for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Supplement_3_MAL_Administration – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Supplement_3_MAL_Administration for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Supplement_4_Content_Validity – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Supplement_4_Content_Validity for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Supplement_5_RMT – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Supplement_5_RMT for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Supplement_6_example_indiv_use – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Supplement_6_example_indiv_use for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Supplement_7_floor_ceiling – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Supplement_7_floor_ceiling for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Supplement_8_DIF_and_local_dependency – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Supplement_8_DIF_and_local_dependency for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Suppl_Fig_1._Threshold_map – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Suppl_Fig_1._Threshold_map for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Suppl_Fig_2._Plot_diagnose – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Suppl_Fig_2._Plot_diagnose for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Suppl_Fig_3._Plot_pre-T_and_post-T_data – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Suppl_Fig_3._Plot_pre-T_and_post-T_data for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Supplemental Material

Suppl_Table_1 – Supplemental material for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis

Supplemental material, Suppl_Table_1 for A Revised Motor Activity Log Following Rasch Validation (Rasch-Based MAL-18) and Consensus Methods in Chronic Stroke and Multiple Sclerosis by Ann Van de Winckel and Lynne Gauthier in Neurorehabilitation and Neural Repair

Footnotes

Acknowledgements

The authors acknowledge Kristina Kelly, PT, Alexandra Borstad, PT, PhD, Kala Phillips, Alli Hall, Mary Russell, Troy Richter, and other members of the Neurorecovery and Brain Imaging laboratory for their contribution toward data collection.

Supplementary material for this article is available on the Neurorehabilitation & Neural Repair website along with the online version of this article.

Authors’ Note

AVW and LG significantly contributed to the concept of the work; LG oversaw data acquisition, obtained content validity ratings, and provided clinical conceptualization; AVW performed the Rasch analysis and interpretation. AVW and LG cowrote the article. Both authors approved the final version and agree to be accountable for all aspects of the work.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr Van de Winckel has nothing to disclose. Dr Gauthier reports grants/contracts from PCORI, AHA, Rudi Schulte Foundation, and National MS Society during the conduct of the study.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial support for this research was provided by the American Heart Association (AHA), National MS Society, Rudi Schulte Foundation, and the Patient-Centered Outcomes Research Institute (PCORI). The statements in this publication are solely the responsibility of the authors and do not necessarily represent the views of the funding agencies. Additional support for participant recruitment and regulatory affairs was provided by the Center for Clinical and Translational Sciences (National Center for Advancing Translational Sciences, Grant No. 8UL1TR000090-05). Access to the original dataset: .

ORCID iD

Ann Van de Winckel .

References

Mark

Taub

Uswatte

et al. Phase II randomized controlled trial of constraint-induced movement therapy in multiple sclerosis: part 1. Effects on real-world function. Neurorehabil Neural Repair. 2018;32:223-232.

Lewthwaite

Winstein

Lane

et al. Accelerating stroke recovery: body structures and functions, activities, participation, and quality of life outcomes from a large rehabilitation trial. Neurorehabil Neural Repair. 2018;32:150-165.

Uswatte

Taub

Morris

Light

Thompson

. The Motor Activity Log-28: assessing daily use of the hemiparetic arm after stroke. Neurology. 2006;67:1189-1194.

Tennant

Conaghan

. The Rasch measurement model in rheumatology: what is it and why use it? When should it be applied, and what should one look for in a Rasch paper? Arthritis Rheum. 2007;57:1358-1362.

Silva

ESM

Pereira

Gianlorenço

ACL

Camargo

. The evaluation of non-use of the upper limb in chronic hemiparesis is influenced by the level of motor impairment and difficulty of the activities—proposal of a new version of the Motor Activity Log. Physiother Theory Pract. 2019;35:964-974. doi:10.1080/09593985.2018.1460430

Chuang

Lin

Hsieh

Liu

Chen

. Using Rasch analysis to validate the Motor Activity Log and the Lower Functioning Motor Activity Log in patients with stroke. Phys Ther. 2017;97:1030-1040.

Marais

. Response dependence and the measurement of change. J Appl Meas. 2009;10:17-29.

Gauthier

Kane

Borstad

et al. Video Game Rehabilitation for Outpatient Stroke (VIGoROUS): protocol for a multi-center comparative effectiveness trial of in-home gamified constraint-induced movement therapy for rehabilitation of chronic upper extremity hemiparesis. BMC Neurol. 2017;17:109. doi:10.1186/s12883-017-0888-0

Uswatte

Taub

Morris

Vignolo

McCulloch

. Reliability and validity of the upper-extremity Motor Activity Log-14 for measuring real-world arm use. Stroke. 2005;36:2493-2496.

10.

Andrich

Sheridan

Luo

. RUMM2030: Rasch Unidimensional Models for Measurement. Perth, Australia: RUMM Laboratory; 2015.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB

0.03 MB

0.02 MB

0.01 MB

0.16 MB

0.01 MB

0.06 MB