Heuristic Evaluation and Usability Testing of an Artificial Intelligence-Powered Therapeutic Exercise System for Managing Knee Osteoarthritis in Older Adults
Restricted accessResearch articleFirst published online September, 2025
Heuristic Evaluation and Usability Testing of an Artificial Intelligence-Powered Therapeutic Exercise System for Managing Knee Osteoarthritis in Older Adults
This study evaluated the usability of an artificial intelligence (AI)-powered therapeutic exercise system for older adults with knee osteoarthritis (KOA) to support home-based strengthening, stretching, and balance exercises. The system integrates motion-capture technology and real-time AI feedback to emulate a physical therapist. Two human factors practitioners independently evaluated the system against established usability and user interface heuristics to identify design deficiencies. Eleven older adults with KOA (mean age ± standard deviation = 67.7 ± 3.0 years) participated in usability testing and completed tasks that simulated real-world system use. A researcher observed and documented their interactions, areas of confusion, errors, and assistance requests. After the testing, the older adults provided feedback on the system through the System Usability Scale and semi-structured interviews. Key usability issues identified included a complex camera setup, insufficiently detailed exercise demonstrations, limited exercise selection flexibility, inability to pause/rewind exercises, and poor error tolerance. The study also identified deficiencies in the graphical user interface. Based on these findings, actionable design improvements are proposed to enhance the system’s usability and guide future digital health technology design.
BrookeJ. B. (1996). SUS: A ‘quick and dirty’ usability scale. In JordanP. W.ThomasB.McClellandI. L.WeerdmeesterB. (Eds.), Usability evaluation in industry (Vol. 189, pp. 4–7). CRC Press.
2.
ChanK. L.LeungW. C.TiwariA.OrK. L.IpP. (2019). Using smartphone-based psychoeducation to reduce postnatal depression among first-time mothers: Randomized controlled trial. JMIR mHealth and uHealth, 7(5), Article e12794.
3.
ChenJ.OrC. K.LiZ.YeungE. H. K.ZhouY.HaoT. (2023). Effectiveness, safety and patients’ perceptions of an immersive virtual reality–based exercise system for poststroke upper limb motor rehabilitation: A proof-of-concept and feasibility randomized controlled trial. Digital Health, 9, Article 20552076231203599.
4.
ChenT.OrC. K. (2021). Development and pilot test of a machine learning-based knee exercise system with video demonstration, real-time feedback, and exercise performance score. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 65(1), 1519–1523.
HwangW.SalvendyG. (2010). Number of people required for usability evaluation: The 10±2 rule. Communications of the ACM, 53(5), 130–133.
7.
LawfordB. J.HallM.HinmanR. S.Van der EschM.HarmerA. R.SpiersL.KimpA.Dell’IsolaA.BennellK. L. (2024). Exercise for osteoarthritis of the knee. Cochrane Database of Systematic Reviews, 2024(12), Article CD004376.
8.
LiuH.OrC.LouV. W. Q.HuY.XiN. (2023). Perceptions of and intention to use wearable and assistive devices among older adults [Conference session]. Proceedings of the 14th International Conference on Applied Human Factors and Ergonomics – Human Factors and Wearable Technologies Track, San Francisco, CA, USA, pp. 101–110.
9.
LiuK.ChanF.-Y.OrC. K.SunD. T.-F.LaiW.-S.SoH.-Y. (2019). Heuristic evaluation and simulated use testing of infusion pumps to inform pump selection. International Journal of Medical Informatics, 131, Article 103932.
10.
MoL.JiangB.MeiT.ZhouD. (2023). Exercise therapy for knee osteoarthritis: A systematic review and network meta-analysis. Orthopaedic Journal of Sports Medicine, 11(5), Article 23259671231172773.
NielsenJ.MolichR. (1990). Heuristic evaluation of user interfaces [Conference session]. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, pp. 249–256.
13.
OrC.TaoD. (2012). Usability study of a computer-based self-management system for older adults with chronic diseases. JMIR Research Protocols, 1(2), Article e2184.
14.
OrC.WongK.TongE.SekA. (2014). Private primary care physicians’ perspectives on factors affecting the adoption of electronic medical records: A qualitative pre-implementation study. Work, 48(4), 529–538.
15.
OrC. K.ChanA. H. (2024). Inspection methods for usability evaluation. In StephanidisC.SalvendyG. (Eds.), User experience methods and tools in human-computer interaction (pp. 170–192). CRC Press.
16.
OrC. K.HoldenR. J.ValdezR. S. (2023). Human factors engineering and user-centered design for mobile health technology: Enhancing effectiveness, efficiency, and satisfaction. In DuffyV. G.ZiefleM.RauP.-L. P.TsengM. M. (Eds.), Human-automation interaction (pp. 97–118). Springer International Publishing.
17.
PetrovčičA.RogeljA.DolničarV. (2018). Smart but not adapted enough: Heuristic evaluation of smartphone launchers with an adapted interface and assistive technologies for older adults. Computers in Human Behavior, 79, 123–136.
18.
SteinmetzJ. D.CulbrethG. T.HaileL. M.RaffertyQ.LoJ.FukutakiK. G.CruzJ. A.SmithA. E.VollsetS. E.BrooksP. M.CrossM.WoolfA. D.HaginsH.Abbasi-KangevariM.AbediA.AckermanI. N.AmuH.AntonyB.ArablooJ.. . . KopecJ. A. (2023). Global, regional, and national burden of osteoarthritis, 1990–2020 and projections to 2050: A systematic analysis for the Global Burden of Disease Study 2021. The Lancet Rheumatology, 5(9), e508–e522.
19.
TaoD.OrC. (2012). A paper prototype usability study of a chronic disease self-management system for older adults [Conference session]. Proceedings of the 2012 IEEE International Conference on Industrial Engineering and Engineering Management, Hong Kong, China, pp. 1262–1266.
20.
VirziR. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough?Human factors, 34(4), 457–468.
21.
WongR. S. M.YuE. Y. T.WongT. W.-L.FungC. S. C.ChoiC. S. Y.OrC. K. L.LiuK. S. N.WongC. K. H.IpP.LamC. L. K. (2020). Development and pilot evaluation of a mobile app on parent-child exercises to improve physical activity and psychosocial outcomes of Hong Kong Chinese children. BMC Public Health, 20, 1–13.
22.
WooJ.LeungJ.LauE. (2009). Prevalence and correlates of musculoskeletal pain in Chinese elderly and the impact on 4-year physical function and quality of life. Public Health, 123(8), 549–556.
23.
XieZ.NaciogluA.OrC. (2018). Prevalence, demographic correlates, and perceived impacts of mobile health app use amongst Chinese adults: Cross-sectional survey study. JMIR Mhealth Uhealth, 6, Article e103.
24.
ZhangM. (2023). Older people’s attitudes towards emerging technologies: A systematic literature review. Public Understanding of Science, 32(8), 948–968.
