ChatGPT is a concise,comprehensive,and accurate alternative to google for answering common hand therapy questions

Abstract

Objective

ChatGPT is a popular artificial intelligence (AI) tool used to answer questions on any subject. Given ChatGPT’s popularity, it is prudent to investigate its ability to answer common patient questions in the field of hand therapy to better guide patients as they navigate the resources available to them.

Methods

This is a cross-sectional, rater-based comparison study. Four common hand therapy questions were entered into ChatGPT version 3.5. The first five answer tabs that appeared with a Google search for the same four questions were downloaded. Three certified hand therapists blindly graded ChatGPT and Google’s answers using Likert scales to assess for answer accuracy (0-6), comprehensiveness (0-3), and conciseness (0-3).

Results

ChatGPT was significantly more accurate, with an estimated marginal mean (EMM) of 5.75 (95% CI: 4.96, 6.54) compared to Google’s 3.48 (95% CI: 2.86, 4.10) (p < 0.001). ChatGPT was significantly more complete, with an EMM of 2.50 (95% CI: 2.10, 2.90) compared to Google’s 1.48 (95% CI: 1.19, 1.77) (p < 0.001). ChatGPT was significantly more concise, with an EMM of 3.00 (95% CI: 2.66, 3.34) versus 1.60 (95% CI: 1.29, 1.91) for Google (p < 0.001).

Conclusion

ChatGPT is a concise, comprehensive, and accurate alternative to a Google search for people seeking information on hand therapy. The free version of ChatGPT does not update its sourcing past 2019, and the software is known to occasionally present false information. Frequently updated academic websites should therefore remain the primary online medical resource for patients.

Keywords

orthopaedics hand therapy occupational therapy wrist hand

Get full access to this article

View all access options for this article.

References

Dreyer

. ChatGPT for lawyers: embracing AI to enhance your legal practice, 2023.

Ghorashi

Ismail

Ghosh

, et al. AI-Powered chatbots in medical education: potential applications and implications. Cureus 2023; 15: e43271. https://doi.org/10.7759/cureus.43271

Nazir

Wang

. A comprehensive survey of ChatGPT: advancements, applications, prospects, and challenges. Meta Radiol 2023; 1: 100022. https://doi.org/10.1016/j.metrad.2023.100022

ChatGPT sets record for fastest-growing user base - analyst note. February 2. (Reuters, 2023).

McCarthy

Scott

Courtney

, et al. What did you google? Describing online health information search patterns of ED patients and their relationship with final diagnoses. West J Emerg Med 2017; 18: 928–936. https://doi.org/10.5811/westjem.2017.5.34108

Nolke

Mensing

Kramer

, et al. Sociodemographic and health-(care-)related characteristics of online health information seekers: a cross-sectional German study. BMC Public Health 2015; 15: 31. https://doi.org/10.1186/s12889-015-1423-0

Luo

Qin

Yuan

, et al. The effect of online health information seeking on physician-patient relationships: systematic review. J Med Internet Res 2022; 24: e23354. https://doi.org/10.2196/23354

Ren

Deng

Hong

, et al. Health information in the digital age: an empirical study of the perceived benefits and costs of seeking and using health information from online sources. Health Inf Libr J 2019; 36: 153–167. https://doi.org/10.1111/hir.12250

Tan

Goonawardene

. Internet health information seeking and the patient-physician relationship: a systematic review. J Med Internet Res 2017; 19: e9. https://doi.org/10.2196/jmir.5729

10.

Bhattacharyya

Miller

Bhattacharyya

, et al. High rates of fabricated and inaccurate references in ChatGPT-Generated medical content. Cureus 2023; 15: e39238. https://doi.org/10.7759/cureus.39238

11.

Daher

Koa

Boufadel

, et al.

Breaking barriers: can ChatGPT compete with a shoulder and elbow specialist in diagnosis and management?

JSES Int 2023; 7: 2534–2541. https://doi.org/10.1016/j.jseint.2023.07.018

12.

Goodman

Patrinely

Stone

Jr , et al. Accuracy and reliability of chatbot responses to physician questions. JAMA Netw Open 2023; 6: e2336483. https://doi.org/10.1001/jamanetworkopen.2023.36483

13.

Crook

Park

Hurley

, et al. Evaluation of online artificial intelligence-generated information on common hand procedures. J Hand Surg Am 2023; 48: 1122–1127. https://doi.org/10.1016/j.jhsa.2023.08.003

14.

Kung

Marshall

Gauthier

, et al. 3rd. Evaluating ChatGPT performance on the orthopaedic In-Training examination. JB JS Open Access 2023; 8: e23. https://doi.org/10.2106/JBJS.OA.23.00056

15.

Lum

. Can artificial intelligence pass the American board of orthopaedic surgery examination? Orthopaedic residents versus ChatGPT. Clin Orthop Relat Res 2023; 481: 1623–1630. https://doi.org/10.1097/CORR.0000000000002704

16.

Massey

Montgomery

Zhang

. Comparison of ChatGPT-3.5, ChatGPT-4, and orthopaedic resident performance on orthopaedic assessment examinations. J Am Acad Orthop Surg 2023; 31: 1173–1179. https://doi.org/10.5435/JAAOS-D-23-00396

17.

Casey

Dworkin

Winschel

, et al. ChatGPT: a concise google alternative for people seeking accurate and comprehensive carpal tunnel syndrome information. Hand Surg Rehabil 2024; 43: 101757. https://doi.org/10.1016/j.hansur.2024.101757

18.

Fox

. The social life of health information. Pew Research Center.

19.

Fraval

Ming Chong

Holcdorf

, et al. Internet use by orthopaedic outpatients - current trends and practices. Australas Med J 2012; 5: 633–638. https://doi.org/10.4066/AMJ.2012.1530

20.

Giorgino

Alessandri-Bonetti

Luca

, et al. ChatGPT in orthopedics: a narrative review exploring the potential of artificial intelligence in orthopedic practice. Front Surg 2023; 10: 1284015. https://doi.org/10.3389/fsurg.2023.1284015

21.

McGowan

Gui

Dobbs

, et al. ChatGPT and bard exhibit spontaneous citation fabrication during psychiatry literature search. Psychiatry Res 2023; 326: 115334. https://doi.org/10.1016/j.psychres.2023.115334

22.

Chung

Oden

Joyner

, et al. Safe infant sleep recommendations on the internet: let's google it. J Pediatr 2012; 161: 1080–1084. https://doi.org/10.1016/j.jpeds.2012.06.004

23.

Steffens

Koob

. [Diagnosis and therapy of tendovaginitis of the extensor carpi ulnaris (stenosis of the 6th extensor compartment)]. Z Orthop Ihre Grenzgeb 1994; 132: 437–440. https://doi.org/10.1055/s-2008-1039850

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.50 MB

0.00 MB