We propose a 10-item questionnaire to assess the impact of a technology or process change on clinician user experience. Questions are slightly modified from peer-reviewed, validated questionnaires that assess the aspects of usability, efficiency, cognitive demand, workflow, alignment with intended work practice, and workarounds. Twenty-eight projects implementing a technology or process change rated the applicability of the questionnaire. The questionnaire was judged applicable and useful for the majority (80%) of the projects, with some questions not relevant to interventional radiology.
BlijlevenV.HoxhaF.JaspersM. (2022). Workarounds in electronic health record systems and the revised sociotechnical electronic health record workaround analysis framework: scoping review. Journal of Medical Internet Research, 24(3), e33046. https://doi.org/10.2196/33046
3.
BoonstraA.JonkerT. L.van OffenbeekM. A.VosJ. F. (2021). Persisting workarounds in electronic health record system use: types, risks and benefits. BMC Medical Informatics and Decision Making, 21(1), 183. https://doi.org/10.1186/s12911-021-01548-0
4.
CarayonP.HundtA. S.KarshB. T.GursesA. P.AlvaradoC. J.SmithM.BrennanP. F. (2006). Work system design for patient safety: The SEIPS model. BMJ Quality & Safety, 15(suppl 1), i50–i58. https://doi.org/10.1136/qshc.2005.015842
5.
DeVellisR. F. (2017). Scale development: Theory and applications (4th ed.). SAGE Publications, Inc.
6.
DillmanD. A. (2011). Mail and internet surveys: The tailored design method—2007 Update with new Internet, visual, and mixed-mode guide. John Wiley & Sons. https://doi.org/10.1002/9780470276385
7.
FlanaganM.ArbuckleN.SaleemJ. J.MilitelloL. G.HaggstromD. A.DoebbelingB. N. (2011). Development of a workflow integration survey (WIS) for implementing computerized clinical decision support. In AMIA annual symposium proceedings (Vol. 2011, p. 427). American Medical Informatics Association.
8.
GarabedianP. M.RuiA.VolkL. A.NevilleB. A.LipsitzS. R.HealeyM. J.BatesD. W. (2023). A multiyear survey evaluating clinician electronic health record satisfaction. Applied Clinical Informatics, 14(4), 632–643. https://doi.org/10.1055/s-0043-1770900
9.
GreenP. E.SrinivasanV. (1990). Conjoint analysis in marketing research: New developments and directions. Journal of Marketing, 54(4), 3. https://doi.org/10.2307/1251756
10.
HalbeslebenJ. R.RathertC.BennettS. F. (2013). Measuring nursing workarounds: tests of the reliability and validity of a tool. JONA: The Journal of Nursing Administration, 43(1), 50–55. https://doi.org/10.1097/NNA.0b013e31827860ff
11.
HartS. G.StavelandL. E. (1988). Development of NASA-TLX (task load index): Results of empirical and theoretical research. In HancockP. A.MeshkatiN. (Eds.), Human mental workload (pp. 139–183). Elsevier Science Publishers B.V.
12.
HeroutJ.MaddoxK.NevilleK. J.PattersonE. S.PerryS. J.RothE. M. (2024, September). Assessing Clinician Experience: Contributions from Human Factors. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting68(1) (pp. 504–509). SAGE Publications.
13.
HollnagelE.WoodsD. D. (1983). Cognitive systems engineering: New wine in new bottles. International Journal of Man-Machine Studies, 18(6), 583–600. https://doi.org/10.1016/S0020-7373(83)80034-0
14.
International Organization for Standardization (ISO). (2024). Ergonomics of human-system interaction — Part 115: Guidance on conceptual design, user-system interaction design, user interface design and navigation design. (ISO 9241-115:2024)
15.
International Organization for Standardization (ISO). (2023). Systems and software engineering—Systems and software quality requirements and evaluation (SQuaRE): General framework for Common Industry Format (CIF) for usability-related information (ISO/TR 25060:2023).
16.
International Organization for Standardization (ISO). (2018). ISO 9241-11: Ergonomics of human-system interaction—Part 11: Usability: Definitions and concepts.
17.
International Organization for Standardization (ISO). (2017). ISO 10075-1:2017: Ergonomic principles related to mental workload — Part 1: General issues and concepts, terms and definitions.
KoppelR.WetterneckT.TellesJ. L.KarshB. T. (2008). Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. Journal of the American Medical Informatics Association, 15(4), 408–423. https://doi.org/10.1197/jamia.M2616
20.
LaugwitzB.HeldT.SchreppM. (2008). Construction and evaluation of a user experience questionnaire. In HolzingerA. (Ed.), HCI and usability for education and work (pp. 63–76). Springer Berlin Heidelberg.
21.
LewisJ. R. (1995). IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction, 7(1), 57–78. https://doi.org/10.1080/10447319509526110
22.
LundA. M. (2001). Measuring usability with the use questionnaire. Usability interface, 8(2), 3-6.
23.
McCordJ. L.LippincottC. R.AbreuE.SchmerC. (2022). A systematic review of nursing practice workarounds. Dimensions of Critical Care Nursing, 41(6), 347–356. https://doi.org/10.1097/DCC.0000000000000549
24.
PattersonE. S. (2018). Workarounds to intended use of health information technology: a narrative review of the human factors engineering literature. Human factors, 60(3), 281–292. https://doi.org/10.1177/0018720818762546
25.
PerrigS. A.AeschbachL. F.ScharowskiN.von FeltenN.OpwisK.BrühlmannF. (2024). Measurement practices in user experience (UX) research: A systematic quantitative literature review. Frontiers in Computer Science, 6, 1–17. https://doi.org/10.3389/fcomp.2024.1368860
van der VeenW.TaxisK.WoutersH.VermeulenH.BatesD. W.Van Den BemtP. M. L. A. (2020). Factors associated with workarounds in barcode-assisted medication administration in hospitals. Journal of Clinical Nursing, 29(13–14), 2239–2250. https://doi.org/10.1111/jocn.15217
28.
ZhangJ.WaljiM. F. (2011). TURF: toward a unified framework of EHR usability. Journal of biomedical informatics, 44(6), 1056–1067. https://doi.org/10.1016/j.jbi.2011.08.005
