Abstract
Background:
Capturing the variability that exists among patients attending an amputee clinic using standardized paper-based questionnaires is time-consuming and may not be practical for routine clinical use. Electronic questionnaires are a potential solution; however, the benefits are dependent on the feasibility and acceptance of this mode of data collection among patients.
Objective:
To determine the feasibility and patient preference/comfort in using a tablet-based questionnaire for data collection in an outpatient amputee rehabilitation clinic compared to a traditional paper-based questionnaire.
Study design:
Observational study.
Methods:
In all, 48 patients with major extremity amputations completed both tablet and paper questionnaires related to their amputation and prosthetic use. Both trials were timed; patients then completed a semi-structured questionnaire about their experience.
Results:
In all, 20.5% of patients needed hands-on assistance completing the paper questionnaire compared to 20.8% for the tablet. The majority of participants (52.1%) indicated a preference for the tablet questionnaire; 64.6% of patients felt the tablet collected a more complete and accurate representation of their status and needs. In all, 70.8% of participants described themselves as comfortable using the tablet.
Conclusion:
Despite comorbidities, patients with amputations demonstrated excellent acceptance of the electronic tablet-based questionnaire. Tablet questionnaires have significant potential advantages over paper questionnaires and should be further explored.
Clinical relevance
A custom electronic questionnaire was found to be beneficial for routine clinic use and was well received by patients in an amputee rehabilitation clinic. Development of such questionnaires can provide an efficient mechanism to collect meaningful data that can be used for individual patient care and program quality improvement initiatives.
Background
Quality improvement in health care requires, among other items, “accurate and powerful” outcome measures to monitor change. 1 However, the cost and feasibility of collecting these outcomes may be the limiting factors preventing their use. Electronic tablet-based, rather than paper-based, questionnaires have the potential to overcome these barriers. 2 These electronic tablet data can be used immediately for point-of-care patient management and support cost-effective program review. 3
Amputee-specific outcome questionnaires have elements that are applicable to all patients with major limb deficiency (acquired or congenital), but data collection is typically inefficient as questions are not tailored to specific patient issues. For example, a 10-item questionnaire that focuses on prosthetic limb use and comfort may be irrelevant to individuals who primarily use wheelchairs. Likewise, a questionnaire that focuses on gait aid use and falls would not capture the pertinent information for an individual who uses their prosthesis for athletic endeavors. Given the heterogeneity of this population, standardized, comprehensive paper-based questionnaires designed to capture this variability are time-consuming and may not be practical for routine clinical use.
To address this issue, a novel, tablet-based, “smart” questionnaire, custom application, called the electronic Amputee Clinic Questionnaire (eACQ), was developed. The eACQ is intended to replace the original Amputee Clinic Questionnaire (ACQ), a single-page, locally developed questionnaire with eight question categories to be answered by patients via checkboxes prior to their clinic visit. Both questionnaires collect patient information fundamental to current prosthetic use that are reviewed by the interdisciplinary team at the patient visit to track progress and help direct patient care. However, the eACQ is designed to go beyond restrictions of a generic questionnaire that attempts to keep all questions relevant to all patients. The eACQ is an adaptive or smart questionnaire that only displays questions that are relevant to the patient based on responses to earlier questions. This format permits the eACQ to efficiently collect high-yield patient data while minimizing questionnaire fatigue and perform real-time interpretation and scoring of patient responses. The tablet questionnaire format is meant to minimize administrative burden to the same or less than that of a paper questionnaire and facilitate error-less data collection to support quality improvement within the amputee clinic.
Despite the presumed value of electronic questionnaires, the benefits depend on the feasibility and acceptance of this mode of data collection with the target patient population. In Canada and most of the developed world, the majority of amputations involve the lower limbs as a consequence of diabetes mellitus (with or without peripheral vascular disease). 4 Complications of diabetes include a number of impairments that make form completion challenging, such as retinopathy-related visual impairment,5,6 motor control and/or tactile dysfunction from neuropathy, 7 limited joint mobility in the hands, 8 and vascular disease impacting cognition. 9 The impact of these comorbidities on tablet use has not been reported. Therefore, the objective of this study was to determine the feasibility and patient preference/comfort in using a tablet-based questionnaire (eACQ) for data collection in an outpatient amputee rehabilitation clinic compared to a traditional paper-based questionnaire (ACQ).
Methods
Questionnaires
The eACQ app was iteratively designed over 18 months starting from the existing clinic questionnaire (ACQ), with intentions to efficiently capture high-yield patient capacity and performance related to their amputation and prosthetic use. A master flow algorithm was designed with a series of subsections called “loops” containing specific algorithms relating to various prosthetic concerns. Within these loops, smaller flowcharts of questions were added that provided the eACQ with the ability to follow-up or probe further. Loops were designed to highlight areas of prosthetic use and wearing time, patient-identified residual limb problems, phantom limb pain, falls, ambulation capacity, gait aid use, wheelchair use, and participation and satisfaction with the prosthesis. The need to complete any of the loops is determined by the response to prior questions. Thus, the eACQ is capable of probing into relevant concerns and avoiding irrelevant lines of questions. The eACQ provides a summary screen of data collected including automatic grading of the patient responses to standardized amputee rehabilitation outcome measures. At the time of the patient encounter, the physician reviews the eACQ report for that specific patient and then signs off a final screen indicating key outcome-related amputation parameters of etiology, level, and patient gender. A check box is required to indicate whether there is medical and prosthetic stability in the opinion of the treating physician.
Versions of the eACQ were trialed on amputee rehabilitation team members, clinic staff, and patients, for feedback pertaining to practical items such as holding the tablet, font size, security, and user interface. The application was developed using Apache Flex software. The Android operating system was selected for practical purposes to facilitate sharing between user groups. Five identical Asus TF300T 10.1 in tablets were used to facilitate flow in clinic. The eACQ uses a local WiFi connection to a single printer for optional printing, but otherwise external Internet and communication functions were disabled on all devices. At the end of the encounter, the questionnaire is reset. Data are manually downloaded off the tablet to an encrypted database; currently, there is no wireless connection with the patient electronic medical record.
Sample
A convenience sample of patients returning to a regional amputee rehabilitation clinic were invited to participate in this study over 5 weeks. Patients were eligible if they were 18 years of age or older and had not been previously exposed to using tablet computers for completing waiting room questionnaires.
Protocol
Individuals providing informed consent for participation were moved to an examination room to complete both the ACQ and eACQ. The order in which the questionnaires were administered was randomized, with the second questionnaire administered immediately following completion of the first. A research assistant was present during the administration of the questionnaires and provided assistance as needed; the type of assistance required for each questionnaire was recorded. Assistance was categorized as “minor” if patients asked for clarification with the task or questions and “major” if hands-on data entry help was provided. The time taken to complete each questionnaire was recorded; time commenced when the questionnaire was received and stopped when the last question was answered. Patients then completed a short questionnaire related to their prior electronic use and familiarity, as well as their impressions and preference of the eACQ compared to the ACQ. The outpatient clinic visit was then resumed. This study was approved by the Research Ethics Board of Western University.
Statistical analysis
Statistical analyses were conducted using Statistical Package for Social Sciences (SPSS; Version 23.0). Descriptive analyses (means and standard deviations (SDs), frequencies) were performed to determine the demographic characteristics of the sample, as well as questionnaire responses about the eACQ use. To consider the potential differences between those who prefer the tablet versus the paper questionnaires, or who have no preference, an analysis of variance (ANOVA) was conducted with means and Fisher’s exact tests were performed for categorical variables.
Results
Demographics
In total, 48 participants, ranging in ages from 19 to 91 years completed the study. Below knee amputations were most common (77.1%), with most amputations being the result of disease (Table 1). Participants took longer to complete the eACQ compared to the ACQ. The ACQ required 196 (SD 115 Seconds (s)), ranging from 62 to 585 s, to complete. The eACQ required 468 (SD 256 s), ranging from 136 to 1217 s, to complete.
Participant demographics (
ACQ: Amputee Clinic Questionnaire; eACQ: electronic Amputee Clinic Questionnaire; SD: standard deviation.
The count for etiologies exceeds 48 as some participants had multiple conditions.
Preference
The majority of participants indicated a preference to use the tablet at future visits (52.1%), with five participants (10.4%) indicating a preference for the paper questionnaire. Of these five, four did not feel the tablet provided a better description of their situation. Preference was not significantly different between genders, those with prior email experience, or those with prior tablet experience (Table 2).
Differences between those preferring the eACQ and the ACQ.
ACQ: Amputee Clinic Questionnaire; eACQ: electronic Amputee Clinic Questionnaire; SD: standard deviation.
Analysis of variance (ANOVA).
Fisher’s exact test.
Assistance provided
In our study, 56.3% of participants previously used touchscreen technology on a tablet or smartphone and 60.4% of participants used email on a regular basis. The need for assistance in completing the questionnaires was similar for both formats. Clarification level (minor) help was required for 18.8% of those using the tablets and 9.1% completing the paper questionnaires. Data entry assistance (major) was required in 20.8% of cases using the tablets and 20.5% of cases using the paper questionnaires.
Impressions of tablet use
Overall, 64.6% of patients felt that the eACQ collected a more complete and accurate representation of their status and needs at the time of the visit, and another 10.4% of patients indicated that they were unsure whether the tablet collected a better description of their situation. The remaining 25.0% did not feel it was advantageous. In all, 70.8% of participants described themselves as “comfortable” or “extremely comfortable” in using the eACQ format, while only one person described the tablet use as “uncomfortable.” Those reporting the eACQ collected a better description of their situation tended to report higher comfort levels using the tablet format.
Discussion
The International Society for Pharmacoeconomics and Outcomes Research ePRO (electronic patient-reported outcomes) Good Research Practices Task Force Report recommends that regardless of psychometrics, all newly introduced electronic patient report tools must be studied for usability. 3 This feasibility study was designed to meet this recommendation, as well as to appreciate the abilities and limitations of introducing a tablet into a clinic. The study found that although the electronic questionnaire took longer to complete, the majority of participants felt comfortable using the tablet (70.8%) and had a preference to continue using it in future visits. Moreover, the data collected were thought to be more complete and accurate by approximately two-thirds of the sample.
The demographic sample of study patients found high percentages of male patients (79%), patients with below-knee amputations (77%), and people with amputations as a result of diabetes and/or peripheral vascular disease (77%); these demographics are representative of both the local and North American epidemiology related to amputations. 4 The majority (52.1%) of participants preferred the tablet for future visits and 37.5% indicated equal preference suggesting that common comorbidities of neuropathy and retinopathy did not impede successful use of the eACQ. Twice as many participants required minor assistance in the form of clarification with the eACQ (18.8% vs 9.1%) which may be related to challenges of introducing and learning new technology. If this is the case, it is hypothesized that, with repeated use of the eACQ and increased familiarity with the technology in general, the need for minor level clarification in future visits may diminish. The need for hands-on major assistance, found to be equal for both paper and tablet formats, is unlikely to change regardless of questionnaire format as this assistance may be due to limitations from comorbidities rather than a user interface issue.
In addition to the preference for the eACQ, 70.8% of the sample reported being comfortable with eACQ format. Other studies demonstrated similar findings and report that while there may be some hesitation prior to the first trial, patients felt overwhelmingly comfortable with the electronic surveys upon completion and preferred to use this method in the future.10–13 Interestingly, others have found that younger patients and those with higher incomes and educational attainment were more likely to prefer the tablet rather than paper version of a feedback-based questionnaire. 14 In this study, the groups indicating preference for the eACQ verses the ACQ were similar in age.
Time to completion of the questionnaire was different between the eACQ and ACQ. It is important to highlight that the questionnaires studied were intentionally different in design. The eACQ followed a series of decision points in the algorithm that determined which questions would follow depending on prior responses. By design, more questions, and questions intended to collect greater detail, were included so that the increased time taken to complete the eACQ may be related to greater volume of collected information rather than difficultly with the user interface. Generally, electronic versions of other questionnaires completed on computers, tablets, or kiosks take the same amount of time 13 or less12,15 than identical paper versions.
Paper questionnaires have many limitations (font size, layout, and page limits) which often create artificial constraints on questionnaire design. Electronic questionnaires may be able to overcome these limitations while collecting data with similar reliability to paper versions.13,16,17 The major tradeoff of an electronic tablet, which restricts entries and requires completion of a section before continuing, relates to ad hoc personalization of data. For example, patient responses to the question “what gait aid do you usually use when walking outside?” on the paper questionnaire may include no checks, multiple checks, or a circle and arrows around multiple boxes with the comment “sometimes.” In contrast, the eACQ limits the response to one single boxed answer. When a question is intentionally passed over on a paper questionnaire, it may indicate that the question is not applicable to the patient, or there is uncertainty in how to answer, and an incorrect response may be averted. Comments written on paper questionnaires may provide great detail and insight into the status of the patient and are lost with the rigid structure of the electronic version. However, the majority of participants felt like the eACQ collected a more complete and accurate representation of their status and needs at the time of the visit. While the inability to add comments does potentially limit the richness of the data, the comprehensive nature and responsiveness of the questionnaire design appear to meet the needs of the majority of participants.
When questionnaires are overly personalized or deviate substantially from the intended data collection, there is increasing probability that the questionnaire cannot be adequately interpreted for the purposes of aggregate data or scoring algorithms. By restricting such personalization with the electronic version, grading algorithms can be consistently completed. In a paper versus tablet study of the Disabilities of the Arm, Shoulder, and Hand (DASH) questionnaire, the tablet version was 14 times more likely to be scorable by algorithm due to a higher completion rate of questions. 18 Electronic questionnaires also benefit with regard to data management given that once patients have completed the questionnaire, the data are instantly analyzed and entered into a database with zero errors, as the data coding and manual entry steps are eliminated.11,15
The differences in questionnaire design may complicate generalizations of the findings. Specifically, the fact that the eACQ was designed to be responsive to patient situations may showcase to the patient the potential of using electronic smart questionnaires. Therefore, it cannot be determined here whether patient preference toward tablet use would have been similar if the identical questionnaire was presented on a tablet versus paper. It appeared that the tradeoff in extra time to complete the tablet questionnaire was not a deterrent as long as the patient felt their status was being sufficiently explored. The extra time is attributed to greater question number and detail with the eACQ, but the study design here cannot exclude that the increased time was due to difficulty with the tablet hardware and interface.
Comorbidities were specifically not tracked or addressed. While there may be particular comorbidities that complicate tablet usage, the overall need for assistance with both formats was similar. A larger study addressing the impact of the number and type of comorbidities on performance with tablet technology would help better target the technology’s use in different populations. Furthermore, given the limited number of individuals preferring the paper questionnaire format, caution should be taken when interpreting the results. A similar study with a larger sample size would be beneficial.
Conclusion
Introducing a custom electronic questionnaire, the eACQ was well received by patients and clinical staff. Despite an increase in time to completion of the eACQ, no more major help was required from clinic staff compared to the standard ACQ, and it was equally accessible to all participants regardless of prior exposure to similar technology. Participants felt the eACQ was comfortable to use despite a range of comorbidities and indicated a preference to continue using the tablet in future visits. The eACQ continues to be used in the amputee clinic, with the ACQ available if desired.
Footnotes
Author contribution
All authors contributed equally in the preparation of this manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
We would like to acknowledge the Amputee Coalition of Canada for the Canadian Amputee Research Award and St. Joseph’s Health Centre in London, Ontario, Canada for the President’s Innovation Award, which funded this project.