Sage Journals: Discover world-class research

Abstract

Patient-reported outcome measures lead to better communication and decision-making between clinicians and patients. Applying a web-based repository system for data collection was previously suggested, but such system is not available. This article introduces the development and implementation of a new web-based application, PROMsBase, in orthopaedics clinical practice. PROMsBase was developed using a web interface, allows access using both desktop and mobile devices. Between 2013 and 2016, a total of 3192 pre-surgery questionnaires were collected. In total, 238 patients completed their post-surgery questionnaire online from home. PROMsBase was well embedded into routine practice without disrupting clinical workflow and overloading clinicians’ and researchers’ workload. Tablets were not more useful and only 37 per cent of the patients completed the questionnaire online. PROMsBase provided a platform to easily collect and store data in clinical practice. If properly integrated, this could promote better care and communication between providers and patients.

Keywords

health information on the web patient care patient-reported outcome measures

Introduction

Patient-reported outcome measures (PROMs)¹ are patient self-assessment reporting that allow clinicians to assess and monitor individual patients without the clinician intervention. PROMs can be used in clinical practice as initial or one-time screening for conditions and can later monitor patients’ progress and facilitate identification of poor outcomes.² In orthopaedics, for example, PROMs may be used to evaluate the effect of total knee replacement over time with feedback to clinicians on joint-specific clinical management as well as more generalized health-related quality of life.³ Evidence shows that the systematic use of information from PROMs leads to better communication and decision-making between clinicians and patients and improves patient satisfaction with care.^4–6

Attempts to embed measurement of patient-reported outcomes into routine practice revealed many technical, social, cultural, legal, and logistical barriers to successful adoption.^7,8 To be successful, the PROMs intervention must be acceptable to clinicians, add value to the clinical encounter, be affordable, and not disrupt clinical workflow. Previous studies identified that future research needed to recognize ways in which PROMs can be better incorporated into the routine care of patients by combining qualitative and quantitative methods and adopting appropriate trial designs.²

In orthopaedics, the New Zealand Joint Registry collects information using the validated Oxford Hip and Knee outcomes questionnaires.^9,10 The questionnaires are mailed to 20 per cent of registered patients 6 months following surgery and then at five yearly intervals with response rate between 70 and 75 per cent. Key limitations to the registry are that it is not web-based and the use of Microsoft Access 97 as the database program to store the data. Microsoft Access is useful for individual departments or small-to-medium business sectors where usage is less than 2 GB, otherwise performance becomes slow. Its structured query language (SQL) is not as robust as MySQL server or Oracle, and all the information from the database is saved into one file that slows down reports, queries, and forms.

A web-based repository system with a robust database engine that allows secured online access and data sharing which offers methods for researchers and clinicians to solve practical problems relevant to health care research or health care delivery would have significant advantages. Van den Berg et al.¹¹ suggested that researchers should strongly consider using web-based questionnaires. Questionnaires that are completed online (Web-based) using personal computers or tablets are less time consuming and less costly.¹² The use of a web-based repository portal was suggested to overcome several of these problems contributing to an improved use of PROMs in clinical practice.¹³ The developed web-based repository will create one centralized database that can store data from several clinics and improve sharing, reduce data entry errors, improve data retrieval, and reduce the time to search specific information. There are no reports found utilizing web-based PROMs in orthopaedics. In paediatric, however, Haverman et al.,¹⁴ reported the development of a web-based application for the use of PROMs in daily clinical practice. The authors concluded that the application is efficient and contributed to systematically monitor and discuss patient issues during consultations.

The purpose of this article is to introduce the development and implementation of a new web-based application, PROMsBase, in orthopaedics clinical practice. In this article, we describe the development, functionality, and process of PROMsBase and how we apply it in a routine clinical practice at both public hospital and private orthopaedic clinic. We hope that it can be a guideline for the researchers and clinicians who plan to establish similar PROMs in their own practice.

Development

The PROMsBase repository has been developed by IT and database specialists to store and access datasets using a web interface. The IT personnel were also responsible for upgrading and administrating the data. The web interface allows access using both desktop and mobile devices such as tablets. Clinicians can sign in from any device and computer, needing no software other than a standard web browser. Several concepts underpin these developments: (1) authentication and audit trail and security, (2) anonymity of patient metadata, (3) visualization of PROMs questionnaire for tablets and touch screens, (4) allowing patients to complete PROMs online from home, and (5) availability of pooled results to allow downloading for use in other software applications such as statistical analysis programs.

There are fundamental requirements for any web-based repository system to be successful in data collection. These requirements consist of system structure, type of data stored, and mechanisms for storing data. The relational database with the two-layer approach, including a database engine and a data processing application, was reported to best match system structure design – avoiding problems with program updates and multi-user environments.^15,16

Access

For the public hospital, PROMsBase is accessed from the hospital intranet network, and for the private clinic from https://hipscope.com.au/db/. Access is performed using private username and password. To fulfil the requirements on security and ownership, PROMsBase utilizes access to data by registration. Users must register to use PROMsBase by sending request to the authors who are responsible of upgrading and administrating the database and its interface. Future development will allow other research groups and clinics to create their own environment, which they can administer and negotiate data sharing with possible partners.

Authentication and audit trail

PROMsBase implements authentication to validate the identity of end users who log in to the system. The internal table-based authentication method is used, which utilizes the storage of username/password pairs in a database table. For security reasons, the password in the database table is not stored as plain text but as an encrypted one-way hash of the password. The system implements an auto-logout 20-min time setting and will automatically log a user out of the system if they have not had any activity (e.g. clicking, typing, and moving the mouse) on their current web page for the set amount of time. This prevents someone else from accessing their account and their project data if they leave a workstation without properly logging out or closing their browser window.

PROMsBase has a built-in audit trail that automatically logs all user activity. Whether the activity be entering new data or modifying a field, PROMsBase logs all actions. The logging record is saved in a separate text file that can be viewed by the administrator.

Security

To help protect and secure the data stored in PROMsBase back end database, various methods are used to protect against malicious users who may attempt to identify and exploit any data in the system. The server uses a Secure Sockets Layer protocol converting the data into virtually impenetrable code using data encryption up to 256 bits. SQL queries sent to the database server from PROMsBase are all properly escaped before being sent. Data used within SQL queries have their data type checked to prevent any mismatching of data types. Before the data is displayed to the screen, it is escaped, which further assists in attacks, such as cross-site scripting (XSS) and SQL injection. Data backup is performed automatically each night saving the backup data into a separate server.

Database structure and content

PROMsBase has two components: the database engine which resides on a server and a web-based graphical user interface (GUI). The database management system is MySQL (www.mysql.com), which supports relational database design, is freely available and widely used. The web GUI is built up utilizing the Ext JS JavaScript application framework (www.sencha.com) and is coded in PHP (PHP: Hypertext Pre-processor, www.php.net), hypertext markup language (HTML), cascading style sheet (CSS), and JavaScript.

The database contains metadata describing the users, patients, list of procedures, patients’ procedures, and related PROMs questionnaires. The following validated PROMs questionnaires were chosen: (1) Oxford Knee Score (OKS), (2) Oxford Hip Score (OHS), (3) International Hip Outcome Tool (iHOT33), (4) The Manchester–Oxford Foot Questionnaire (MOxFQ), and (5) Disabilities of the Arm, Shoulder, and Hand (DASH). Example of the metadata describing the patient, their procedure, and PROMs is listed in Table 1.

Table 1.

Example of metadata describing the patient, procedure, and their completed PROMs of the Oxford Hip Score questionnaire.

Table	Field	Example data
Patient	ID (auto increment)	1874
	UR (patient unit record number)	01184
	Gender	M
Patient procedures	ID (auto increment)	725
	UR (patient unit record number)	01184
	Procedure	Adult: Hip: Elective: THR
	Procedure date	13 August 2015
	Side	L
Oxford Hip Score	ID (auto increment)	284
	UR (patient unit record number)	01184
	Collection date	24 May 2015
	Side	L
	Procedure	Adult: Hip: Elective: THR
	q1	2
	q2	2
	q3	1
	q4	0
	q5	1
	q6	4
	q7	0
	q8	1
	q9	0
	q10	2
	q11	1
	q12	4
	Score	18
	Event	Pre-op

THR: total hip replacement.

Ethical and legal issues

Clearly, it should not be possible to identify individuals from the data stored within PROMsBase. All data are de-identified. PROMsBase is managed by the orthopaedic unit. Our institutional Ethics in Human Research Committee has granted its approval in collecting PROMs data (QA2013.114). Patient details such as first name, surname, phone number, and email address are kept in separated secured database. Manually matching patient unit record (UR) number with the corresponding UR in PROMsBase allows future patient contact, such as sending web links to access PROMs online.

Data entry

Pre-surgery data entry

All patients that visit the clinic and are scheduled for a surgery are asked to complete a PROMs questionnaire relevant to their procedure. The clinic scholar selective students and registrars were the clinicians responsible for patient interaction and data collection. Before the patient can complete the questionnaire, the clinician must enter the patient UR number and type of scheduled surgery, for example, ‘Total Hip Replacement’. Based on this information, the web application highlights the appropriate PROMs questionnaire such that the clinician knows what PROMs to give to the patient. The ‘pre-surgery’ time event is then assigned to the patient’s PROMs questionnaire record. The time event assists in identifying when the questionnaire was completed in relation to the date of operation. Patients may complete the questionnaire using tablet device, but a paper-based questionnaire option is provided if they are uncomfortable using tablet technology. If paper-based questionnaire is chosen, the data is later manually entered by the clinician or researcher into the database.

Post-surgery data entry

Patients who are scheduled for a post-surgery PROMs questionnaire are tracked using the ‘Scheduled’ functionality. With the ‘Scheduled’ feature, the clinician can list patients that are due for PROMs by selecting from the list a time point/event and the type of PROMs questionnaire. Upon selection, the database searches for all patients who are due for the selected event and PROMs. For example, a clinician who is interested in finding all patients due to complete 1 year post-surgery OKS questionnaire will select 1 year post-surgery for the event and OKS for the PROMs questionnaire. The output is displayed in a tablature format showing patient’s UR, operated limb, surgery date, surgery type, number of days from surgery, pre-surgery, 6 weeks post-surgery, 6 months post-surgery, and 1 year post-surgery PROMs results if completed (see Figure 1).

Figure 1.

PROMsBase example of the ‘Scheduled’ feature output for all patients completed Oxford Knee Score questionnaire and are due for 1 year post-surgery questionnaire.

Using the ‘Scheduled’ output, the clinician can select a patient UR to display their contact details. The patient’s contact details include whether the patient prefers to complete the PROMs questionnaire by a phone call or online using the web from home. A link to the web online PROMs questionnaire is emailed when the patient prefers the online option. The web online PROMs questionnaire displays each PROMs question and related answers at a time with a next and previous buttons. Only once the answer to the question is selected will the ‘next’ button be enabled allowing the patient to continue and display the following question and answers (see Figure 2).

Figure 2.

Online Oxford Knee Score questionnaire (question 8) for patients preferring to complete the questionnaire from home.

Displaying information

Information on an individual patient is displayed in table format. Upon entering patient’s UR number, information on previous and future surgeries is displayed with related PROMs data (see Figure 3). It is possible to filter patients by the type of PROMs. PROMs filter is selected from a list of available PROMs questionnaires that upon a filter being selected, a list of all patients that filled out that specific PROMs questionnaire is listed. It is then possible to select a patient by single mouse click to display their past and future procedures and completed PROMs data.

Figure 3.

The PROMsBase web interface showing individual patient surgical procedures and completed Oxford Knee Score PROMs before surgery (BokkedWL) and following 6 months post-surgery.

Implementation

PROMsBase was tested in a public hospital for pre-surgery assessment and a private orthopaedic clinic during both pre- and post-surgery assessments. Ethics to collect data and store in PROMsBase was approved from the hospital ethical committee #QA2013.114.

Public hospital

In the public hospital during initial outpatient consultation, every patient who was scheduled for hip, knee, shoulder, or ankle/foot procedures was asked to complete the OHS, OKS, DASH, and MOxFQ questionnaires, respectively. The OHS,⁹ OKS,¹⁷ DASH,¹⁸ and MOxFQ^19,20 questionnaires are frequently used in orthopaedics and were shown to be valid. Between June 2013 and May 2016, a total of 3192 pre-surgery PROMs were collected and entered in PROMsBase with an average of 90 ± 41 questionnaires collected monthly (Figure 4). For the OKS and OHS, there are 12 questions with the scores ranging from 4 to 0. A total score of 48 is the best, indicating normal function. A score of 0 is the worst, indicating the most severe disability. In addition, the total score can be grouped according to the system published by Kalairajah et al.²¹ These groups each score into four categories: Category-1 >41 – excellent, Category-2 between 34 and 41 – good, Category-3 between 27 and 33 – fair, and Category-4 <27 – poor. Mean score from 1061 completed pre-surgery OKS questionnaires was 19.9 ± 10.6, which is categorized as poor. Similar pre-surgery scores were found from 493 OHS questionnaires with mean total score of 15.7 ± 10.2.

Figure 4.

Total PROMs (OKS, OHS, DASH, MOxFQ, and iHOT33) collected each month from June 2013 to May 2016.

The MOxFQ questionnaire consists of 16 items measuring three domains: (1) walking/standing (7 items), pain (5 items), and social interaction (4 items). Response options consist of a 5-point Likert scale ranging from no limitation to maximum limitation. Scores for each domain are calculated by summing the responses to each item within a given domain. Raw scores are converted to a 0–100 metric where 100 corresponds to most severe. Mean total score for the MOxFQ questionnaire was 70.5 ± 20.4 calculated from 247 patients. For the three domains, mean scores were 70.1 ± 25.4, 67.3 ± 24.1, and 63.9 ± 26.9 for the walking/standing, pain, and social, respectively.

The DASH Outcome Measure is a self-report questionnaire designed to measure physical function and symptoms in patients with any or several musculoskeletal disorders of the upper limb. The DASH Outcome Measure is scored in two components: the disability/symptom section (30 items, scored 1–5) and the optional high-performance sport/music or work section (4 items, scored 1–5). For the first component, at least 27 of the 30 items must be completed for a score to be calculated. The total disability score is transformed to be out of 100 having a higher score indicating greater disability. Using the PROMs database, 326 DASH questionnaires have been collected pre-surgery with a mean disability score of 46.5 ± 26.8.

Private orthopaedic clinic

Further exploration of the PROMsBase functionality was achieved by implementing the system in a private orthopaedic practice that specializes in hip arthroscopy. We were interested to know the viability and efficiency of the system to collect the iHOT33 questionnaire from prospective hip arthroscopy patients; first at pre-surgery and second at post-surgery using the web online tool. The iHOT33 was developed for active patients aged between 18 and 60 years presenting with a variety of hip pathologies. The iHOT33 questionnaire is comprised of 33 questions relating to (1) symptoms and functional limitations, (2) sports and recreational activities, (3) job-related concerns, and (4) social, emotional, and lifestyle concerns. Patients are asked to place a mark on a Visual Analogue Scale line for each question, with scores ranging from 0 to 100 having ‘significantly impaired’ on the far left [0], and ‘no problems at all’ on the far right [100]. For pre-surgery, 1049 patients completed the iHOT33 with mean score of 36.6 ± 20.3. From 238 patients that initially consent using the post-surgery web interface, a web link to an iHOT33 questionnaire was sent by email to each patient allowing the completion of the questionnaire from home. Only 37 per cent of the patients completed the questionnaire using the online web feature.

Discussion

Patient-reported outcome scores are based on direct reporting by patients without the intervention of an observer. They include the self-assessment of functional status, symptoms, and other concerns such as needs and satisfaction with care. In order to improve patient care at our orthopaedic department, we implemented PROMs and developed a web-based repository portal (PROMsBase) to assist clinician with data collection. Furthermore, we implemented the system in an orthopaedic private clinic specializing in hip arthroscopy to assist in data collection and analysis pre- and post-surgery. The purpose of this article was to provide a ‘proof of concept’, describing the development and implementation of the system as guidelines for clinical researchers who plan to establish similar PROMs in orthopaedic and/or different domains.

We found PROMsBase to be substantially helpful in data collection. Clinicians from different locations were able to log in from any computer. Clinicians reported that the system was user friendly with minimal instructions needed for new users. PROMsBase was well embedded into our routine practice allowing the collection of large amount of PROMs data without disrupting clinical workflow and overloading clinicians’ and researchers’ workload.

The structure and the functionality of PROMsBase fully meet the requirements discussed earlier. It provides security and patient’s confidentiality with the functionality of allowing access to the system from virtually any computer. Data stored in PROMsBase include outcome measures from the most accepted lower and upper limb questionnaires within the orthopaedic community. Researchers found the capacity to monitor when patient is due to post-surgery PROMs and prepare a web link to online questionnaire to be sent to patient very useful.

The main contribution of the PROMsBase was providing a platform to easily store more than 3000 collected PROMs questionnaires from majority of patients who visit the clinic for first consultation and are due for surgery. The ability to collect and interrogate large amounts of PROMs data is important to aid management of patients and assess quality of care. Interestingly, we did not find the tablet more useful than using hard copy questionnaire in our diverse patient population. Patients preferred completing the PROMs questionnaire using paper-based questionnaires instead of tablets. This may be due to patient’s language difficulties or unfamiliarity with tablets due to age or demographic background. The implementation of information and communication technologies (ICTs) in clinical settings is a new emerging research area that needs further studies and tests to reduce possible digital divides. In recent systematic review, Gagnon et al.²² suggested that perception of the benefits of the system usefulness was the most common facilitating factor, followed by ease of use. Issues regarding design, technical concerns, familiarity with ICT, and time were the most frequent limiting factors identified. Further studies are needed to reduce limiting factors found in the use of digital devices, developing better health information literacy tactics to increase the number of patients’ participation. Meantime, it is suggested that during the patient’s PROMs first-time collection, having at least one clinician to assist is important regardless if it is paper-based questionnaires or tablet.

The system allows screening and monitoring interventions by collecting data post-surgery. We have implemented and tested a post-surgery online interface to allow the patients to complete the PROMs online at their own computer and most convenient time and place. Unfortunately, only 37 per cent of patients responded and completed the online PROMs. This is surprising, as we expected that connecting to PROMs from home and filling out the questionnaire from home will be more appealing and easier to patients. It may be that non-responders were patients who substantially were doing well following surgery and, therefore, less likely to spare time for this. Future work should utilize the System Usability Scale and conduct a user satisfactory survey to test whether PROMsBase is a good product and to elucidate why response rate was low.

In a broader sense, PROMsBase is a patient-measuring outcome repository system that offers methods for researchers and clinicians to solve practical problems relevant to health care research or health care delivery. It provides a knowledge-based system that is available when needed, easy to aggregate, and produces significant clinical value. If properly integrated, this could promote better care and communication between providers and patients. Future work should explore possibilities of sharing data between public and private practices. This, for example, may be implemented by integrating PROMsBase with the Australian National E-Health Transition Authority e-Health system. The authors believe that the main barrier lies in the regulations, ethical considerations, and implementation rather than to system functionality.

Clinical relevance statement

Information from PROMs leads to better communication and decision-making between clinicians and patients and improves patient satisfaction with care. PROMsBase simplifies the collection of PROM questionnaires within public and private orthopaedic clinics before and after surgery. It improves patient care through enhanced use of information system, efficient data collection, and analysis for research.

Footnotes

Author’s note

Phong Tran is also affiliated to Australian Institute for Musculoskeletal Science (AIMSS), The University of Melbourne and Western Health, St. Albans, VIC, Australia.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Basch

New frontiers in patient-reported outcomes: adverse event reporting, comparative effectiveness, and quality assessment. Annu Rev Med 2014; 65: 307–317.

Greenhalgh

The applications of PROs in clinical practice: what are they, do they work, and why?

Qual Life Res 2009; 18: 115–123.

Wylde

Blom

Whitehouse

, et al. Patient-reported outcomes after total hip and knee arthroplasty: comparison of midterm results. J Arthroplasty 2009; 24: 210–216.

Chen

Hollis

SJ.

A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res 2013; 13: 211.

Nelson

Eftimovska

Lind

, et al. Patient reported outcome measures in practice. BMJ 2015; 350: g7818.

Santana

Feeny

Framework to assess the effects of using patient-reported outcome measures in chronic care management. Qual Life Res 2014; 23: 1505–1513.

Aaronson

Snyder

Using patient-reported outcomes in clinical practice: proceedings of an International Society of Quality of Life Research conference. Qual Life Res 2008; 17: 1295.

Dawson

Doll

Fitzpatrick

, et al. The routine use of patient reported outcome measures in healthcare settings. BMJ 2010; 340: c186.

Dawson

Fitzpatrick

Carr

, et al. Questionnaire on the perceptions of patients about total hip replacement. J Bone Joint Surg Br 1996; 78: 185–190.

10.

Xie

Zhang

, et al. Extension from inpatients to outpatients: validity and reliability of the Oxford Knee Score in measuring health outcomes in patients with knee osteoarthritis. Int J Rheum Dis 2011; 14: 206–210.

11.

Van den Berg

Overbeek

Van der Pal

, et al. Using web-based and paper-based questionnaires for collecting data on fertility issues among female childhood cancer survivors: differences in response characteristics. J Med Internet Res 2011; 13: e76.

12.

Heiervang

Goodman

Advantages and limitations of web-based surveys: evidence from a child mental health survey. Soc Psychiatry Psychiatr Epidemiol 2011; 46: 69–76.

13.

Guyatt

Ferrans

Halyard

, et al. Exploration of the value of health-related quality-of-life information from clinical research and into clinical practice. Mayo Clin Proc 2007; 82: 1229–1239.

14.

Haverman

Engelen

Van Rossum

, et al. Monitoring health-related quality of life in paediatric practice: development of an innovative web-based application. BMC Pediatr 2011; 11: 3.

15.

Berghof

Zwick

Gaitbase – a relational database for the management of gait laboratory data. Gait Posture 1995; 3: 291–292.

16.

Tirosh

Baker

McGinley

GaitaBase: web-based repository system for gait analysis. Comput Biol Med 2010; 40: 201–207.

17.

Dawson

Fitzpatrick

Murray

, et al. Questionnaire on the perceptions of patients about total knee replacement. J Bone Joint Surg Br 1998; 80: 63–69.

18.

Hudak

Amadio

Bombardier

. Development of an upper extremity outcome measure: the DASH (Disabilities of the Arm, Shoulder and Hand) [corrected]. The Upper Extremity Collaborative Group (UECG). Am J Ind Med 1996; 29: 602–608.

19.

Dawson

Coffey

Doll

, et al. A patient-based questionnaire to assess outcomes of foot surgery: validation in the context of surgery for hallux valgus. Qual Life Res 2006; 15: 1211–1222.

20.

Garrow

Papageorgiou

Silman

, et al. Development and validation of a questionnaire to assess disabling foot pain. Pain 2000; 85: 107–113.

21.

Kalairajah

Azurza

Hulme

, et al. Health outcome measures in the evaluation of total hip arthroplasties – a comparison between the Harris hip score and the Oxford Hip Score. J Arthroplasty 2005; 20: 1037–1041.

22.

Gagnon

Desmartis

Labrecque

, et al. Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals. J Med Syst 2012; 36: 241–277.

PROMsBase: Web-based repository portal for patient-reported outcome measures in orthopaedics

Abstract

Keywords

Introduction

Development

Access

Authentication and audit trail

Security

Database structure and content

Ethical and legal issues

Data entry

Pre-surgery data entry

Post-surgery data entry

Displaying information

Implementation

Public hospital

Private orthopaedic clinic

Discussion

Clinical relevance statement

Footnotes

Author’s note

Declaration of conflicting interests

Funding

References