Synoptic operative reports for spinal cord injury patients as a tool for data quality

Abstract

The advent of synoptic operative reports has revolutionized how clinical data are captured at the time of care. In this article, an electronic synoptic operative report for spinal cord injury was implemented using interoperable standards, HL7 and Systematized Nomenclature of Medicine–Clinical Terms. Subjects (N = 10) recruited for a pilot study completed recruitment and feedback questionnaires, and produced both an electronic synoptic operative report for spinal cord injury report and a dictated narrative operative report for an actual patient case. Results indicated heterogeneity by subjects in access and use of electronic sources of patient data. Feedback questionnaire results confirmed that subjects were comfortable using both methods for data entry of operative reports, and that some were unable to find the diagnosis terms they needed in electronic synoptic operative report for spinal cord injury. Data quality improved. Electronic synoptic operative report for spinal cord injury reports were more complete (95.26%) than dictated (80%) for all subjects. An accuracy assessment, which considered usability for secondary data use, was conducted and the electronic synoptic operative report for spinal cord injury was demonstrated to improve accuracy.

Introduction

The primary role of an operative report for a spinal cord injury (SCI) patient is clinical communication, and the secondary role is research and administration. Data from the reports are collected by the Canadian Institute for Health Information Discharge Abstract Database (CIHI DAD) for administrative purposes. The Rick Hansen Spinal Cord Injury Registry (RHSCIR), a national multi-centre Canadian SCI registry, uses operative report data for research purposes. The idea of the synoptic operative report for SCI patients is to provide ready access to electronic information necessary to achieve efficiencies and improve data quality. ¹ A standardized set of clinical information is needed to encourage a shift towards evidence-based practice and to provide feedback to clinicians on which interventions are beneficial to patients. ²

Synoptic reporting is a data collection method based on a template. It helps ensure high data quality, improved patient safety, learning in the practice setting and cost-efficiencies. This method is rapidly gaining importance at various levels of data recording and reporting including radiology, pathology, clinical assessment and surgical reports.^3–5 The synoptic operative report differs from a dictated narrative operative report in four ways: (1) it is based on a template designed to capture discrete data elements; (2) it aims to capture coded and complete data through the use of lookup lists, drop-down menus and checkboxes; (3) it makes available a human-readable operative report at the time of data entry; and (4) it recognizes the needs of the secondary data users.

Data collected in conventional narrative operative reports require interpretation by human readers who then code the data so they can be used by researchers and hospital administrators. The electronic synoptic operative report for spinal cord injury (eSOR-SCI) template serves as a prompt for pertinent information, which can then be reused to support the needs of researchers and administrators.

Methods and materials

The study population was those residents and neurosurgeons listed on the Neurosurgery House Staff Master Schedule between October 2012 and March 2013 in Queen Elizabeth II Health Sciences Centre, Capital District Health Authority (CDHA), Halifax, Nova Scotia. The study received approval by the CDHA Ethics Board and subjects provided informed consent prior to participation.

The intervention was the eSOR-SCI. The implementation of the eSOR-SCI template was completed in September 2012. It was designed to be conformant with the HL7 Clinical Document Architecture (CDA) Implementation Guide for the Operative Note ⁶ and to represent clinical activity using the HL7 Clinical Statement pattern encoded with Systematized Nomenclature of Medicine–Clinical Terms (SNOMED CT) terminology.^7,8

Surgical residents and house staff were recruited to produce an operative report using both the current method of recall dictation for a narrative report and the proposed method of data entry via the eSOR-SCI template. All subjects were given a recruitment questionnaire and a feedback survey. Subjects were given a 2-minute orientation about the template layout and data entry process. We sought to investigate the acquisition of a skill using a teaching-at-the-time-of-practice approach. We randomly assigned subjects to two groups. Group 1 was asked to dictate an operative report followed by eSOR-SCI, and Group 2 used eSOR-SCI for data entry followed by dictation. A stop watch was used to capture the time subjects took to complete the operative report using the two methods.

A validated instrument, Structured Assessment Format for Evaluating Operative Reports (SAFE-OR), ⁹ was adapted for the completeness analysis. According to personal communication with Sam Minor, a surgery educator and co-developer of SAFE-OR, the SAFE-OR format is used to teach dictation skills to all Dalhousie University junior surgery residents through a didactic session and a memoir aid.

The secondary uses of these data include the RHSCIR and the CIHI DAD. RHSCIR Spine Diagnoses and Spine Procedure forms were used in the design of eSOR-SCI, so that data interoperability would be achieved. This research examined congruence between the two versions of the operative report and the CIHI DAD through terminology mapping from SNOMED CT to the International Classification of Diseases–Version 10–Canadian Edition (ICD-10-CA) for diagnoses and from SNOMED CT to the Canadian Classification of Interventions (CCI) for procedures. The instruments used for the study, therefore, included the eSOR-SCI template, a recruitment questionnaire, a feedback questionnaire, a data quality assessment, and a reuse assessment.

Results

eSOR-SCI template

The eSOR-SCI used interoperable standards for its design and implementation. This was facilitated by published implementation guides, ⁸ including the HL7 V3 CDA Release 2 Implementation Guide for the Operative Note. ⁶ It was influenced by the need for an information infrastructure to enable shared understanding and pragmatic interoperability ¹⁰ and by the US meaningful use legislation which rewards clinicians and institutions for their use of health information technology ¹¹ and for formatting the electronic patient summary using HL7 CDA, Release 2 and SNOMED CT. ¹² The pilot study used the Microsoft Office InfoPath software on a stand-alone computer to build the eSOR-SCI template and bind the HL7 CDA XML schemas with two interoperable terminology standards: SNOMED CT and Logical Observation Identifiers Names and Codes (LOINC). ⁸ The current version of the eSOR-SCI template uses a web-based platform.

There were 402 clinical data elements in the eSOR-SCI template. The sensitivity and positive predictive value (PPV) metrics were respectively 97 and 99 per cent using pre- and post-coordinated SNOMED CT terms. Almost 3 per cent of the clinical data elements were considered to be difficult to encode using the SNOMED CT expressions in either pre- or post-coordinated form. ¹³ All section names were expressed using LOINC codes as per conformance criteria in the HL7 CDA Implementation Guides. This implementation demonstrated the potential value of SNOMED CT for surgeons and enabled a more rigorous health information exchange as described by Lau and Lee ¹⁴ in the SNOMED CT Meaningful Use Framework. Table 1 lists the required and optional sections for the Operative Note.

Table 1.

Conformance profile for HL7 CDA* Release 2 Operative Note.

HL7 CDA Operative Note Section	Conformance criteria
CDA header	Required
Pre-operative diagnosis	Required
Post-operative diagnosis	Required
Surgery description	Required
Surgical operation note findings	Required
Anaesthesia	Required
Estimated blood loss	Required
Complications	Required
Specimens removed	Required
Planned procedure	Optional
Indications	Optional
Disposition	Optional
Plan	Optional
Operative note fluids section	Optional
Surgical drains section	Optional
Implants section	Optional
Operative note surgical procedure section	Optional

*CDA: Clinical Document Architecture.

Subjects and case breakdown

The synoptic operative template was tested by residents and staff surgeons at an academic centre. The subjects were randomized to either complete a traditional narrative dictation followed by the eSOR-SCI or vice versa. The patient diagnoses were categorized as traumatic or non-traumatic SCI, as indicated in Table 2 (S = Synoptic; D = Dictation; T = Traumatic; NT = Non-Traumatic).

Table 2.

Subjects and cases.

Subject	Completed first	Case
1	S	T
2	S	NT
3	S	T
4	D	NT
5	S	NT
6	S	NT
7	D	NT
8	D	NT
9	D	T
10	D	T

S: Synoptic; D: Dictation; T: Traumatic; NT: Non-Traumatic.

We timed eight of the ten subjects, and five of the eight completed the synoptic report more quickly than the dictated report. Average time to complete the synoptic report was 22.7 min (standard deviation 7.86 min), while average time to complete the dictation was 26.0 min (standard deviation 8.8 min).

Recruitment questionnaire

The demographics and computer usage characteristics of the study subjects (N = 10) are described in Table 3. For computer experience, the options were Never, Once or Twice, Monthly, Weekly or Daily.

Table 3.

Recruitment questionnaire: demographics and computer usage.

Subject	Mother tongue	Word processor	Internet search	Medline search
1	Non-English	Once or twice	Once or twice	Once or twice
2	Non-English	Daily	Daily	Weekly
3	Non-English	Daily	Daily	Daily
4	Non-English	Daily	Daily	Daily
5	English	Daily	Daily	Weekly
6	English	Weekly	Daily	Weekly
7	English	Weekly	Daily	Weekly
8	English	Weekly	Daily	Weekly
9	Non-English	Daily	Daily	Daily
10	English	Daily	Daily	Daily

We asked about access and use of Electronic Medical Records (EMR), Horizon Patient Folder, Sunrise Clinical Manager, Meditech and operative report documentation during their medical education and training. Of the ten subjects, five had entered patient data into an EMR; all had viewed patient data using Horizon Patient Folder; two had entered patient data using Sunrise Clinical Manager; and two had accessed patient information via the Meditech system. One had completed a surgical rotation that required him/her to enter a synoptic operative report, and eight had previously used the dictation and transcription system to enter an operative report.

Feedback questionnaire

A 5-point Likert scaling (i.e. Strongly Agree (SA); Agree (A); Neutral (N); Disagree (D); Strongly Disagree (SD)) was used to gather feedback. Whereas the results for eSOR-SCI are shown in Table 4, the results for the dictation and transcription system are given in Table 5.

Table 4.

Feedback questionnaire about eSOR-SCI.

Statement about eSOR-SCI Subjects:	1	2	3	4	5	6	7	8	9	10
I was able to enter all the important data elements	SA	N	SA	SA	A	A	A	A	A	A
I found the diagnosis terms I needed in the template	A	D	A	SA	D	D	A	D	D	D
I found the terms related to operative procedure I needed in the template	A	N	A		A	A	A	D	A	A
I found that structuring content into Procedure, Graft and Implant sections was relevant to me	A	A	A	A	A	A	A	A	A	A
I am comfortable using a computer for data entry of operative report template	A	SA	SA	SA	SA	A	N	A	A	SA
I found the Electronic Synoptic Operative Report easy to use	A	N	A	SA	SA	A	N	D	A	A

eSOR-SCI: electronic synoptic operative report for spinal cord injury; SA: Strongly Agree; A: Agree; N: Neutral; D: Disagree; SD: Strongly Disagree.

Table 5.

Feedback questionnaire about transcription system.

Statement about transcription system Subjects:	1	2	3	4	5	6	7	8	9	10
I was able to enter all the important data elements	A	SA	SA	A	SA	A	SA	SA	A	SA
I dictated in a way that makes it easy for transcriptionists to transcribe the dictation	A	SA	D	A	A	A	N	A	A	SA
I was sufficiently informative for the purpose of an operative report	A	SA	A	A	A	A	A	A	SA	SA
I am comfortable using dictation for data entry of operative report	A	SA	N	A	A	A	A	SA	SA	SA
I found the Transcription System easy to use	A	SA	N	A	SA	A	A	SA	SA	SA

SA: Strongly Agree; A: Agree; N: Neutral; D: Disagree; SD: Strongly Disagree.

The results of the feedback questionnaire confirmed that subjects were comfortable using either method for data entry of operative reports. However, only four of the ten answered ‘Strongly Agree’ or ‘Agree’ to the question ‘I found the diagnosis terms I needed in the template’. Two were reporting on a trauma diagnosis (N = 4), and two were reporting on a non-trauma diagnosis (N = 6). One commented ‘under diagnosis, myelopathy unable to pick spinal stenosis’. The RHSCIR forms for diagnoses and procedures provided the minimum core data elements for the eSOR-SCI template. Although all the cases in RHSCIR are traumatic, ² their diagnosis form is intended to capture both traumatic and non-traumatic diagnoses. It appears clinicians found it challenging to map both traumatic and non-traumatic diagnoses to those provided on the template. One issue in delivering SNOMED CT to users is the design of a well-suited data entry method when there is a long list of coding choices, such as in the case of the diagnosis list. ¹⁵

Evaluation of operative reports

Educational research in surgery has determined that critical aspects of operative procedures are often omitted in dictated operative reports, whereas superfluous aspects are detailed. ¹⁶ The SAFE-OR tool was designed to assess operative reports in general surgery ⁹ and is currently used to assess the dictated operative reports of Dalhousie surgery residents. SAFE-OR is composed of simple items graded either 0 or 1, and complex items graded on a 5-point Likert scaling. Similarly to the SAFE-OR tool and the methodology used by Gur et al. ¹⁷ and Wiebe et al., ¹⁸ we identified 19 key data elements that should be present on an operative report. The focus was on inclusion of data items that were clinically relevant and captured in both dictated and eSOR-SCI methods. The information obtained from both methods was graded as either 0 (absent) or 1 (present).

Table 6 provides a count and a percentage (%) of the number of times each data element was included in the narrative or eSOR-SCI version of the operative report. Just as in the case of Gur et al., ¹⁷ the individual data items and the overall completeness of the narrative reports and eSOR-SCI reports were compared using the appropriate Chi-square and Fisher’s exact tests. A two-tailed p value of 0.05 or less was considered statistically significant. The p values in Table 6 were obtained using the Fisher’s exact test.

Table 6.

Completeness assessment: narrative report versus eSOR-SCI report.

Data item	Narrative count (%)	eSOR-SCI count (%)	p value (two-tailed)
Pre-operative diagnosis	10 (100)	10 (100)	1
Diagnosis location	7 (70)	8 (80)	1
Post-operative diagnosis	8 (80)	10 (100)	0.4737
Indication	10 (100)	10 (100)	1
Indication location	9 (90)	10 (100)	1
Pre-operative clinical status: ASIA Impairment Scale	1 (10)	7 (70)	0.0198
Type of intubation	6 (60)	9 (90)	0.3034
Type of anaesthesia	8 (80)	10 (100)	0.4737
Procedures performed	10 (100)	10 (100)	1
Type of surgical approach	7 (70)	10 (100)	0.2105
Procedure location	10 (100)	10 (100)	1
Patient positioning	10 (100)	10 (100)	1
Surgical drains	6 (60)	8 (80)	0.6285
Implant	10 (100)	10 (100)	1
Implant location	10 (100)	10 (100)	1
Bone graft	7 (70)	9 (90)	0.5820
Adjunct drugs/procedures	9 (90)	10 (100)	1
Estimated blood loss	8 (80)	10 (100)	0.4737
Discharge disposition	5 (50)	10 (100)	0.0325

eSOR-SCI: electronic synoptic operative report for spinal cord injury.

From Table 6, the completeness of the eSOR-SCI reports was statistically significant to the completeness of the dictated reports in two key data items: Pre-operative clinical status: ASIA Impairment Scale with p value = 0.0198 and Discharge disposition with p value = 0.0325. Whereas the completeness of 13 key data items were found to be 100 per cent in eSOR-SCI reports, only seven key data items had 100 per cent completeness in the narrative reports. The overall completeness of the eSOR-SCI reports was found to be 95.26 per cent compared to the narrative reports, which had 79.47 per cent completeness with a p value of less than 0.01.

Beyond the completeness assessment, an accuracy assessment was performed, considering clarity, reproducibility and usability of the eSOR-SCI template for secondary data use. The clarity considered the description of included steps. There was variability among subjects with respect to the information included in the dictated reports. However, all subjects completed all sections of the eSOR-SCI template, thus increasing clarity and usability. The diagnoses in the CIHI DAD are coded to ICD-10-CA and the procedures are coded to CCI.

The accuracy assessment identified a discrepancy for one patient’s operative report between the eSOR-SCI operative report and the CIHI DAD. In the eSOR-SCI, the approach is entered as anterior: cervical. In the CIHI DAD, the CCI code is given as 1SC75PFNWK, which indicates an open posterior approach; however, the approach was not specifically stated in the dictated report from which the DAD information was abstracted. This demonstrates efficacy of the synoptic approach, eSOR-SCI, to improve the accuracy of the data for secondary data use. Two of the trauma cases and three of the non-trauma cases were given identical ICD-10-CA codes for the spinal diagnoses in the CIHI DAD. None of the cases had the same diagnosis in either the dictated or eSOR-SCI operative report. This confirmed the findings of Noonan et al. ¹⁹ that codes used in the CIHI DAD were often found to be a false positive when compared with RHSCIR spine diagnoses and RHSCIR spine procedures.

Conclusion

The pilot study of the eSOR-SCI was completed by ten subjects. Early indications are that the synoptic template, eSOR-SCI, leads to more accurate and complete clinical documentation than the dictated narrative reports, and that this form of reporting is not difficult for residents to use. Limitations of the study were a sample size based on convenience and a lack of training on data entry into a synoptic operative report. Further research is needed to validate a data quality instrument to evaluate synoptic operative reports and to apply it to the web-based iteration of the eSOR-SCI template.

Diagnosis screens for the pilot study of the eSOR-SCI were based on pre-existing research case report forms. We have modified subsequent iterations of the synoptic operative report to make finding the appropriate diagnosis more intuitive for clinical use. SNOMED CT codes ensure consistent data across institutions, but each institution can tailor the interface to the terminology used by clinicians at that particular institution. This facilitates coding at the point-of-data entry while improving data quality and reuse for researchers and administrators. The synoptic operative report coded to SNOMED CT is now being implemented for all spinal surgery in the Queen Elizabeth II Health Sciences Centre.