Abstract
Background
Quality in laboratory medicine is often described as doing the right test at the right time for the right person. Laboratory processes currently operate under the oversight of an accreditation body which gives confidence that the process is good. However, there are aspects of quality that are not measured by these processes. These are largely focused on ensuring that the most clinically appropriate test is performed and interpreted correctly.
Methods
Clinical quality indicators were selected through a two-phase process. Firstly, a series of focus groups of clinical scientists were held with the aim of developing a list of quality indicators. These were subsequently ranked in order by an expert panel of primary and secondary care physicians.
Results
The 10 top indicators included the communication of critical results, comprehensive education to all users and adequate quality assurance for point-of-care testing. Laboratories should ensure their tests are used to national standards, that they have clinical utility, are calibrated to national standards and have long-term stability for chronic disease management. Laboratories should have error logs and demonstrate evidence of measures introduced to reduce chances of similar future errors. Laboratories should make a formal scientific evaluation of analytical quality.
Conclusions
This paper describes the process of selection of quality indicators for laboratory medicine that have been validated sequentially by deliverers and users of the service. They now need to be converted into measureable variables related to outcome and validated in practice.
Introduction
Quality in laboratory medicine is often described as doing the right test at the right time for the right person. This process embraces the process from the moment the test is chosen during patient evaluation to the point when the test is interpreted and a clinical conclusion is developed. 1 Nowadays, the sample collection stage, analysis and reporting components of the testing process is performed under the oversight of an accreditation body which gives confidence that the process is good. Indeed, this is supported by the falling error rates during the analytical stage. 2
However, the accreditation process is designed to ensure that a laboratory performs its processes correctly and by using the Donabedian 3 principles of structure, process and outcome, ensures that the test is done correctly. It does not, however, guarantee that the right, i.e. most appropriate test is selected for analysis. This has been revealed by the close evaluation of laboratory performance by exposing the misuse of tests outside the laboratory. 4 This provides an arena where the quality of the testing process can be further improved.
There are many areas of excellent practise where laboratories contribute to patient care by ensuring that the analytical method is optimal and by helping to minimize misuse (both under and overuse) of the service. These contributions need to be identified so that they can be introduced into all laboratories. This paper outlines a process to identify the most important such indicators using an adapted form of the RAND® UCLA appropriateness model. 5
Method
A modification of the RAND® UCLA appropriateness method was used. 4 As there are no agreed quality indicators, the first stage which should be a literature review, was replaced by a consensus list extracted from focus group meetings. The second phase was performed by an expert panel who rated the indicators with emailed questionnaires but no expert panel was physically held.
Selecting clinical quality indicators
Phase 1: Three discussion group meetings (three meetings consisting of 12, 14 and 36 individuals, respectively) were held during summer and autumn 2010. Minutes of all meetings were taken and summaries of all discussions were shared with all the participants and agreed with all contributors. The final set was distilled from a collation of all meetings with a total of 119 quality indicators.
Assessing validity
Questions posed to the expert panel
Phase 2: An expert panel of clinical users was invited to score the questionnaire. This group consisted of five hospital physicians, four general practitioners and four hospital-based medical registrars. All users were known to the author and were based in North and West Yorkshire. All 13 invitees responded. This group independently scored the questionnaire (see paragraph above). The indicators were then ranked in order of highest approval ratings.
Results
Two high-scoring indicators were that abnormal results should be flagged and that cumulative reporting should be used, but since all laboratories do this, they have been excluded from this report. Within the top 10 scoring indicators were three for education at undergraduate, postgraduate and primary care levels which have been merged, as have two indicators for errors (maintaining a log and recording the action taken for errors). Following these actions, the top 10 indicators have been identified and are recorded in Box 1. The scores for these were 1.67 ± 0.08 (mean ± SD) where 1 was strong agreement and 9 was poor agreement. All indicators had at least 12 responses except the point-of-care (POC) testing indicator which had only eight.
Critical results should be telephoned to requesting clinicians Under- and postgraduate and primary care teaching Laboratories should not perform out-of-date tests and tests with no clinical utility Laboratories should ensure that analytical techniques use correct calibration and units as determined by (inter)national guidance Laboratories should ensure there is a log for documenting laboratory-based errors and should demonstrate evidence of measures introduced to reduce chance of similar future errors Laboratories should ensure that national recommendations for use of tests are implemented (in conjunction with clinical specialists) Laboratories should ensure long-term stability of methods (specificity, e.g. for HbA1c, peptides, tumour markers) for patient care and to ensure methods match values in guidance (see above) Laboratories should ensure that point-of-care testing has adequate quality assurance for all users Laboratories should make a formal scientific evaluation of analytical quality and include in their annual report Laboratories should establish utility of new tests prior to introductionTop 10 indicators identified by the appropriateness model
Discussion
The most comprehensive programme targeting quality in laboratory medicine is being driven by the Division of Laboratory Systems in the Centre for Disease Control in the USA. They have established a group to examine clinical quality issues with an aim of producing best practice guidelines. 6 To date, they have pilot-tested new methods to conduct laboratory medicine quality improvement reviews and have commissioned the first systematic review of quality indicators in laboratory medicine. 7 This study noted that there was a ‘…considerable challenge in identifying, defining, and, ultimately, implementing indicators…’. They found an evidence base for only 14 criteria across the whole range of pathology. These were: test order appropriateness, wristband id errors, patient satisfaction with phlebotomy, specimen quality, (adequacy, contamination, information error), proficiency performance, cervical cytology–histopathology mismatch, availability of inpatient results, corrected laboratory reports, critical value reporting, turnaround time, clinician satisfaction and follow-up of abnormal cervical cytology.
The majority of the indicators identified in the review above were operational whereas the aim of this study was to identify clinical indicators; and so the RAND® UCLA appropriateness method was used but was modified slightly by using consensus to define best practices as the first stage prior to expert panel validation. This method provided a clear set of indicators which have been regarded as important by both providers and users of the clinical laboratory. This appropriateness model is being used by other disciplines to select indicators 8 and in the absence of significant evidence, expert panel guidance may be considered to be the best first-line approach.
The initial phase of this study was the identification of quality indicators considered important by (mostly) consultant medical and non-medical clinical biochemists. It is noteworthy that some important quality indicators are not included such as turnaround time and quality of interpretative comments, similarly a patient focus on such aspects as quality of phlebotomy and the speed of response to abnormal results. These may be raised in the next iteration of quality indicator development.
The second phase was the importance rating by clinical users of the service. The majority of indicators that concerned this user group consisted of analytical factors to the exclusion of preanalytical factors and the only postanalytical factor was critical result reporting. The analytical factors were largely concerned with ensuring that available tests gave the maximal clinical utility. There was clearly also importance in having systems to maximize quality and minimize errors. Although quality assurance for POC testing was within the top 10 indicators, it is worth noting that most of the indicators relating to POC had large numbers of ‘don't know’ responses by the expert panel of users. This is a worrying aspect. It suggests that the clinical users have little knowledge of POC testing, which is of concern considering that POC testing is such an important modality for diagnostic testing. Finally, user education about laboratory medicine was highly rated.
Most clinical biochemists, particularly in specialist laboratories, provide clinical interpretation on reports. They will find it challenging that despite a number of mentions of data interpretation in the list in Table 1, this indicator was not identified as highly important by the expert group. This may be because only a small proportion of overall test reports do have interpretative comments and that comments are unusual for any one clinician, or due to bias in the selection of the expert user group.
The next step will be to translate the indicators into very specific questions that can be measured. These can then be used for benchmarking and quality improvement. The choice of specifics will need a balance to be generalizable across all laboratories.
The transfer into specific measurables will require further analysis. Firstly, the indicator chosen will need to be assessed to determine its relationship with an outcome. Secondly, this relationship will need to be quantified to determine whether a statistical improvement can be demonstrated. Indicators that cannot be significantly related to an outcome cannot be justified unless they are endorsed by national guidance. The assessment of indicators should include cross-sectional and longitudinal evaluation as these can give conflicting impressions of change. 9
In summary, this paper describes the selection of quality indicators for laboratory medicine that have been validated sequentially by deliverers and users of the service. They now need to be converted into measureable variables related to outcome and validated in practice.
DECLARATIONS