Abstract
Background
Reflective testing, i.e. interpreting, commenting on and, if necessary, adding tests in order to aid the diagnostic process in a meaningful and efficient manner, is an extra service provided by laboratory medicine. However, there have been no prospective randomized controlled trials investigating the value of reflective testing in patient management.
Methods
In this trial, primary care patients were randomly allocated to an intervention group, where general practitioners received laboratory tests results as requested as well as add-on test results with interpretative comments where considered appropriate by the laboratory specialist, or to a control group, where general practitioners only received the laboratory test results requested. Patients’ medical records were evaluated with a follow-up period of six months. For both groups, the primary outcome measures, i.e. both intended action and actual management action, were blindly assessed by an independent expert panel as adequate, neutral or inadequate.
Results
In 226 of the 270 cases (84%), reflective testing was considered to be useful for the patient. In the intervention group (n = 148), actual management by the general practitioner was scored as adequate (n = 104; 70%), neutral (n = 29; 20%) or not adequate (n = 15; 10%). In the control group (n = 122), these numbers were 57 (47%), 37 (30%) and 28 (23%). This difference was statistically significant (P < 0.001).
Conclusion
This randomized controlled trial showed a positive effect of reflective testing in primary care patients on the adequacy of their management, as documented in medical records.
Introduction
The core business of the clinical laboratory is to provide results of tests requested by physicians and other health care workers. 1 More generally, it can be defined as ‘giving support in solving diagnostic problems’. In the post-analytical phase, laboratory professionals can add value to the purely analytical service, when the laboratory support extends to the interpretation of laboratory test results. 2 , 3 This includes the recognition and interpretation of unexpected pathology and pathology outside the normal scope of physicians’ expertise. Examples of disorders typically recognizable by distinct laboratory findings are haemochromatosis, monoclonal gammopathy, hyperparathyroidism, vitamin B12 deficiency, thalassaemia, metabolic syndrome with steatosis, hepatitis or Gilbert’s syndrome.
The laboratory specialist can take other available (medical) information into account and decide whether additional tests are indicated. In most cases, these tests may be performed with patients’ samples that are already available in the laboratory. Comments added to the report can then support the requesting physician in the interpretation of the results. This activity has been named ‘reflective testing’, i.e. after inspection of (reflection on) the results. 4
There have been several studies by other researchers as well as our group, supporting the usefulness of reflective testing, as shown by a positive appreciation of physicians5–8 and patients, 9 and by a learning effect in physicians. 10 The results of these studies were largely based on retrospective studies and questionnaires that documented an evaluation of reflective testing by the stakeholders. However, there is a need for additional research to deliver further scientific evidence of the clinical effectiveness for patient management. 11
To date no prospective randomized studies are available documenting the additional value of reflective testing. We designed a randomized controlled trial in primary care patients to evaluate the effect of reflective testing on patient management.
Materials and methods
General design
We designed a randomized controlled trial (RCT), starting with laboratory requests of general practitioners (GPs). In the intervention arm, the results of reflective testing (results of requested tests plus add-on tests and interpretative comments) were presented to the GPs; in the control arm, only the results of requested tests were reported. After a follow-up period of six months, the medical records of all patients were assessed for medical management actions (such as further diagnostic procedures, treatment actions, referrals) by an expert panel that was unaware of the allocation of the included patients.
After approval by the local ethics committee (METC Z, Heerlen, the Netherlands, 08-N-73) and in cooperation with the local organization of GPs in the region Eastern South Limburg (Heerlen and surrounding towns), all local GPs (n = 161) were informed about this trial. Information material was supplied to GPs and patient specific information was made available in waiting rooms of GP surgeries.
Selection of laboratory reports
The Department of Clinical Chemistry and Hematology of Zuyderland Medical Centre in Heerlen, the Netherlands, introduced several years ago the procedure of reflective testing to local general practitioners. During the daily routine of authorization, laboratory reports with (a combination of) deviating test results were evaluated by the laboratory specialist (WO), using pre-installed filters in the laboratory information system based on abnormal values and delta-checks (see Supplemental file). Taken into consideration previous test results and known medical history (if available), one or more tests could be added and an interpretative comment was given.
Although there are certain routines, reflective testing differs from reflex testing, i.e. testing according to fixed, pre-defined test protocols. In reflective testing, it is ultimately the expertise of the laboratory specialist that will determine whether additional testing and personalized interpretative comments can be helpful. Examples of routines for reflective testing are presented in Table 1 (see also http://www.reflectivetesting.com/uk/index.htm). It should be noted that tests were not added as reflex tests, but only in the cases where it was considered helpful as indicated by the laboratory specialist.
Examples of reflective testing.
Note: Correct interpretation might involve additional test results, including historical results. For more examples and detailed information, see http://www.reflectivetesting.com/uk/index.htm.
ALT: alanine aminotransferase; anti-TPO: thyroid peroxidase antibody; CK: creatine kinase; CRP: C-reactive protein; eGFR: estimated glomerular filtration rate; ESR: erythrocyte sedimentation rate; fT4: free thyroxine; HDL: high-density lipoprotein; LDL: low-density lipoprotein; MCV: mean corpuscular volume; NT-proBNP: N-terminal pro b-type natriuretic peptide; PTH: parathyroid hormone; TBII: thyrotropin binding inhibiting immunoglobulins; TSH: thyroid-stimulating hormone.
According to this reflective testing procedure, laboratory reports from 600 patients were collected during a 16 weeks’ period (September 2009 – January 2010). Patients were given a consecutive trial code number.
Randomization
Reports that were eligible for the addition of one (or more) test(s) were included in the study. At the date of blood sampling, tests and comments were added to all reports, following the routine procedure of our laboratory. After the tests were completed and comments added – with the report ready for reporting – the patients’ trial code number was linked to a random number from a sequence of 600 numbers that was computer-generated. The even numbers in the randomized list corresponded to the intervention group, the odd numbers to the control group. The laboratory specialist who selected the reports for suitability to be included in the study was unaware of the assignment of reports in intervention or control group, thereby concealment of allocation was fulfilled. GPs of 300 patients in the intervention group received reports with additional tests and comments; GPs of the other 300 patients only received the results of the originally requested tests, without further comments (control group). The results of the additional tests and comments of the patients in the control group remained available at the laboratory.
Informed consent and patient follow-up data
Due to the structure of the trial, informed consent allowing the expert panel to assess relevant outcomes in the medical record could only be obtained after selection of the reports. Patients were contacted and asked for consent to obtain data from their GPs’ medical records. Patients did not receive information on the group they had been allocated to. After a follow-up period of six months, copies of the medical records of consenting patients were collected, with information about subsequent laboratory reports, other additional diagnostic procedures, treatments, referrals and specific patient data (i.e. medical history, medication).
Evaluation of the effect of reflective testing
An expert panel evaluated the laboratory reports and follow-up data as available in the medical records based on information up to six months after the reporting of the laboratory tests. The panel consisted of a specialist in internal medicine (CvD), a general practitioner (HS) and a clinical chemist who was not involved in the routine procedure of reflective testing (BvA). The panel was blinded for the allocation of patients and received the results of reflective testing for all patients, intervention and control group.
First, each panel member individually assessed the laboratory results and medical records of each included patient. Then, three consensus meetings were arranged by the first two authors (WO, WV), to discuss the evaluation and reach consensus.
The evaluation consisted of three elements:
The
Example:
A comment is correct in terms of content; it contained new information for the GP (scored as useful) or no new information (e.g. the condition was already known, scored as neutral). An incorrect interpretation or outdated comment was scored as not useful.
2. The
Example:
A comment may advise repeat testing of the thyroid function after three months and this recommendation is recorded by the GP in the medical record of the patient. This intended action of the GP is scored as adequate and in accordance with the national guideline, while no action is scored as not adequate.
Example:
A comment may advise to perform an ultrasound of the liver to check for non-alcoholic steato-hepatitis. Although the ultrasound was not mentioned in the medical record, the GP did report monitoring liver enzymes. The intended action of the GP is scored as neutral.
3. The
Example:
A comment indicates that a patient has a very low vitamin D status. The medical record of the patient contains no further diagnostic (e.g. repetition of laboratory tests) or therapeutic action (e.g. vitamin D supplementation). The intended
Example:
A comment may advise to check for the coagulation disorder, Von Willebrand’s Disease in a female patient with chronic iron deficient anaemia. The medical record showed that the patient suffered from menorrhagia. The GP recorded in the medical record that the patient was to be treated with iron supplements for a month and haemoglobin to be monitored afterwards. In the follow-up period, the patient was indeed treated with iron supplements but no laboratory testing was performed. The actual management was scored as neutral, since there was an action based on the iron deficient anaemia, but laboratory testing was not sufficient.
Discrepancies within the panel were solved by discussion. The evaluation was completed in 2013.
Analysis
In each group, we calculated the proportion of patients in which reflective testing, the intended action and the actual management were classified as useful or adequate, neutral and not useful/not adequate. Differences in proportions between the intervention and control group were tested for statistical significance with Pearson’s Chi-squared test statistic.
Results
Study population
We started with laboratory reports of 600 patients that were deemed suitable for reflective testing. For 79 of these patients (intervention group: n = 40; control group: n = 39), additional tests were performed, but a suspected diagnosis or condition was not confirmed and adding a comment had not been considered as useful. These reports were not included in the analysis.
Example:
A patient has an unexpected high sedimentation rate (ESR). CRP and total protein are added to evaluate the probability of the presence of a monoclonal gammopathy. A high CRP and normal total protein rule out a suspected presence of gammopathy as cause of the high ESR. In this case, no interpretative comment would be added to the laboratory report, since the suspected diagnosis is not confirmed and a comment is considered to have minor relevance for the GP.
Reports with an interpretative comment were selected for follow-up. Patients who did not provide informed consent for collecting follow-up data were excluded (intervention group: n = 112, control group: n = 139; χ2 = 5.41; P < 0.05). Thus, data of 270 patients could be further analysed (Figure 1).
Flowchart illustrating the number of laboratory reports at the start of the study and after follow-up. *Results of additional tests and comments remained available at the laboratory.
Median age of the patients was 59 years, range 5–89 years. The intervention group included 148 patients (68 males and 80 females); the control group 122 patients (63 males and 59 females). Classification according to (suspected) diagnosis is shown (Table 2); most common were thyroid disorders, steatosis/non-alcoholic steato-hepatitis and anaemia.
Classification of the patients in which follow-up data were obtained.
*Including biliary tract disorders, alcohol abuse, (chronic) viral hepatitis, heart failure, hypercalcaemia and vitamin deficiency.
Usefulness of reflective testing according to the expert panel
Reflective testing was considered to be useful for the patient, in the opinion of the expert panel in 226 of the 270 cases (84%); in 41 patients (15%), the adding of tests and interpretative comment was judged to be neutral. In the remaining three patients (1%), reflective testing was not found to be useful: two patients were already treated or referred to a specialist based on the suggested diagnosis; the laboratory results of the third patient reflected transient symptoms. The results were comparable between the intervention group (useful: 85%, neutral: 14%, not useful: 1%; n = 148) and control group (useful: 82%, neutral: 17%, not useful: 1%; n = 122). The usefulness of reflective testing in different (suspected) diagnoses is shown (Figure 2).
How useful were the additional tests and/or comments for the patient? This question was scored by the expert panel as ‘useful’, ‘neutral’ or ‘not useful’. Results are presented for the total group of patients (n = 270). NASH: non-alcoholic steato-hepatitis. *Including biliary tract disorders, alcohol abuse, (chronic) viral hepatitis, heart failure, hypercalcaemia and vitamin deficiency.
Primary outcome: Management actions by the general practitioner
In 119 of the 148 patients in the intervention group (80%), GPs had intended to perform an adequate action, according to the panel's assessment. In 18 patients (12%), the intended action of the GP was neutral, and in 11 patients (7%) inadequate. In the control group (n = 122), these numbers were 69 (57%), 29 (24%) and 24 (20%). This difference was statistically significant (χ 2 = 18.38; P < 0.001) (Figure 3).
Was there an intention to perform an adequate management action? This question was scored by the expert panel as ‘adequate’, ‘neutral’ or ‘not adequate’. Differences between intervention and control group were statistically significant (χ 2 = 18.38; P < 0.001).
The actual management was also different between patients of the intervention and control group (χ 2 = 16.27; P < 0.001). In patients of the invention group, actual management of the GPs was scored adequate (104/148; 70%), neutral (29/148; 20%) or inadequate (15/148; 10%), compared to the control group (adequate: 57/122, 47%; neutral: 37/122, 30%; inadequate: 28/122, 23%) (Figure 4).
Was there a recorded management action with respect to the care process of the patient? This question was scored by the expert panel as ‘adequate’, ‘neutral’ or ‘not adequate’. Differences between intervention and control group were statistically significant (χ 2 = 16.27; P < 0.001).
Discussion
In this study among GPs in our region, it was demonstrated that reflective testing in primary care has a significant positive effect on patient management, as interpreted by a panel consisting of a specialist in internal medicine, a GP and a clinical chemist, and according to available guidelines where applicable. To our knowledge, this is the first large scale prospective randomized controlled trial on reflective testing.
The term ‘reflective testing’ was introduced in 2004 by Paterson and Paterson in a study on the usefulness of adding-on tests. 4 Reflective testing can be seen as an additional dimension of the service of the clinical chemistry laboratory. 12 , 13 According to a recently developed Dutch guideline, reflective testing is defined as the process of interpreting, commenting on and – if necessary – adding tests in order to complete the diagnostic process meaningfully and efficiently. 14 In this definition, a certain emphasis is put upon the interpretation, as the addition of information was considered to be the most important aspect. This definition also covers those cases where no additional testing is considered necessary, but additional interpretative information is indicated. It should be noted that in the present study, almost without exception both add-on tests and comments were added to the report.
In a questionnaire-based study, we previously showed that the majority of the GPs in our region were in favour of reflective testing in almost all scenarios. 15 Those results were highly comparable with the results of a similar study in the UK. 5 Other studies were performed among local general practitioners (GPs) and nurse practitioners (NPs), showing that GPs and NPs overwhelmingly supported the activity of adding tests by the laboratory and incorporating clinical interpretative comments on biochemistry results. 7 , 8 In addition, the effect of adding-on tests and comments to laboratory reports on the diagnostic and therapeutic process has been evaluated retrospectively.16,17 Of the GPs, 99% indicated that the information had been useful, and 53% indicated a positive influence of reflective testing on patient management. This was expressed in an earlier diagnosis, earlier start of treatment, earlier referral to a medical specialist, adjustment of medication, additional diagnostics, etc. That study again confirmed that reflective testing is highly valued by GPs. Moreover, it leads to an improvement of the diagnostic and therapeutic process, at least in the opinion of the GPs served by our laboratory.
In other studies, reflective testing strategies were shown to be valuable in the detection of certain conditions such as pituitary dysfunction 18 and hereditary haemochromatosis (HH). 19 Raised ferritin is a frequent incidental finding in primary care, and reflective testing with add-on iron studies could be valuable in guiding the need for genotyping and identifying patients with HH. 4
Studies concerning reflective testing, including in our laboratory, have mainly been retrospective, i.e. assessed the subjective value based on the opinion of physicians, nurses and patients. Objective studies only addressed the outcome, based on the number of identified patients of a certain condition (often expressed as the number needed to diagnose or NND). 20 For HH, the authors reported a NND of 35, while others reported NNDs of 18.84 and 29.3. 19 One could argue that the limitation of the existing literature is a focus from the perspective of laboratory professionals, i.e. on readily measurable aspects like NNDs as a measure of effectiveness, instead of a focus on the clinical perspective.
There are important juridical considerations when performing reflective testing which are addressed in a recently developed Dutch guideline on reflective testing. 14 In short, there should be an agreement with the GPs on reflective testing, including a mandate for the laboratory specialist to add tests without consultation with the GP on a case by case basis. Secondly, it is the responsibility of the GP to inform the patient that additional testing might be undertaken, giving the patient the opportunity to decline this option. Some tests (e.g. expensive tests, DNA-testing) should only be performed after specific case by case consultation of the GP. It is recognized that national regulations on this are likely to differ between countries.
Although the current RCT represents the existing practice of reflective testing for primary care patients in our laboratory, there are some shortcomings. Reflective testing depends on the selection of reports by applying filters, as not all reports can be subjected to visual inspection by the laboratory specialist. These filters (see Supplemental file) will determine which reports will be evaluated for reflective testing; the use of other different filters or other approaches to reflective testing might influence the outcomes.
Because reflective testing in this form covers a wide range of different patterns of laboratory results and patient conditions, it is challenging to measure an improvement of the health condition in an unselected patient group. A common outcome measure had to be used, and for that reason we applied a scoring system by a panel evaluating the quality of management by the GPs. The care provided by the GP was scored by a panel as adequate, neutral or inadequate. Current practice guidelines of the national GP organization (https://www.nhg.org/richtlijnen-praktijk) were used in this evaluation and served as a benchmark for the quality of the management.
Although the evaluation of care was planned to be as objective as possible, it cannot be entirely excluded that a certain degree of bias arose during the assessment process. There was a significant difference between the number of patients that consented in each group for follow-up review of their medical records. Since patients were not aware of the group randomization, there is no obvious explanation for this finding. It is possible to speculate that GPs communicated the addition of tests and comments with patients, and therefore patients in the intervention group might have been more motivated to give informed consent. Medical records were studied for the period of six months after the laboratory results were reported. It was assumed that the influence of test results on medical decisions was limited to this period. It proved to be necessary to make a distinction between the intention of GPs for certain action, and actual management. It was not uncommon, usually for reasons that were unclear, that intended actions were not actually undertaken. Both results, intended action and actual management were recorded separately.
In conclusion, the results of this RCT contribute significantly to the evidence on the effectiveness of reflective testing. Reflective testing improves the quality of management decisions by general practitioners. Although our results show a significant beneficial effect, additional research is needed to evaluate the balance between costs and benefits. It might be expected that in the near future reflective testing will be an expert system supported activity. 21 That could shift the cost-benefit balance considerably in the desired direction.
Supplemental Material
sj-pdf-1-acb-10.1177_0004563220968373 - Supplemental material for Reflective testing – A randomized controlled trial in primary care patients
Supplemental material, sj-pdf-1-acb-10.1177_0004563220968373 for Reflective testing – A randomized controlled trial in primary care patients by Wytze P Oosterhuis, Wilhelmine PHG Verboeket-van de Venne, Cees TBM van Deursen, Henri EJH Stoffers, Bernadette AC van Acker and Patrick MM Bossuyt in Annals of Clinical Biochemistry
Supplemental Material
sj-pdf-2-acb-10.1177_0004563220968373 - Supplemental material for Reflective testing – A randomized controlled trial in primary care patients
Supplemental material, sj-pdf-2-acb-10.1177_0004563220968373 for Reflective testing – A randomized controlled trial in primary care patients by Wytze P Oosterhuis, Wilhelmine PHG Verboeket-van de Venne, Cees TBM van Deursen, Henri EJH Stoffers, Bernadette AC van Acker and Patrick MM Bossuyt in Annals of Clinical Biochemistry
Supplemental Material
sj-pdf-3-acb-10.1177_0004563220968373 - Supplemental material for Reflective testing – A randomized controlled trial in primary care patients
Supplemental material, sj-pdf-3-acb-10.1177_0004563220968373 for Reflective testing – A randomized controlled trial in primary care patients by Wytze P Oosterhuis, Wilhelmine PHG Verboeket-van de Venne, Cees TBM van Deursen, Henri EJH Stoffers, Bernadette AC van Acker and Patrick MM Bossuyt in Annals of Clinical Biochemistry
Supplemental Material
sj-pdf-4-acb-10.1177_0004563220968373 - Supplemental material for Reflective testing – A randomized controlled trial in primary care patients
Supplemental material, sj-pdf-4-acb-10.1177_0004563220968373 for Reflective testing – A randomized controlled trial in primary care patients by Wytze P Oosterhuis, Wilhelmine PHG Verboeket-van de Venne, Cees TBM van Deursen, Henri EJH Stoffers, Bernadette AC van Acker and Patrick MM Bossuyt in Annals of Clinical Biochemistry
Footnotes
Acknowledgements
The authors acknowledge the participation of the general practitioners and patients in this trial.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was partly supported by a grant of Stichting Kwaliteitsgelden Medisch Specialisten (SKMS), the Netherlands.
Ethical approval
Approval of the local medical ethics research committee (METC Z, Heerlen, the Netherlands) was obtained (08-N-73).
Guarantor
HA Kleinveld (Head of the Department of Clinical Chemistry and Hematology).
Contributorship
WP Oosterhuis and WPHG Verboeket-van de Venne designed and performed the study, and wrote the article; CTBM van Deursen, HEJH Stoffers and BAC van Acker participated in the trial as panel experts and critically reviewed the article; PMM Bossuyt advised on the design and analysis of the trial and critically reviewed the article.
Trial registration
Registered in Netherlands Trial Registry: NL1817 (NTR1927).
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
