Abstract
When building a house, it is important to select the right tools and material for the job. Why build a house of straw when you can build one with bricks? Just as builders select the right tools for a job, physicians look to rigorous science to identify the best treatment options for their patients based on available evidence. As a community, we have turned to systematic reviews and meta-analyses to better advise on these questions. We commend the authors of Forsberg et al., ‘No support for lipid rescue in oral poisoning: A systematic review and No support for lipid rescue in oral poisoning: A systematic review and analysis of 160 published cases analysis of 160 published cases’, for their attempt to evaluate the efficacy of lipid rescue therapy (LRT) but find considerable fault with their choice of tools. Their analysis simply doesn’t justify their conclusion or the message conveyed in the title.
Forsberg et al. represent the third systematic review in 2 years investigating lipid emulsion in oral overdose. 1,2 It essentially repeats a 2011 systematic review by Cave et al. 3 The previous three publications all concluded that there is evidence to consider the use of lipid in non-local anaesthetic toxicity. The American Academy of Clinical Toxicology (AACT) lipid emulsion therapy workgroup further provided an accompanying recommendation that physicians could consider LRT in non-local anaesthetic toxicity, when other treatments failed. 4 What new information does the paper by Forsberg et al. contribute that sets it apart from the prior reviews? Forsberg et al. set the goal of their study ‘…to present a systematic review and case analysis of practically all published reports on humans treated with lipid rescue for LAST or oral poisoning’. However, they excluded both randomized human data 5 and randomized animal data 6 and only examined case reports. As such, we cannot expect their analysis to say anything new. Instead the article reinforces Ioannidis’ recent assertion about the epidemic of flawed systematic reviews on high profile topics that at best contribute very little of value to a field. 7 At worst papers like Forsberg et al. can unintentionally mislead physicians and potentially diminish life-saving opportunities for a patient in extremis.
Regarding their choice of tools, the authors modified the World Health Organization-Uppsala Monitoring Center (WHO-UMC) pharmacovigilance tool to assess the contribution of LRT to patient recovery. Unfortunately, the WHO-UMC is not validated as a method to assess causality in case reports for treatment effects. Using this method, the authors essentially ask, ‘What fraction of case reports purporting a benefit of LRT do we think really show a benefit?’ Unfortunately, without matched controls for other treatments, the numbers generated mean very little, and their effort is wasted. As mentioned earlier, Cave et al. attempted the same thing but Forsberg’s implementation differed markedly. Cave et al. used three reviewers, showed each rater’s assessments and provided Kappa statistics. In contrast, Forsberg et al. did not report the level of agreement or distribution of ratings by their two experts who resolved discordance for 36% of classifications by ‘discussion’. This ignores the skewing effect if the discussion always favours the same rater’s choice. The assessments are subjective and assigning numerical values doesn’t change that. The concern for confirmation bias holds since one of the raters recently wrote an opinion piece that was unfavourable to using lipid rescue. 8 The title, ‘No support for lipid rescue…’, confirms the problem since it more accurately conveys a goal or intent of the study than its results.
Additionally, the respective conclusions of the Cave’s study and the Forsberg’s study cast doubt on the validity of the WHO-UMC as a tool to assess treatment effect. Cave et al. concluded that ‘…data suggest a possible benefit of LRT in potentially life-threatening cardiotoxicity from bupivacaine, mepivacaine, ropivacaine, haloperidol, tricyclic antidepressants, lipophilic beta blockers and calcium channel blockers’, 3 while Forsberg et al. concluded that they ‘…would not recommend its use at all in cases of oral poisoning’. The gross disagreement between the two studies points to poor reproducibility and further undermines the value of the WHO-UMC for this purpose.
On a final note, Forsberg et al. missed an important point in correlating LogP with their rating – a previously uncharacterized relationship. Recent advances in our understanding of the mechanism of lipid resuscitation indicate that mechanisms other than partitioning are at play, including direct cardiovascular benefits 9 and post-conditioning. 10 Lipid solubility of the offending drug is only one factor among several that contribute to reversibility by LRT. Therefore, relying on a correlation with LogP ignores the fact that the ‘lipid sink’ is no longer considered the prime moderator of rescue 11 and such a correlation is not a test for the efficacy of lipid resuscitation.
We commend Forsberg and colleagues for the effort but they plainly missed the mark. The title notwithstanding, the study is an opinion piece masquerading as a systematic review. We view their choice of the untested WHO-UMC tool as a means of conveying point of view, not proof of principle. In our opinion, the field must look to more exacting, quantitative and robust methods for objective answers. These include properly performed meta-analyses of animal models, registries and propensity score–matched trials. Randomized studies with defined inclusion-exclusion criteria and well-defined controls are needed to assess more accurately the role of LRT in treating oral overdose. Animal and computational models will further inform and help optimize LRT for this indication. We hope the community will build towards this same goal by identifying the right tools to assemble the evidence. If you want to drive a nail, use a hammer.