Abstract
The search for valid biomarkers to aid in epilepsy diagnosis and management is a major goal of the Epilepsy Research Benchmarks. Many papers and grants answer this call by searching for new biomarkers from a wide range of disciplines. However, the academic use of the word “biomarker” is often imprecise. Without proper definition, such work is not well-prepared to progress to the next step of translating these biomarkers into clinical use. In 2016, the Food and Drug Administration and National Institutes of Health collaborated to develop the BEST (Biomarkers, EndpointS, and other Tools) Resource as a guide to adopt formal definitions that aid in pushing successful biomarkers toward regulatory approval. Using the vignette of high-frequency oscillations, which have been proposed as a potential biomarker of several potential aspects of epilepsy, we demonstrate how improper use of the term “biomarker,” and lack of a clear context of use, can lead to confusion and difficulty obtaining regulatory approval. Similar conditions are likely in many areas of biomarker research. This Resource should be adopted by all researchers developing epilepsy biomarkers. Adopting the BEST guidelines will improve reproducibility, guide research objectives toward translation, and better target the Epilepsy Benchmarks.
Biomarkers! You keep using that word. I do not think it means what you think it means.
To illustrate the importance of using correct nomenclature, we searched PubMed for articles containing “epileptogenic zone” and “biomarker” that were published in 2022. Of the 15 articles that were returned, 13 of them directly referred to “biomarkers of the epileptogenic zone” or indirectly implied that the epileptogenic zone was a directly measurable quantity. There is an obvious academic appeal to finding the epileptogenic zone, but claiming to have a biomarker of it is problematic. First, there is no standardization: Even the basic term “epileptogenic zone” is poorly defined and continues to change, as described in detail by Jehi. 4 Second, the epileptogenic zone is a theoretical concept that cannot be directly measured or validated. Regulatory approval of a clinical biomarker would require validation to show sensitivity and specificity—it is difficult to conceive of a rigorous (or ethical) way to validate identifying “the epileptogenic zone.” Thus, there is a disconnect between the language used commonly in academia and the requirements of governmental agencies tasked with translating academic projects into real clinical practice. The BEST Resource is designed to correct this disconnect.
BEST Resource Guidelines
Before continuing with this brief review, we invite the reader to explore the full BEST Resource, a free, online book that is “meant to be a ‘living’ resource that will be periodically updated.” 3 We also point out a crucial aspect of the BEST Resource: it is coauthored by the FDA, which issues final approval of new medical products in the United States. This demonstrates the primary goal of this initiative—to get these biomarkers out of the research lab and into clinical practice, with one of the essential steps being proper validation. Thus, there are several aspects of this plan that may be unfamiliar to academic researchers.
One of the advantages of the BEST guidelines are standardizing presentation of information. This helps ensure each study has complete information and eases the comparison across studies. A biomarker is “a defined characteristic that is measured as an indicator of normal biological processes.” Biomarkers should have 2 key elements:
To follow these BEST guidelines, future biomarker research should clearly establish the description and context of use. The text should also define the biomarker using a phrase of the form “A [insert category] biomarker of [insert clinical goal].” An example for HFOs is shown in Table 1.
Example Biomarker Description Table.a
Abbreviation: HFO, high-frequency oscillations.
a In this example, HFO rate is being analyzed as “a Prognostic Biomarker of Epilepsy Surgery Outcome.”
An essential goal of this clearly defined biomarker and context is to facilitate verification, validation, and translation. Recall that the final goal of finding a biomarker is to use it in patients, which will require both verification (of what is claimed to be measured) and validation (of the context of use) by government regulators. This necessitates clear definitions. For our specific example of HFOs listed in Table 1, verification would entail ensuring that the HFO detections were really HFOs, so the methods of detection and measurement must be clearly defined. Validation would require assessing if the HFO rate successfully indicates surgical prognosis using appropriate data and statistics. A paper stating that context of use, but without performing that assessment, would be inadequate. For a biomarker to be translatable, it is thus essential that regulatory agencies can verify the measurements and that the context of use has directly measurable endpoints, such as surgery outcome, seizure onset location or time, tissue pathology, and so on. Any biomarker that could not be verified, or a context of use that could not be validated, is unlikely to receive clinical approval and thus has minimal clinical utility.
Applying BEST Methods to HFOs as a Biomarker (…Of What?)
While a similar discussion could be had in many other types of biomarkers, we here illustrate several of the pitfalls that arise with electrographic biomarkers such as HFOs. But we have already made a mistake: even in the first sentence of this paragraph, we did not define the context of use. High-frequency oscillations are a biomarker…of what? Despite decades of casually saying that HFOs are a biomarker of “epilepsy,” this is imprecise. The correct verbiage would be “a diagnostic biomarker of having epilepsy,” but this is not how HFOs are used; that is; HFOs are not used to diagnose epilepsy. Seizures on standard scalp EEG are a much better (though still imperfect) “diagnostic biomarker of having epilepsy.”
Defining the context of use for HFOs and other EEG data is not straightforward. In recent years, it has become increasingly common to talk about research with a “biomarker” of the “epileptogenic zone” or “epileptic network.” As described above, making such a claim is problematic because it would be extremely difficult or impossible to prove it to the FDA. One popular alternative is to use HFOs as a biomarker of the seizure onset zone (SOZ). The SOZ can be defined as the area of tissue that was recorded by EEG that the reading clinician thought initiated the seizure. It is easy to validate, but it has severe limitations as well. First, the SOZ will vary greatly depending on the reader, so it is not a strong gold standard. Second, there is ambiguity in what constitutes success: must the biomarker identify the exact channels? A subset? Such flexibility allows for statistical manipulation. But even ignoring those limitations, the very idea of finding a biomarker of SOZ is questionable. What is the goal—to replace EEG monitoring? Do we expect clinical practice to change in such a way that clinicians will not observe seizures in surgical candidates? The SOZ is the standard of care and it is unlikely that clinicians will be willing to base surgical decisions without seeing actual seizures. There may be some acceptable contexts such as intraoperative analysis, but an even bigger problem is that comparison with SOZ is inherently flawed because SOZ is flawed. We already know that a large proportion of epilepsy surgeries, which are based upon the SOZ, often do not resect the full SOZ, and fail to achieve seizure freedom even when they do (Figure 1, also 5,6 ). A perfect biomarker of SOZ would be expected to have the same rate of failure. Thus, since we know SOZ is a disappointing biomarker of surgical outcome, it is a fallacy to develop a biomarker of SOZ and expect it to improve outcomes. Although a biomarker of SOZ could be validatable, its context of use is not particularly useful. What we need is not a biomarker of SOZ, but something that replaces or augments the SOZ.
Outcome versus SOZ resection. Twenty-eight patients with epilepsy (University of Michigan available patients from 2018-2019) show that percent resection of SOZ does not predict outcomes. Stratifying location (ie, temporal lobe) did not improve results.
There are many potential biomarkers at this stage of development, waiting for an appropriate context of use. For HFOs, after considering the challenges above, one context of use may be to prognosticate outcome. Indeed, the first prospective trial of HFOs focused on this context, 7 though not incorporating all the BEST methods. We propose that one clinically useful, validatable context of use for HFOs is to prognosticate outcome by measuring how much of the biomarker is resected in the surgery, with seizure freedom as the measurable outcome. Because this is similar to determining which tissue needs to be resected, it is tempting to state that HFOs are a biomarker of the epileptogenic zone. However, that shorthand description is imprecise, because it is not a consensus definition that is measurable, and ethically challenging to verify experimentally. If we hope to move our research into humans, clear definitions and validatable measurements are paramount.
For example, Table 1 shows an example description of a biomarker description table that could be included in a manuscript about an HFO biomarker. The HFO detection method is clearly stated. The context of use is defined as prognosticating outcome. The specific methods used would be the subject of the manuscript; that is, determining which values are considered abnormal, calculating how many of them will be resected, and comparing with outcome. The aspect of unique identifiers is beyond the scope of this manuscript, though we envision a day when specific detection methods, including respective data acquisition protocols and requirements, are registered and labeled with a unique identifier. We recommend a similar table in publications involving epileptic biomarkers, including electrographic biomarkers. Specific to electrographic biomarkers, we further recommend that the “source” include key details on the data requirements, including the ictal-state (interictal, ictal, etc), the type of electrographic modality (scalp EEG, intracranial EEG, etc), the general scale of the duration of data analyzed (minutes, hours, days), and, when necessary, the recording environment (intraoperative, out-patient, at-home, etc). We recommend “measurement method” include data on the algorithms used to extract the biomarker from the source information.
There are a wide range of biomarkers under investigation for several different contexts of use in epilepsy. Recent governmental initiatives now seek to increase the rigor of such research, and it is critical for the epilepsy research community to be aware of the BEST Guidelines and incorporate them into future research. We have demonstrated how this format affects HFO research, but it is applicable to all biomarker research. Hopefully, by incorporating these Guidelines, the term “biomarker” will mean what we think it means and lead to clinical translation.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors declare a licensing agreement with Natus Medical, Inc. Natus had no involvement with this manuscript.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institutes of Health (grants K01-ES026839 and R01-NS094399).
