Abstract
We love doing science; we relish the research enterprise; we live for the challenge and the gratification of opening young, gifted minds in the classroom and the inquisitive minds of the typical consumer. Importantly, we hold our responsibility to help ensure the quality of research as a sacred trust. But we who carry the mantle of science are arguably in the midst of a crisis of confidence. Without delving into the multidimensional sociological underpinnings of the problem, a recent survey suggests that both the public, professional consumers of science and the press have lost faith in the quality of our peer-reviewed published work. 1 And, again, without detailing the forces that motivate and result in poor and uneven research production and consumer/professional trust, we felt compelled to address the easiest-to-remedy aspect of the problem. A few errors and deficiencies that we observe repeatedly in submitted manuscripts especially involving nutrition and toxicology that come across our laptops for review.
Research on antioxidants
The use of oxygen radical absorbance capacity (ORAC) method is, at best, antiquated and should not be used even as a complementary analytical tool in the investigative process. Vital information about metabolism, bioavailability, and mechanisms of action and efficacy is not measured by any such in vitro assay. That some contemporary investigators have used or are using ORAC to assess antioxidant potential is not justification for inclusion. In simple terms, the free radicals used in an ORAC test do not occur in the human body. There are over 1 million different types of oxidizing reactions that can take place in the body, and an ORAC score measures exactly one of them. ORAC scores can actually differ for the same nutrient depending on the method used. ORAC scores can fall by as much as 90% when the food is cooked or processed. Different antioxidant molecules behave very differently and idiosyncratically in an ORAC test. The measuring units of the ORAC score can change with the density of the food. So a grape can actually have a lower antioxidant score than a raisin which comes from the exact same plant source.
This means the antioxidizing capabilities of food or food components which are high in water content can be underestimated, while highly processed foods can be overestimated using an ORAC score. The ORAC score simply has not been validated, and thus it has no utility in vitro or in vivo! It is also important to understand that the value of antioxidants is dependent on the bulk quantity or dose consumed.
While testing prospective antioxidant benefits with the ex vivo cellular antioxidant activity (CAA) model is an improvement over a biochemical noncellular method such as ORAC, it should not be confused with an in vivo physiological environment that encompasses the complexity of a higher organism with multiple and highly developed systems and tissues, and within which absorption, distribution, metabolism, and excretion (ADME) of nutrients contained by a food matrix constitute a milieu in which in vitro and ex vivo results may have little relevance.
A CAA assay for assessing antioxidants properties in foods, and dietary supplements such as the Wolfe and Liu’s CAA method 2 is certainly more biologically relevant, and together with ORAC values imparts greater analytical value to investigation of possible mechanisms by which bioactive compounds may exert effects; but the leap from isolated cell cultures to the metabolic pathways that regulate health and noncommunicable diseases in human beings requires considerably more data and a dose of scientific humility.
We would refer readers to the work by Schaich, Tian, and Xie. 3 These investigators specifically and systematically summarize the problems associated with the “antioxidant bandwagon” of research that looks at antioxidant capacity, activity, and potential of various whole foods. Three major threats to validity are particularly well described.
One threat is that although antioxidant assays were designed to identify the antioxidants that are to be expected to provide the greatest protective effects against free radicals in vivo, the radical scavenging observed in test tubes probably does not occur in vivo. Cell culture assays attempt at least in part to include effects of metabolites, but most cell types used in assay cultures never see antioxidants in vivo. It is important to note that in 2012, the US Department of Agriculture withdrew from the tables of ORAC the values of many foods, herbs, and spices which had been posted on their website, leaving behind the following statement: There is no evidence that the beneficial effects of polyphenol-rich foods can be attributed to the antioxidant properties of these foods. The data for antioxidant capacity of foods generated by in vitro (test-tube) methods cannot be extrapolated to in vivo (human) effects and the clinical trials to test benefits of dietary antioxidants have produced mixed results. We know now that antioxidant molecules in food have a wide range of functions, many of which are unrelated to the ability to absorb free radicals. For these reasons the ORAC table, previously available on this web site has been withdrawn. (https://www.ars.usda.gov/northeast-area/beltsville-md/beltsville-human-nutrition-research-center/nutrient-data-laboratory/docs/oxygen-radical-absorbance-capacity-orac-of-selected-foods-release-2-2010/)
A third conceptual limitation is that the assays in common use do not address radical reactions in lipids, when membrane lipids are clearly involved in oxidation of both membranes and foods. A few lipophilic versions of assays have been described, but these approaches mostly solubilize lipophilic antioxidants for reaction in aqueous phase rather than testing reactions in lipid phases. As a result, relatively little information is available about how natural antioxidants partition into and interact with lipids.
Attribution of an effect to a single variable
In a sea of possible antecedent or intervening variables including activation of genes upstream and downstream of the observed effect, attributing these outcomes to a single dietary component is beyond our fundamental understanding of metabolic processes. The all too frequent examples of this error are seen in studies of chemoprevention or attenuation of cancers as a unitary phenomenon, or modification of the number and morphology of aberrant cellular structure and function by such variables as exenatide, metformin, garlic, flavonoids, ascorbic acid, green tea phenolics, caloric restriction, and many more. It is time we saw rigorous, highly focused, and tightly controlled studies that are genuinely analytically illuminating in advancing our understanding of a multifactorial disease process.
Exaggerated dosing
Administration of compounds at excessive, unrealistic doses (well beyond no observed adverse effect levels (NOAELs) or human exposures) in question in order to yield a reportable outcome is another of the most egregious and common errors we see in manuscripts and the popular press. Studies of rats often seem to utilize quite arbitrary and exaggerated dosage ranges of any compound, including food ingredients with a history of safe usage as required by US regulations. When these high doses are converted by weight alone (without accounting for route of administration and/or genetic and metabolic differences), the amounts a human would need to consume or absorb would be staggering and often impossible to achieve in clinical or free-living reality. Even if we accept the convention that the NOAEL for a human being is 100× relative to this model and adjusts the doses accordingly, still the amounts are wildly in excess of the estimated dietary intake for a human being and exceed the acceptable daily intake.
Translational research and the failure of humility
The general challenge with otherwise fine bench research is the translation to human systems and the absence of qualification in the conclusion. There should be explicit discussion that the experimental system is an animal model and that extension to the human situation at this juncture is very tenuous. The value of the work is in demonstrating a concept that may serve as a hypothesis builder and research design generator for studies in higher animals and perhaps in human trials.
We reviewed a recent study of carcinogenesis utilizing a classical 1,2-dimethylhydrazine (DMH)-induction of colon cancer in rats. The authors concluded that the putative antioxidative effects of a food extract appeared to have an impact on atherogenesis and also attenuated colon cancer. The threat to validity, even clarity, is the excessively ambitious scope of this work coupled with rather simplistic conclusions. Between the spectrum of oxidative variables, lipid issues, carcinogenesis, and atherogenesis, there was no clear delineation of independent and dependent variables, and beginning to evaluate and make sense of the streams of data, never mind any possible causal relationships that may have been demonstrated was a serially confusing exercise.
As for the DMH model itself, we understand that methylation of the estrogen receptor gene is associated with atherosclerosis and it has long been known that hypomethylation is a consistent biochemical characteristic of human colonic tumors that appears to precede malignant transformation. However, the clinical presentation is complicated by the fact that metabolically activated DMH modifies not only nucleic acids but also histones and other DNA-binding proteins in target cells and tissues. 4 The methylation of DNA and apoptotic or antiapoptotic gene expression is another possible confounding or contributory variable.
Failure to invoke basic and rigorous methods
In the most general sense, the typical threat to the validity of this kind of work is in the failure to approach it through a systematic and classical ADME approach to efficacy, bioavailability, or toxicity in an in vivo model that has clinical relevance to humans. Most studies fail to explore classical bioavailability and provide toxicology data that address fundamental questions—what is the ADME behavior of these compounds in an in vivo system, how are the compounds or metabolites in question absorbed and metabolized, what is their impact in target tissues or lesions in a suitable in vivo model, and one in which the impact of a representative food matrix is explored.
Failure to perform a proper literature search
It is imperative that investigators conduct a thorough and current literature review. It is astonishing that many of the papers we see fail to recognize and cite what are relevant data developed and reported by others. A cursory search on Google Scholar using basic search terms applied over the last decade will go a long way to pick up work that may be illuminating and will strengthen the manuscript, its value, and credibility. Convergent validity is a powerful marker of scientific value.
Again, the failure of modesty in conclusions
We see meticulous bench research that reflects an enormous amount of molecular biology and use of breathtaking and high-tech instrumentation. However, these studies often have the carefully developed data rendered ridiculous by clinical proclamations and purported applications that are either unsupported or completely nonsequitur. In a general sense, research that is devoid of scientific humility is without value. Investigators must strive to match data with our so-what declarations, otherwise the entire enterprise is subverted by the cry for immediate, sensational, and simple cause and effect. In the long run, we only compromise the integrity of our science, weaken the confidence among various publics, and ultimately sacrifice our personal values as investigators and human beings.