Abstract
As reports on the safety of various nanomaterials have yielded conflicting results, assessment of the reliability of each study is required to objectively interpret overall safety of the nanomaterial. A 2-step method to assess the quality of nanotoxicity studies is described. The first step uses a publicly available tool to rank the reliability of the study based on adequacy of design and documentation of methods, materials, and results, providing a “study score.” The second step determines the completeness of physicochemical characterization of the nanomaterial/nanomaterials assessed within the study, providing a “nanomaterial score.” This approach is encouraged to promote the notion that for studies conducted with nanomaterials, the combination of a reliable study and sufficient nanomaterial characterization is of significantly greater value than either of these alone. It is anticipated that the use and evolution of this approach will assist with the design and interpretation of studies assessing nanomaterial toxicity.
Introduction
The promise of nanotechnology contributing to significant advances in fields as diverse as electronics, consumer products, nutrition, and medicine is quickly becoming a reality. Materials and products that are based on nanotechnology or that contain a nanomaterial have been introduced or are being developed for use in these and other areas at an increasing rate. It is recognized, however, that the optimistic outlook for a future that is improved by nanotechnology must be weighed against the realization that we currently know relatively little about the potential adverse effects of nanomaterials on human health and the environment.
Although the definition of a nanoparticle is generally considered to be a particle with at least 1 dimension of 100 nm or less, 1,2 many authoritative bodies including the US Food and Drug Administration (FDA) have not as of yet adopted a formal definition for a nanoparticle for regulatory purposes. 3 Nonetheless, it is recognized that as a result of their small size and unique physicochemical properties, the toxicological profiles of nanoparticles may differ considerably from those of larger particles composed of the same materials. 4,5 Nanoparticles of different materials (eg, gold, silica, titanium, carbon nanotubes, quantum dots) are not expected to interact with and affect biological systems in a similar fashion; as a result, it is unlikely that the toxic potential and/or mechanisms of nanoparticles can be predicted or explained by any single unifying concept. 6
The testing of engineered nanomaterials for potential toxicity is an area of continued growth and refinement. Desired (ie, efficacious) and unwanted (ie, toxic) effects of nanomaterials can be examined using in vitro methods, in vivo models, and combinations of the 2 that are relevant to the particular end points of interest. A cursory review of the scientific literature reveals substantial growth in the number of publications reporting on studies of the potential adverse effects of engineered nanomaterials. Specifically, a search of the PubMed database (http://www.ncbi.nlm.nih.gov/sites/entrez) conducted on December 2, 2009, which used the search terms ([nanoscale or nanoparticle or nanomaterial* or nanotoxic* or nanoscience or nanotechnolog*] and [safe* or toxic or toxicity or toxicology or hazard* or adverse effect]) identified 70 unique articles that were published in 2003; the same search revealed that 893 unique articles were published in 2008, representing an increase of more than 1100% in just 5 years (Figure 1). Based on the identification of 1020 unique articles that had been published between January 1 and November 30, 2009, it is estimated that the number of articles that will be published in 2009 is 1112. No limitations such as route of exposure, nanomaterial type, or experimental system/model were applied to this search of the PubMed database. Thus, the results of this survey give a general indication of the volume of literature that is being generated just on the potential safety aspects of nanomaterials; it is not unreasonable to predict that an equally large amount of literature is being generated that pertains to the desired biological effects of these materials.
Scientific publications related to the potential adverse effects of engineered nanoparticles. The number of articles published in each of the past 10 years was identified by searching the PubMed database (http://www.ncbi.nlm.nih.gov/sites/entrez) using the search terms (nanoscale or nanoparticle or nanomaterial* or nanotoxic* or nanoscience or nanotechnolog*) and (safe* or toxic or toxicity or toxicology or hazard* or adverse effect). The search was conducted on December 2, 2009. The value for 2009 is a projected value that is based on the number of publications (1020) that were published from January 1 to November 30, 2009.
Developing strategies for assessing the safety of nanomaterials has been the topic of a number of recent review articles. 1,7 –9 The general consensus is that tiered approaches (ie, in vitro and in vivo screening studies followed by pivotal in vivo studies and supportive mechanistic studies) are best suited for this purpose. 9,10 This approach is not unlike the general safety testing strategy that is used in the development of pharmaceutical drug candidates and food additives. As highlighted by Ostrowski and coworkers, 11 however, most of the research on the safety of nanomaterials that has been conducted to date has tended to emphasize acute toxicity and mortality rather than chronic exposure and morbidity; in other words, the pivotal studies are lacking. Although it is important to characterize acute effects, it is chronic exposure to nanomaterials that arguably may be more relevant to occupational exposure scenarios and to food, pharmaceutical, and other applications. It is important to acknowledge that although engineered nanomaterials are being created and introduced to the market at a rapid rate (see http://www.nanotechproject.org/inventories/consumer/analysis_draft/ for consumer product information; last accessed March 22, 2010), in vivo testing of all forms of engineered nanomaterials as they emerge is simply not practical. Moreover, long-term in vivo testing is inherently high in cost in terms of time and of the number of animals and the amount of test material that are required. That being said, subchronic toxicity studies have been conducted with some of the more well-known and characterized nanomaterials, including 13-week inhalation studies with titanium dioxide (TiO2), 12 silver, 13 and carbon black 14 and a 13-week dietary study with selenium. 15 The results of these studies provide a basis for additional long-term studies that likely will be necessary as specific regulatory requirements evolve.
Physicochemical characterization of nanomaterials that are tested in toxicology studies is of utmost importance. Indeed, lack of adequate nanomaterial characterization limits the value and significance of a given study and renders it impossible to compare studies and recognize parameters that might influence toxicity. 1,16 Numerous nanomaterial characteristics can be measured and described, but attempting to satisfy a “laundry list” of parameters often is not practical or necessary. Efforts are underway at the international level to develop lists of key physicochemical parameters that are deemed essential to adequately characterize nanomaterials in studies of their biological activity, and several groups have put forth their proposed lists. 10,16 –20 To date, however, an agreement has not been reached on an adequate and suitable base set of such parameters.
Although increased attention is being given to ensuring appropriate physicochemical characterization of nanomaterials in studies of their potential toxicity, it is apparent that sufficient attention is not always being paid to the reliability of the studies themselves. This is unfortunate, given the possibility that toxicological studies conducted with nanomaterials may be reviewed at some point in regulatory evaluations of one form or another. Such evaluations may, for example, include those for new substance notifications or new drug applications in various regulatory regions or for registration under the Registration, Evaluation, Authorisation and Restriction of Chemical substances (REACH) regulation in the European Union. Accordingly, adequate reporting of the methodological details and experimental results of a given study is crucial in ensuring a high degree of confidence in the data and conclusions that are generated from that study.
To assist with the design and interpretation of studies that examine the potential toxicity of nanomaterials, this article outlines a 2-step method to assess and rank their overall quality. The method involves evaluation of both the reliability of a given study and the adequacy of the characterization of the physicochemical parameters of the nanomaterial/nanomaterials that is/are examined in that study.
Methods
The method is a 2-step process. The first step involves the use of a publicly available tool to rank the reliability of a given study based on the adequacy of its design and documentation of methods, materials, and results; this provides a “study score.” The second step involves the determination of the completeness and reporting of the physicochemical characterization of the nanomaterial/nanomaterials that is/are being assessed within the study; this provides a “nanomaterial score.” The “study score” and the “nanomaterial score” are combined to derive a nano study score, a 2-part numerical value that provides an indication of the overall quality of the study. The 2 steps that are used to derive a nano study score are described individually below.
Step 1: Assessment of Study Reliability
The first step uses the ToxRTool, a software-based tool that was designed, tested, and released for public use by the European Centre for the Validation of Alternative Methods (ECVAM). It is freely available for noncommercial use at the ECVAM Web site (http://ecvam.jrc.it/; “Publications” section) and its development and testing were described in detail by Schneider and colleagues.
21
This tool is based on the categorization system for toxicological and ecotoxicological data that was proposed by Klimisch et al.
22
There are 2 versions of the tool, one for evaluating in vivo studies and the other for evaluating in vitro studies. Based on yes/no answers to a number of questions related to study design, the test article, the test organism, and the results documentation for a particular study, a study is assigned a calculated score of 1 (
Information Required for a Study to Be Assigned to Klimisch Category 1 or 2 a
a Also described as “red criteria.”
b Refers to the criteria number in the ToxRTool spreadsheets for in vivo and in vitro studies (available for download at http://ecvam.jrc.it/; “Publications” section).
Step 2: Assessment of Nanomaterial Characterization
The second step of the method involves determining the completeness of assessing the physicochemical parameters of the nanomaterial/nanomaterials that is/are being investigated within a given study and, by default, the adequacy of reporting of these parameters. As mentioned earlier, numerous groups have put forth recommended lists of nanomaterial physicochemical parameters for which determination is deemed valuable. These lists of parameters are presented in Table 2 . As can be seen, the lists proposed by different groups differ regarding some of the parameters that have been suggested. On inspection, however, it can be seen that there are commonalities among the lists. Thus, rather than recommending that the parameters for determination be selected from one of these lists over another one, the lists were compared to derive a new one composed of the most common parameters among them.
Physicochemical Parameters Recommended to Be Determined When Conducting Toxicological Studies With Engineered Nanomaterials
a An “X” indicates that the parameter was listed in a given reference as a recommended physicochemical parameter.
An amalgamated list of key parameters was derived based on a review of those presented in Table 2. A parameter was included in our list of key parameters if it were proposed as a suggested parameter in at least 3 of the 6 references that were reviewed. 10,16 –20 The amalgamated list of key parameters is presented in Table 3 .
Short List of Key Physicochemical Parameters to Be Determined When Conducting Studies of the Potential Toxicity of Engineered Nanomaterials a,b
Abbreviations: NCI, National Cancer Institute; NCL, Nanotechnology Characterization Laboratory.
a Specific parameter not listed but likely captured under another (surface characteristics) that is listed on the NCI/NCL Web site.
b Specific parameter not listed but captured under another (Zeta potential) that is listed on the NCI/NCL Web site.
In addition to this list of key physicochemical parameters, characterization of nanoparticles in the relevant experimental media (cell culture media, dosing solution, etc) for a particular study is an important consideration when assessing the results of studies conducted with nanoparticles. The significance of this aspect of nanoparticle testing has been stressed by others, 23 and we concur that it is desirable to document these parameters in the experimental exposure media to the greatest extent possible. This is due to the fact that some physicochemical parameters are likely to differ depending on whether they are determined in the experimental media or in the bulk (ie, “as received”) state. Thus, this was added to the list of key parameters outlined in Table 3. We have recommended the reporting of these characteristics of nanomaterials used in food-related studies as well. 24
The final list of 10 parameters to assess the adequacy of nanomaterial characterization is presented in alphabetical order below. Based on whether information related to these parameters was reported, a given study can be assigned a “nanomaterial score” of 0 ( agglomeration and/or aggregation chemical composition crystal structure/crystallinity particle size/size distribution purity shape surface area surface charge surface chemistry (including composition and reactivity) whether any characterization was conducted in the relevant experimental media.
Overlapping Considerations
We are aware that in our 2-step method there is some overlap between the assessment of study reliability and the assessment of nanomaterial characterization. In particular, it is noted that information relevant to criteria numbers 2 (Is the purity of the substance given?) and 4 (Is all information on the nature and/or physicochemical properties of the test item given, which you deem
In step 1, for criteria number 2 (related to purity of the substance) of the ToxRTool, we recommend indicating whether the purity of the nanomaterial has been provided and scoring the criteria as a 1 (
Also in step 1, for criteria number 4 (
First, the purpose of step 1 of the overall assessment is to judge the quality of the study as a whole; specific assessment of nanomaterial characterization is conducted in step 2. Thus, as criteria number 4 is not one of the “red criteria” for determining a study score, giving a study the benefit of the doubt on this single step 1 criterion is unlikely to significantly affect the study’s overall Klimisch ranking. If doing so improves the ranking of a study from Klimisch category 3 to Klimisch category 2, then the study obviously already contains substantial flaws in other aspects of its design and/or reporting, otherwise it would not be on the border between a Klimisch 2 and 3 ranking. In such a situation an evaluator would have the option (as they always do) to manually assign the study to a different Klimisch category, in this case to category 3, and to justify their reason for doing so by citing the numerous other flaws contained within it.
Second, at this point in time it is unclear which parameters may be considered necessary to judge nanomaterial characterization as being “sufficient” in a given study. Arbitrarily stating, for example, that a study must report at least 4 of the 10 parameters for characterization to be considered sufficient is not justifiable, as some parameters may be more important for one nanomaterial than they are for another. Attempting to rank the importance of the physicochemical parameters in a standardized manner also is not considered a viable option at this time because of the uncertainty surrounding which parameter should be considered the most useful, the second most useful, and so on for all nanomaterials. The most important parameter of a specific nanomaterial to be reported for a toxicity study may also differ depending on the biological response being measured and the route of exposure. Characterization parameter lists that are nanomaterial-specific may be generated in the future to address this issue, but such lists are not considered viable at the current time. It is anticipated that these issues will be clarified as experience is gained with the characterization and toxicity testing of specific (classes of) nanomaterials.
Sample Applications of the Method
An example of a study deemed to be of high overall quality is the one by Folkmann et al 25 in which oxidative damage to DNA was examined in rats following a single oral exposure to C60 fullerenes or single-walled carbon nanotubes. Female Fischer 344 rats were administered a single oral gavage dose of fullerenes or single-walled carbon nanotubes (each at 0.064 or 0.64 mg/kg body weight) suspended in saline or corn oil (which were used as negative controls). Measurements of oxidative DNA damage and of messenger RNA (mRNA) expression levels of various DNA repair enzymes were made in the colon mucosa, liver, and lungs at 24 hours post dosing. The authors reported increased levels of 8-oxo-7,8-dihydro-2'-deoxyguanosine (8-oxodG) in the liver and lungs but not in the colon. With one exception, no significant effects of nanoparticle dosing were noted on mRNA expression levels of various enzymes involved in removing 8-oxodG from DNA. This study garnered a study score of K1 (reliable without restrictions) in step 1 of the 2-step assessment method. A positive control was not included in the study, but the authors refer to a 2008 published article that includes positive control data from a concurrent study from their laboratory; this reference to positive control data is deemed acceptable for a nonguideline study of this type. This study garnered a nanomaterial score of N7 in step 2 of the 2-step assessment method. A considerable amount of characterization was conducted in the dosing vehicles (saline and corn oil) and the only parameters not reported or discussed were crystal structure/crystallinity (not necessarily relevant), surface charge, and surface chemistry. Thus, the nano study score for this study is K1-N7, indicative of high overall quality.
An example of a lower quality study is that by Zhang et al
26
in which the subacute oral toxicity of nano-selenium was examined. Male Kunming mice were orally administered nano-selenium (20-60 nm; doses ranging from 2 to 6 mg/kg body weight/d) for 12 or 15 days, after which they were killed and livers and blood examined for evidence of oxidative stress and toxicity. The authors reported that nano-selenium was generally less toxic to mice than was sodium selenite (a macro-size comparator). This study garnered a calculated study score of K1 (
The preceding examples highlight the use of the 2-step method to assess the overall quality of nanomaterial toxicity studies. Those which provide detailed experimental procedures and results along with extensive information on nanomaterial physicochemical parameters are considered of better quality and thus receive higher nano study scores than do those lacking in either of these categories. We are currently using this assessment method to evaluate the safety implications of nanoscale science and technology in food and food applications. A sampling of the range of nano study scores for oral toxicity studies assessed in our comprehensive review is provided in Table 4 ; details of our evaluation and expanded discussions on each assessed study will be published separately (manuscript in preparation).
Representative Examples of Nano Study Scores for Publications of In Vivo Studies Relevant to the Oral Toxicity of Food and Food-Related Nanomaterials
Discussion
This article outlines a strategy that can be used to assess the overall quality of studies that examine the toxicity of engineered nanomaterials, regardless of the route of administration that is being used. The 2-step approach generates a nano study score for a given study that is composed of a “study score” (indicating the reliability of the study) and a “nanomaterial score” (indicating the completeness of nanomaterial physicochemical characterization). This approach is encouraged to promote the notion that for studies conducted with nanomaterials, the combination of a reliable study and sufficient nanomaterial characterization within that study is of significantly greater value than either of these alone. Indeed, science is based on the ability to test, reproduce, and build upon the reported results of others, and the best quality scientific manuscripts and study reports provide information that allows the scientific process to proceed as openly and effectively as possible. When critical information is lacking from a manuscript or study report, it becomes very difficult for a hypothesis or finding to be verified by others.
For obvious reasons, determining the reliability of a given study is conducted as the first step of our method; it is deemed of minimal value to proceed with determining the extent of nanomaterial characterization for a study that is judged unreliable because of other factors. Thus, only studies that are placed in Klimisch category 1 or 2 are recommended to be further assessed for their sufficiency in reporting physicochemical data for the nanomaterial that was studied. Using this approach, an “ideal” study would be one that is judged to be in Klimisch category 1 and that receives a nanomaterial score of 10, providing a nano study score of K1-N10 (the K and N represent Klimisch [ie study] and nanomaterial scores, respectively; Figure 2). A lesser quality study, for example, would be one that is judged to be in Klimisch category 2 and that receives a nanomaterial score of 3, for a nano study score of K2-N3 (Figure 2). Unreliable studies (Klimisch category 3) are assigned a nano study score of K3-X (the X represents the fact that a “nanomaterial score” was not calculated). The possible range of nano study scores and their interpretations are provided in Figure 2. As was demonstrated earlier, use of the 2-step method to assess published studies by Folkmann et al 25 and Zhang et al 26 revealed that they fall into the higher (K1-N7) and lower (K2-N2) quality zones of the range, respectively.
A schematic depiction of the assessment of the overall quality of a nanomaterial toxicity study based on its derived nano study score. The clear area represents the range of nano study scores for which a study can be considered of high overall quality; conversely, the shaded area represents the region of low overall quality.
As for any study, those that are conducted with nanomaterials need to consider the importance of general study design and adequate documentation of methods, materials, and results. It is somewhat disconcerting that one can identify peer-reviewed published articles in which it is difficult or impossible to discern simple yet critical experimental details, such as the number of animals per treatment group, the exact route of exposure, or the doses or concentrations of the substance that was tested.
30
As is evident from the essential reliability criteria outlined by Schneider et al
21
(Table 1), publications must contain information such as this to be considered for assignment to Klimisch category 1 (
The categories proposed by Klimisch et al 22 and the development and application of the ToxRTool 21 were intended for the assessment of toxicological data, and this also is what we propose that it be used for. Nonetheless, there is no foreseeable reason these tools cannot also be used, perhaps with some modifications, to determine the reliability of data from studies that are designed to assess the desired biological activity (as opposed to the undesired or toxic activity) of substances, including nanomaterials.
Various methods can be used to determine the different physicochemical parameters of nanomaterials, and we do not advocate the use of one over another for any given parameter. Rather, we place a degree of trust in those who are doing the primary research that they are familiar with and confident that the methods that they use are the most appropriate for characterizing their nanomaterial/nanomaterials of interest. It is recognized, however, that for some nanomaterials there are some physicochemical parameters that may turn out to be difficult or even impossible to describe or measure with accuracy. An example of this is purity. As highlighted recently by Walker and Bucher,
31
an engineered nanomaterial may be composed of multiple entities, have a distribution profile for its primary particle size, may aggregate, and may also have a surface coating. In cases such as this, it is unclear how purity (and deviation from the stated purity) would best be described. It also is recognized that there are parameters in the list that is used in step 2 of our method that are not relevant to specific nanomaterials and that may negatively affect a study’s nanomaterial score as a result. For example, crystallinity is not applicable to carbon nanotubes, and so assigning a score of 0 (indicating
It is envisioned that this 2-step method may be applicable in many ways, including in aiding the development of toxicological screening methods for (classes of) nanomaterials. Specifically, the use of a standardized method to assess manuscript quality would provide a consistent and transparent mechanism by which to judge, compare, and position data from toxicity studies conducted with nanomaterials. This would allow for the identification of strengths and weaknesses of study design and data reporting, thereby constructively affecting the development and validation of toxicological screening methods such as those outlined by Oberdörster et al and Warheit et al. 9,10 In this regard, researchers can use the method as a guideline as they design and conduct their studies of the biological activities of nanomaterials and prepare grants or manuscripts for submission. Likewise, journal reviewers and editors can use it to judge the quality of submitted manuscripts. The method also may find use in various types of regulatory reviews of submission packages that cite data from studies conducted with nanomaterials. Based on what they are using it for, however, it is the user of this 2-step method who is ultimately responsible for critically evaluating and determining the relevance of the information that it provides (and of the information contained within the study report or publication that was assessed). We are currently using the nano study score assessment method to evaluate the safety implications of nanoscale science and technology in food and food applications, the results of which will be published separately.
In summary, it is anticipated that the use and evolution of this 2-step approach will assist with the design, reporting, review, and interpretation of studies evaluating the potential toxicity of nanomaterials.
Footnotes
Acknowledgments
We acknowledge the financial support from the Institute of Food Technologists (IFT), the Grocery Manufacturers Association (GMA) Science and Education Foundation, and the Technical Committee on Food and Chemical Safety of the International Life Sciences Institute (ILSI) North America.
The author(s) declared a potential conflict of interest (eg, a financial relationship with the commercial organizations or products discussed in this article) as follows: The authors are employed by Cantox Health Sciences International, a scientific consulting company with interests in safety and regulatory aspects of nanotechnology.
The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: The work described in this manuscript was partially supported with funds from the Institute of Food Technologists (IFT), the Grocery Manufacturers Association (GMA) Science and Education Foundation, and the Technical Committee on Food and Chemical Safety of the International Life Sciences Institute (ILSI) North America as part of an ongoing project to assess the safety of oral exposure to nanoscale materials.