Abstract
Various systems for prioritizing biological agents with respect to their applicability as biological weapons are available, ranging from qualitative to (semi)quantitative approaches. This research aimed at generating a generic risk ranking system applicable to human and animal pathogenic agents based on scientific information. Criteria were evaluated and clustered to create a criteria list. Considering availability of data, a number of 28 criteria separated by content were identified that can be classified in 11 thematic areas or categories. Relevant categories contributing to probability were historical aspects, accessibility, production efforts, and possible paths for dispersion. Categories associated with impact are dealing with containment measures, availability of diagnostics, preventive and treatment measures in human and animal populations, impact on society, human and veterinary public health, and economic and ecological consequences. To allow data-based scoring, each criterion was described by at least 1 measure that allows the assignment of values. These values constitute quantities, ranges, or facts that are as explicit and precise as possible. The consideration of minimum and maximum values that can occur due to natural variations and that are often described in the literature led to the development of minimum and maximum criteria and consequently category scores. Missing or incomplete data, and uncertainty resulting therefrom, were integrated into the scheme via a cautious (but not overcautious) approach. The visualization technique that was used allows the description and illustration of uncertainty on the level of probability and impact. The developed risk ranking system was evaluated by assessing the risk originating from the bioterrorism threat of the animal pathogen bluetongue virus, the human pathogen Enterohemorrhagic Escherichia coli O157:H7, the zoonotic Bacillus anthracis, and Botulinum neurotoxin.
Several biological agents (bacteria, viruses, and toxins) can cause severe diseases and are seen as a potential threat with regard to bioterrorism. Besides well-known agents like Bacillus anthracis or Botulinum neurotoxin, viral hemorrhagic fever viruses and newly emerging agents like the yellow fever virus are also listed in various prioritizing lists.2-5
In order to set guidelines for decision makers and stakeholders, it is necessary to conduct a risk assessment of specific biological agents. One way of doing so is to create a risk ranking that categorizes the agents according to their feasibility to be used as threat agents as well as their impacts in case of use in a bioterrorism incident. 6
A ranking system can be created by various methods—either a qualitative or a (semi)quantitative approach can be used—and the exact definition of risk must be determined. Fosse et al. established a quantitative approach based on the creation of a typology of hazards and calculated a risk score to support veterinary public health decision makers. 7 This very precise system is generated out of detailed data for each ranked agent.
The crucial point for conducting a reliable quantitative risk ranking is the quality of underlying data. For exact quantitative risk rankings, the data necessarily have to be quantitative and precise. Unfortunately, these exact data are not always available for emerging or rare pathogens. Additionally, the collection of quantitative data is usually only feasible in crisis preparation, whereas time limitations in emergency situations would not allow the collection of information with the necessary exactness.
In contrast, Davis 8 as well as Franz et al. 9 and Irlenkäuser 10 developed qualitative risk ranking systems that allow differentiation between hazardous agents and probably nonthreatening agents in an approximate manner and without gradation between risk originating from the different agents. However, for optimal preparedness and containment, it would be useful to separate the ranked agents according to their posed threat and relevance. A qualitative risk ranking, in contrast to a quantitative risk ranking, is often based on expert opinions or estimated data, and therefore it can be conducted even if there is a medium to low level of knowledge about an agent.5,11-15 A middle approach is risk evaluation in a semiquantitative manner. This is a compromise regarding the preciseness of the results, but the complexity and duration of this kind of assessment as well as its dependence on exact data is lower than for quantitative risk rankings.2,13,15-18
A detailed overview on available risk assessment methods, their focus, and risk ranking structures is presented elsewhere. 19 It is most crucial to develop a risk ranking scheme for highly pathogenic agents that is generic and therefore usable for all fields of application and all target populations. 19 This would ideally lead to a universally accepted risk ranking system for the support of decision makers and stakeholders in emergency situations.
Methods
Based on the literature survey already published, we revised the criteria and compared them in order to get an overview of the most important factors various authors identified for their risk ranking systems. 19 Since most criteria describe only abstract concepts, like accessibility, it was additionally necessary to determine measures (desirably measurable parameters) that characterize each criterion explicitly. The measures were chosen carefully so that they would be unambiguous within the set of criteria; it also was preferable that they represent parameters that can be characterized with explicit information like a number, defined range, or clear definitions and facts. Special care was taken that criteria were formulated that describe unique concepts and that could be assigned unambiguously to categories. Criteria that were originally used to describe different categories in other ranking systems were evaluated according to their main contributing aspect and assigned to the respective category. The same approach was used on the measure level. Each measure was assigned to 1 criterion so that each criterion was described by at least 1 measure. Relevant criteria were clustered in 11 categories representing miscellaneous perspectives with regard to a bioterrorism incident, starting with the agent selection and ending with the impact on public health and other consequences caused by the release of the agent (see Figure 1).
Flow chart of categories used for ranking highly pathogenic biological agents to assess the risk related to a bioterrorism scenario (right-hand side: list of the 11 categories, white boxes contain fact and analytical categories; grey boxes with black barriers include categories with criteria and measures for the containment of the agent)
As a next step, a literature search was performed regarding the range of data of measures and agents to assess which measures can be used to describe high-risk biological agents comprehensively and objectively. According to our literature survey, it was assessed whether data for the measures could be obtained in general or for the examined highly pathogenic agents in particular. For example, the measure “stability of the agent in the environment” was deleted from the criteria set because no complete global data coverage exists that would presume to make a point regarding this complex measure. Furthermore, it has to be considered that, for example, the measure “stability of the agent in the environment” cannot be unambiguously assigned to 1 criterion or category. It can be used to describe the prevalence of the agent in the environment and therefore the accessibility of a specific agent to a perpetrator. Additionally, stability can be used to describe the shelf-life of a biological agent and thus the production efforts and storability. Moreover, stability has implications for the agent's dispersal in various matrices and hence influences directly human and veterinary health. In order to avoid overlap in the criterion “accessibility in general,” the measure “stability” was not used additionally, since it was already included in the measure “prevalence in the environment.” Also, some measures were deleted because appropriate data provision turned out not to be feasible.
In summary, the assignment of the measures was used to describe the criteria as exactly as possible. So the criteria cannot be equated with data regarding the biological agents because the defined criteria often are described by a combination of several measures to include different aspects. This constitutes the main difference with other risk ranking systems of highly pathogenic biological agents where criteria are used directly for the characterization of the agents.13,14
As a result of this approach, we created a hierarchical evaluation schema that is grounded on 51 measures describing 28 criteria, which in turn are unambiguously assigned to 11 descriptive categories that encode the risk perspectives “probability” and “impact” (Table 1).
Overview of the Used Categories, Criteria, and Measures Including the Binned Classes in the Risk Ranking System
Classification of case numbers was chosen to display the approximate ratio (1:10) of specific population numbers in Europe to global count.
BSL-3** = from Technical Rules for Biological Agents (TRBA, Germany). The risk to workers of infection is limited because infection by the airborne route is not normally possible.
LD50 = lethal dose 50%.
cfu = colony-forming units.
pfu = plaque-forming units.
BW = body weight
In order to allow a comparative ranking of different high-risk biological agents, each measure was graded in a class ranging from “1” to “4” following the method of McKenzie et al. 20 The gradation into 4 classes allows a clear and easily distinguishable subdivision of values for each measure and avoids center values for measures where expert opinion is needed. It is well known from surveys that the center value is predominantly chosen for measures where no explicit quantitative data are available but a qualitative estimation is at choice. It has to be kept in mind that the value of “1” refers to the classification of “none to low.” This implies that a disease with no morbidity is scored in the lowest possible class (1), which includes as well, for example, diseases that have low lethality.
Depending on the source employed and the information available, agent data were classified into 1 of the 4 classes according to the adopted gradation. If epidemiologic data were available, the median was calculated to eliminate discordant values—for example, for the number of cases, morbidity rate or case-fatality rate. For this purpose, the median (eg, of numbers of cases) of the past 5 years was calculated to depict the most comprehensive picture of the disease. If no epidemiologic data were available, arithmetic means of mentioned values out of several references were used to characterize the specific biological agent. Additionally, the minimum and maximum values of the measure values formed the basis for generating possible minimum and maximum classes to illustrate the bandwidth of the real data including natural variations. This concept was also used in other risk ranking systems. 5
Then, the final value of each criterion was calculated as the arithmetic mean value of all measures associated with the respective criterion. Subsequently, the arithmetic mean values of all criteria values were used to compose a mean value for their assigned category. These mean category values are in turn used to calculate the 2 multiplying perspectives of risk: probability and impact as arithmetic means of the category values (Figure 2).
Description of the calculation of risk from measures, criteria, and categories to probability or impact and risk. Values for each criteria, category, and probability and impact originate from the arithmetic mean of the level before. In contrast, risk is defined as the product of probability and impact. 21
The same calculations were also conducted for the minimum and maximum values on each level to allow for the visualization of uncertainty. Risk is defined as the product of probability and impact by the ISO definition as well as by Salerno et al. and Tucker21-23 and is therefore calculated this way in the ranking schema as well. This enables the visualization of risk in a probability-impact matrix according to Salerno et al. 22
All calculations and visualization were performed using a VBA (Visual Basic for Applications) program in combination with a Microsoft Excel® database that provides the necessary data. Inserted raw data are automatically categorized in the presented classes, calculation steps as described above are conducted, and final values are determined and visualized. The presented risk ranking system additionally provides the possibility for weighting on each of the hierarchical levels (measures, criteria, categories). These weighting factors could be applied by skilled users to focus the risk ranking on specified populations or to adapt the system to individual demands. For a basic risk ranking, as presented in this article, the values for the biological agents were calculated using equal weighting factors of 1 on each level.
Results
The grading system of the developed risk ranking is presented in Table 1. The table describes which categories determine the risk perspectives probability and impact. The chosen categories for the description of probability were the history of use, the accessibility of the agent, and possible paths of introduction and contamination as well as the feasibility of agent production and storage. For the complete estimation of the impact, it is necessary to consider not only the agent's effects on human and veterinary public health, but also possible countermeasures that could contain the spread of the disease (vaccination and treatment) within the possible target populations. In addition, the diagnostic aspects as well as the economic and socioeconomic consequences and potential for panic represent issues that influence the impact caused by a specific biological agent. Moreover, Table 1 illustrates the assignment of criteria to the specific category for a more precise description of the aspect as well as an explanation and the describing measurable parameters (measures) for each criterion. Furthermore, the underlying classification of the measures with its class limits is given.
Some measures that are used in other publications for the description of a specific criterion or category might be arranged in a different manner in the developed ranking system. For example, “number of human/animal cases in Europe/world” is not used to describe how hazardous an agent is because of its naturally occurring prevalence but to depict the perspective of accessibility to the specific biological agent for a potential perpetrator.5,13,14,24
By using the minimum and maximum values of the measure values, the basis for the generation of possible minimum and maximum classes was provided to illustrate the data variation caused by differences in data availability or natural variability, for example. It is possible that some measures like “case-fatality rate” cover broad ranges, with the result that the variation of data for this measure would include 2 or even 3 classes. In order to solve this problem, a mean or if possible a median was averaged over the range of epidemiologic data—for example, a value of 22.5% as the arithmetic mean if the bandwidth was 5% to 40%. The variation was delineated through the mentioned minimum and maximum measure classes (on the basis of the chosen example with a minimum of 5% and a maximum of 40%). These minimum and maximum measure classes were thus considered when calculating the value of the criteria. Like the minimum and maximum criteria classes, minimum and maximum category values also were generated and could be shown visually as data uncertainty boxes in the resulting probability-by-impact matrix (presented in Figure 3 on the basis of 4 biological agents).
Risk matrix of 4 selected biological agents illustrating probability and impact calculated by the generic risk ranking system. The symbols represent the mean value calculated for the respective agents, whereas the lines of the surrounding box represent possible minimum and maximum values for probability and impact resulting from the data's natural variability and uncertainty on measure-level. Both axis scales derive from the gradation of classes from 1 to 4. The more toward the top right the agents are located in the risk matrix, the higher their risk characterized by probability and impact.
This means that we neither ignore uncertainty by assigning no value nor delete the specific criterion or measure. Instead, we outline and represent the uncertainty in a clearly structured and comprehensible manner. The uncertainty boxes around the scores represent minimum and maximum values of available data. Uncertainty is particularly great for many exotic or newly emerging viruses. Risk analysts can assess what the risk of a specific biological agent might be by examining the uncertainty boxes. Every value within the area of the uncertainty boxes is realistic and could occur, but statistically the median or mean value is the most probable value and therefore used as a central value in the risk ranking (Figure 3).
Another challenge that emerged during the data collection for high-risk biological agents constitutes the entire absence of specific data for some measures or agents. Criteria for which enough data are not available cannot be ignored by deleting those criteria or measures because this pretends some kind of certainty. There are several possible ways to handle uncertainty caused by a lack of data, each having specific disadvantages:
25
1. The deletion of measures, as performed by Branquart,
26
would lead to an incompleteness of information and to misinterpretation for some exotic or newly emerging diseases. The results of the risk ranking would overestimate the relevance of available information. Furthermore, comparative considerations between the agents would not be possible because of a consequently different composition of criteria and foci in the characterization of the pathogens. 2. The assignment of the minimum value to measures with missing information; this would cause an underestimation of the specific criteria and the risk thereof. 3. The attribution of the maximum value to measures, as, for example, by McKenzie et al.;
20
unknown diseases that are mostly insufficient characterized would be overestimated concerning their specific risk.
In order to solve this problem, the measure values were assigned to values ranging from the minimum to the maximum possible item (eg, a case-fatality rate from 0% to 100%). Converted into the 4-classes gradation, a minimum class of 1 and a maximum class of 4 with a mean value of 2.5 would arise. As a consequence, a cautious but not overcautious estimation of unknown values was adopted in the calculations through this procedure, a method that has already been used in other publications. 27
Consequently, the authors decided to mention if there is a lack of data when collecting information on specific highly pathogenic biological agents and to represent the uncertainty through uncertainty boxes. If the available literature is almost complete for the characterization of the agent and natural variation of data is low, the uncertainty boxes are smaller in comparison to a newly emerging disease with lack of data. The less data available for a specific biological agent, the more uncertain becomes the consideration, characterized by larger uncertainty boxes.
Four selected biological agents are presented as an example in a risk-matrix in Figure 3, illustrating their risk, composed of probability and impact. The presented generic risk ranking system poses the unprecedented possibility of categorizing agents with multiple impacts by considering the comprehensive aspects on various populations as described above. Additionally, it includes a system that allows for a weighting to focus on a selected population according to the user's demands. Current systems rank biological agents for specific purposes, such as animal diseases or human public health issues, but do not include criteria for an overall risk ranking. The agents were chosen to exemplify differences and difficulties in illustrating solely animal pathogenic agents (bluetongue virus), mainly human pathogenic agents (Enterohemorrhagic Escherichia coli [EHEC] O157:H7), the zoonotic noncontagious Botulinum neurotoxin, and the zoonotic agent Bacillus anthracis. The risk of all these different agents can be calculated likewise by the presented risk ranking system, which allows a direct comparison of the risk posed by human, animal, and zoonotic pathogens. As the developed risk ranking system considers effects on both human and animal populations, those agents that cause damage in both populations are ranked highest, whereas agents that are pathogenic in only 1 population get lower probability and impact scores, since some measures considering the unaffected population are scored with low class affiliation. The uncertainty boxes of different sizes for the 4 selected agents show that the data coverage with regard to information needed for the risk ranking system is more profound for bluetongue virus and EHEC O157:H7 than for Bacillus anthracis and Botulinum neurotoxin (Figure 3).
Discussion
A new generic risk ranking system for comparing highly pathogenic agents through the consideration of various aspects of the risk related to their potential usage as bioterrorism agents has been developed and was tested for 4 biological agents. The ranking results are based on scientific facts and data (referred to as measures) as they are currently available for each agent. In the presented basic configuration of the ranking system, each measure contributes equally to the score of the corresponding criterion, meaning that it does not matter how many single measures characterize a criterion since all criteria have identical weights against each other. This kind of equal contribution was used in the whole risk ranking system when calculating the risk considering human, animal, and zoonotic aspects in similar amounts. Nevertheless, the weighting of different criteria can be conducted by the user, for example, depending on a unique target population or a specific perception by assigning different weights to categories, criteria, or even measures. This allows the user of the risk ranking system to exclude criteria and categories (by assigning a weight of 0) that do not contribute to the risk that a particular biological agent poses to a specific target population. For example, all criteria dealing with human aspects (eg, treatment in human population) are irrelevant when calculating the specific risk for an animal population and could therefore be assigned a weight of 0.
On the level of measures as describing instrument for the criteria, it is not critical that the measures of different criteria have varying weights in comparison to each other (eg, one-eighth weight of a measure of a criterion described by 8 measures as opposed to a one-quarter weight of a measure of a criterion described by 4 measures), since the measures are used only for the calculation of a data-based score for each criterion. So the score of a criterion and not of a measure determines the risk estimate.
The use of mean values in the calculation led to the approximation of values of more and less hazardous agents at the middle of possible values. 25 We are aware of this fact and accept this condition given by the calculation, since the multiplication within the measure, criterion, or category level would lead to an overemphasis of the agents' differences. The authors had to decide whether to obtain the approximation or the spreading out of results and chose the approximation since the pure totaling or multiplication would lead to preweighting due to the different number of measures per criterion.
As soon as specific data for the individual agents have been entered into the database, the criteria set can be used for arbitrarily chosen biological agents.
Comparing results from our risk ranking system with other schema, it becomes evident that there are clear differences, especially for diseases that affect solely the human or the animal population versus agents that can infect both populations. At this juncture, all measures and criteria are weighted equally regardless of whether humans or animals are seen as the target population. However, since a zoonotic agent affects both populations, more measures will be scored higher, resulting in higher mean values for the categories (eg, in the category “accessibility” the measures “number of human cases” and “number of animal cases” are included). Higher values for the criteria lead to higher values for the corresponding categories and consequently to a higher ranked probability and impact that determine the specific risk.
Nevertheless, it is possible to compare agents that are only pathogenic to humans regarding their risk to the human target population through the adjustment of the multiplying weighting factors. The same procedure can be conducted for animal pathogens and their probability and impact in the animal target population. The adaptation of the risk ranking scheme through weighting on measure and criteria levels as well as on the category level dependent on the target population or the considered focus is easy to perform.
In comparison to other risk ranking systems, the measures and criteria of the established system cover almost every subject area and risk connected with a natural, accidental, or deliberate release of highly pathogenic biological agents.
Most other risk ranking systems focus on particular risks, like the introduction into a specific country or the impact solely on the human population.5,14 By contrast, the present risk ranking system allows the ranking of high-risk biological agents (human pathogenic, animal pathogenic, and zoonotic agents) with potential for bioterrorism application and can be applied to almost every biological agent. Furthermore, 51 measures describing 28 criteria allow a more comprehensive description of risk by consideration of numerous aspects, compared to systems outlining only a smaller number of criteria.5,13,18,28-30 Other risk ranking systems have chosen to tackle the complex topic of ranking highly pathogenic biological agents by different methods—for example, a probabilistic decision tree approach.31,32
The present system is most suitable for ranking highly pathogenic agents since pathogens that can be found naturally in human and animal populations in high prevalence and that cause simultaneously no symptoms or only mild disease might be ranged higher in probability due to elevated scores (eg, in the accessibility category). Nevertheless, endemic nonhazardous agents would not obtain high total risk scores because of very low results in impact and the multiplying calculation method of risk.
The advantages of the developed risk ranking system for classifying highly pathogenic biological agents may be summarized as follows:
• It is adaptable to different target populations. • It considers a remarkably broad set of aspects in its criteria. • It is applicable for low and high data availability, while uncertainty and even natural variation of data are considered and visualized. • The weighting is adaptable on every level (measures, criteria, and categories), for example, for adaptation to different target populations. • The class limits of all measures may be adjusted by the user according to particular requirements. • The formation of a risk value/risk ranking is transparent.
The developed risk ranking system could support the target user group represented by decision makers and stakeholders by providing the needed information and facts that might be important for an extensive risk assessment in emergency situations. The semiquantitative approach, in contrast to the quantitative approach, has the advantage of requiring less time for the collection and evaluation of data while providing all necessary information. The present risk ranking system is intuitively understood by the target user group and could consequently contribute to strategic decisions and an effective immediate reaction in threat situations of natural or deliberate origin. Furthermore, the developed risk ranking can be used to categorize biological agents in risk groups as executed for diverse risk lists (eg, references 2, 4, 33) or to compare agents that are mostly not listed, like bluetongue virus, with agents quoted in these risk lists. It can therefore be used as decision support for researchers and decision makers when considering the classification of newly emerging pathogens (eg, in CBRN legislation). By means of the introduced risk ranking system with its comprehensive approach ranking, experts and scientists can also be assisted in detecting vulnerable points in combination with vulnerability analysis of facilities.
The authors developed a highly adjustable system for the comparable assessment of highly pathogenic biological agents that could be used for bioterrorism threats. It is especially characterized by its broad coverage of aspects within the considered categories and criteria and by its adaptability to specific target populations or to a population spanning analysis.
Footnotes
Acknowledgments
This research was supported by and executed in the framework of the EU project AniBioThreat (Grant Agreement: Home/2009/ISEC/AG/191) with financial support from the Prevention of and Fight against Crime Programme of the European Union, European Commission—Directorate General Home Affairs. This publication reflects the views only of the authors, and the European Commission cannot be held responsible for any use that may be made of the information contained therein.