Abstract
I argue that marking a proper distinction between two types of research in the cybersecurity field obviates the present debate concerning a “science of cybersecurity.” Once the terminology has been properly disambiguated, the accurate descriptor for the current state of the practice of cybersecurity becomes apparent: it is a protoscience. Further, once a definition for ‘science’ is specified, it becomes clear that the protoscientific state is capable of trending in the proper direction, namely toward science and away from pseudoscience.
1 Introduction
In a recent publication, Alexander Kott notes that “the central role of models in science is well recognized; it can be argued that a science is a collection of models, or that a scientific theory is a family of models or a generalized schema for models.” (Kott,
1
p. 30). Although there are alternative proposed solutions to the demarcation problem, taking Kott’s model-based approach to defining science affords the opportunity to address—and make progress toward solving—the recent debate concerning a potential “science of cybersecurity.”
i
Kott references the debate in terms of a lacuna in our present categorization of cybersecurity research: even for those in the cyber security community who agree with the need for a science of cyber—whether it merits an exalted title of a new science or should be seen merely as a distinct field of research within one or more of [the] established sciences—the exact nature of the new science, its scope and boundaries remain rather unclear. (Kott,
1
p. 1)
Kott’s statements provide the backdrop for two tasks, both of which I aim to accomplish here. The first is to articulate, in an accessible fashion, a disambiguation of the phrase “science of cybersecurity.” Doing so addresses Kott’s query about the scope of a science of cybersecurity. The second task is a consequence of the first: it is an assessment of the current state of the practice of cybersecurity in terms of whether it is, or can be, scientific.
I argue that marking a proper distinction between two types of research in the cybersecurity field obviates the present debate concerning a science of cybersecurity. Once the terminology has been properly disambiguated, the accurate descriptor for the current state of the practice of cybersecurity becomes apparent: it is a
Section 2 is spent identifying the distinction between the referents of the terms
2 Cybersecurity: Science and practice
The process of disentangling
By contrast, is it clear that cybersecurityp is distinct from the (potential)
The second indicator arises from the traditional scope of scientific research. The distinction between cybersecurityp and cybersecuritys fits the landscape of “traditional” sciences. Consider physics, for example. Physicists do physics: they generate theories, they think of clever ways to test the theories, and they experiment to instantiate the clever tests. Additionally, however, there is a set of people who study what the physicists do, how the physicists operate, and what the implications are of what the physicists are reporting. These people—philosophers of physics—study the way the practice of physics is undertaken, as well as the implications of the discoveries of the physicists. The philosophers of physics focus on experimental method, the epistemological foundations that ground the claims of the physicists, and the like. There are other groups of this sort for other sciences, too: philosophers of biology, philosophers of cosmology, and so forth. Collectively, the group comprises the philosophy of science field. Sometimes, the scientists themselves are also philosophers of their respective sciences, but those cases are fairly rare. By and large, but not exclusively, a researcher of the sort under discussion here is chiefly a scientist (a physicist, a biologist, a cosmologist, etc.) or chiefly a philosopher of science. Correspondingly, the distinction between cybersecurityp and cybersecuritys fits this structure: cybersecurityp researchers correspond to the scientists, and cybersecuritys researchers correspond to the philosophers of science. In that sense, cybersecuritys could instead be labeled “philosophy of cybersecurity.” iv
The third indicator that cybersecurityp and cybersecuritys are distinct fields of research is that the distinction enables the evaluative element of science to arise for cybersecurity. Just as the physicist could do physics research either scientifically or unscientifically, so, too, a cybersecurityp researcher is capable of undertaking cybersecurityp research scientifically or unscientifically.
v
Note, importantly, that in both cases, the scientist still undertakes the “base” activity: the physicist still undertakes physics, and the cybersecurityp researcher still undertakes cybersecurityp research, regardless of whether the research is done scientifically or unscientifically. Neither researcher undertakes the philosophical task: the physicist (in the example) is not undertaking philosophy of physics while performing physics research, and the cybersecurityp researcher is not undertaking cybersecuritys research while performing cybersecurityp research. These are different fields, with different targets of study. The same separation holds true for the evaluation of the research, as well: the cybersecurityp researcher, regardless of whether he or she does the work scientifically or not, still does not undertake cybersecuritys research. If we fail to distinguish cybersecurityp from cybersecuritys, we are left only with “cybersecurity research” as a singular bulk entity, and our hunt for a “science of cybersecurity” reduces to the oversimplification of attempting to judge whether cybersecurity research as a whole is scientific. No unifying answer will surface from that inquiry (indeed, none has thus far), chiefly because that task is ambiguous. To disambiguate the task, we need the capability to evaluate cybersecurityp research and distinguish
Two small tangent discussions involving the qualities of cybersecuritys and cybersecurityp are in order before proceeding.
vi
The first involves defining the members of the group labeled “cybersecurityp researchers.” The uniqueness of the computing discipline blurs the line that separates engineers and scientists in that domain.
vii
The blurring introduces a potential complication: insofar as the narrative thus far has correlated cybersecurityp researchers with “scientists,” it is unclear (for example) whether industry software engineers are scientists—cybersecurityp researchers—in the same way that university computer science department faculty members are scientists. In the present context, the complication is sidestepped by recognizing that it is not a researcher’s
Second, with respect to the domains of the two fields, the inherent adversarial nature of cybersecurityp reveals an interesting difference between its output and the output of some of the traditional sciences to which it has been analogized thus far. Because cybersecurityp researchers operate within a dynamic (adversarial) environment, the results of their research often are rendered obsolete by their adversaries. Accordingly, cybersecurityp researchers must frequently revisit (and revise) previously-obtained results. Traditional scientists (physicists, for example) face no such dynamic environments. The results obtained by those scientists (assuming the results are veridical) remain valid and can be built upon, so the need rarely arises to return to square one. This distinction potentially explains an apparent mismatch between traditional scientists and cybersecurityp researchers. Whereas traditional scientists are able to proceed, as part of the traditional scientific trajectory, from obtaining their results to interpreting them (for example, by attempting to generate laws to describe large swaths of results), cybersecurityp researchers instead must contend with adversarial actions that render obsolete their previously-obtained results. viii In these cases, tasks (such as the foundational unification of data and results into law-like generalities) that would normally be undertaken by scientists (cybersecurityp researchers) fall instead into the domain of cybersecuritys researchers.
Because cybersecurityp is in a nascent state, it is only beginning to develop its own set of researchers focused on cybersecuritys. At this early stage, the cybersecuritys literature arises mainly from a select few researchers in cybersecurityp who branch out with the occasional publication addressing cybersecuritys. Examples of this sort include Roy Maxion (with his focus on experimental practice in cybersecurityp), Fred Schneider (with his focus on urging cybersecurityp researchers to search for potential scientific laws of cybersecurityp), and Alexander Kott (with his approach to defining science and evaluating the fit between that definition and cybersecurityp).1,11,12 Although the bulk of the work done by these researchers lies in the cybersecurityp domain, they address cybersecuritys when they study the way cybersecurityp is undertaken.
Although cybersecuritys could be referred to by a different label (perhaps, as suggested earlier, ‘philosophy of cybersecurity’), it has instead received the ambiguous label ‘science of cybersecurity’, which has generated the confusion that the present work is aiming (in part) to disentangle. Indeed, the distinction between cybersecurityp and cybersecuritys has, in fact, been identified in several places in the literature. For example, the abstract to the 2010 JASON report opens by stating: JASON was requested by the DoD to examine the theory and practice of cyber-security, and evaluate whether there are underlying fundamental principles that would make it possible to adopt a more scientific approach, identify what is needed in creating a science of cyber-security, and recommend specific ways in which scientific methods can be applied. (JASON,
13
p. v)
In the present terminology, JASON was contracted to investigate cybersecurityp to determine whether cybersecuritys could be developed. The following year, the US Government issued its strategic plan for cybersecurity research, in which it is noted that “the science of security has the potential of producing universal laws that are predictive and transcend specific systems, attacks, and defenses.” (Executive Office of the President, 14 p. 11). It is not cybersecurityp that would produce the universal laws; rather, cybersecuritys would generate the laws that would thereby facilitate a higher level of success for cybersecurityp. ix
Nonetheless, in the years since, the debate concerning the “science of cybersecurity” has devolved due to the loss of keeping the referents of cybersecuritys and cybersecurityp distinct from each other. To illustrate, I will close this section with a clear, extended example of the field’s deep-rooted conflation of cybersecuritys with cybersecurityp. Although what follows is merely a single episode, it accurately exemplifies the lack of the distinction between cybersecuritys and cybersecurityp in the cybersecurity field.
In a two-issue burst in 2012, the National Security Agency’s
Then, in 2015, It is essential that this new science be grounded on common definitions. Throughout the years, there has been much debate about the nature of science—what it is, and what methods are best. The works of Karl Popper on falsifiability, of Pierre Duhem on the testing of hypotheses as parts of whole bodies of theory, and of Thomas Kuhn on shifts in scientific paradigms, are fine examples of this. No doubt we will continue that broader discussion in relation to security science… (Krohn,
17
p. i)
The mention of Popper, Duhem, and Kuhn, all of whom are renowned philosophers of science, highlights the earlier-articulated relationship between philosophers of science and cybersecuritys researchers. The inclusion of those philosophers of science in the discussion appears to bode well for the establishment of the distinction between cybersecuritys and cybersecurityp. However, what follows the ellipsis in the quote above reveals that cybersecuritys and cybersecurityp are, instead, viewed as interchangeably as they ever have been: “…but that is not our interest here.” (Krohn,
17
p. i). Krohn then describes the contents of the issue, which is dedicated to “the next move” in “building a science of cybersecurity”: This issue of
Despite directly distinguishing cybersecuritys research and accurately identifying it as the philosophy of science correlate to cybersecurityp, Krohn proceeds to conflate cybersecuritys with cybersecurityp by classifying material that unambiguously falls under cybersecurityp as “research contributing to the development of security science.” The result is that the issue dedicated to building a science of cybersecurity—dedicated to cybersecuritys research of the sort initially mentioned by Krohn in his column—instead is filled with cybersecurityp research.
We see, then, that when one inquires whether there can be a “science of cybersecurity,” there are two significant questions being asked simultaneously:
“Can cybersecurityp be scientific?”
“Can cybersecuritys exist?”
The “debate” about the issue arises when discussants intend to address one interpretation of the question without acknowledging which interpretation is intended. By distinguishing cybersecurityp from cybersecuritys, we are able to navigate the ambiguity.
3 Cybersecurity as protoscience
Karl Popper notes that “science must begin with myths – and with the criticism of myths; neither with the collection of observations, nor with the invention of experiments, but with the critical discussion of myths, and of magical techniques and practices.” (Popper,
18
p. 177). This longer view of scientific development involves the
From this protoscientific state, two possible trajectories follow. On one trajectory, the protoscience could develop the observation-gathering, experiment-designing, model-based capabilities cited by Popper and Kott as the requisite qualities for genuine science. On the other trajectory, the protoscience could remain in the myth-criticizing state whereby its techniques and practices are neither fundamentally understood nor rigorously reliable. This latter type of development is one from protoscience to
Distinguishing cybersecurityp from cybersecuritys leaves us in position to ascertain where cybersecurityp rests on the trajectory of scientific development. More specifically, because classifying cybersecurityp is a task that lies under the purview of philosophy of science, we see that assessing the scientificity of cybersecurityp is a cybersecuritys task. xiv The argumentation contained in the previous section, then, can be put to work to indicate whether cybersecurityp is scientific. It will be cybersecuritys, particularly with respect to its overlap with philosophy of science, that will provide answers about whether cybersecurityp is scientific.
In fact, there is a very clear sense in which the “science of cybersecurity” case is an ideal candidate for adjudication via the philosophy of science literature. Sven Ove Hansson characterizes the denotation of our use of the word ‘science’ as follows: It [our usage of the term ‘science’] can focus on the descriptive contents, and specify how the term is actually used. Alternatively, it can focus on the normative element, and clarify the more fundamental meaning of the term. The latter approach has been the choice of most philosophers writing on the subject. (Hansson,
8
p. 4)
We see (from the material presented in Section 2) that the cybersecurity literature currently conflates the two possible denotations—cybersecurityp and cybersecuritys—of our usage of the ‘science of cybersecurity’ terminology. These correspond to Hansson’s descriptive and normative uses, respectively. Cybersecurityp is the term that describes the activity—the practice—of cybersecurity. By contrast, cybersecuritys is the term that captures the “normative element,” the “fundamental meaning” of the term ‘cybersecurity,’ in terms of the discipline’s properties and whether they qualify the practice of cybersecurity as scientific. Accordingly, by distinguishing the two uses, we are in position to put that distinction to work by using cybersecuritys to analyze cybersecurityp. The remainder of this section takes the first steps—but only the first steps—down that path.
It is worth considering the scientific status of cybersecurityp because understanding that status can guide our expectations about the evolution of the field and the results we hope to achieve via cybersecurityp. Indeed, perhaps the impetus behind the recent urgency to establish cybersecuritys is the fear that cybersecurityp could end up being pseudoscientific. This is the first clue that cybersecurityp is protoscientific: those who are pioneering the cybersecuritys literature raise concerns that cybersecurityp exhibits traits that are symptoms of a pseudoscience. For example, nearly all calls for the development of a “science of cybersecurity” cite the present lack of predictive ability, the lack of repeatable results, and the need to resort to reacting to attacks (rather than the ability to anticipate them) that pervade cybersecurityp: We need a In cyber security we also need measurements that are dependable and error-free; undependable measurements make for undependable values and analyses, and for invalid conclusions. A rigorous experimental methodology will help ensure that measurements are valid, leading to outcomes in which we can have confidence (Maxion,
11
p. 344).
These claims suggest a concern that cybersecurityp is pseudoscientific. This would cast cybersecurityp with the same lot, for example, as astrology, climate change denial, and psychokinesis. Those pseudosciences present sparse and uncontrolled evidence for their claims, fail under comprehensive scrutiny, lack metrics for accurate measurement, and are unable to generate the reliable predictive claims that legitimate sciences are capable of generating. Some of those qualities appear to be applicable to cybersecurityp presently.
To characterize these concerns a bit more formally, we return to the work of Hansson. Hansson advances, through extended analysis, the following definition for ‘pseudoscience’: “A phenomenon is
Concerning (1), cybersecurityp has not yet reached a state where its models are well-established. Indeed, a driving motivation for this special issue is to investigate the very utility of models and their use in cybersecurityp. Finer features of the models that Kott deems worthy to generate a scientific cybersecurityp include the requirements that the models:
Are expressed in an appropriate rigorous formalism;
Explicitly specify assumptions, simplifications and constraints;
Involve characteristics of threats, defensive mechanisms, and the defended network;
Are at least partly theoretically grounded;
Yield experimentally testable predictions of characteristics of security violations (Kott, 1 p. 3). xvii
The fledgling cybersecuritys work that exists directly challenges the applicability of many of the features on Kott’s list to the existing models in cybersecurityp, which thereby challenges whether such models qualify as “well-established.” For example, Maxion et al. 21 lament the lack of transparency in cybersecurityp experimentation. Schneider 12 notes the present lack of scientific laws in cybersecuritys research, which impacts the scope of the models that can be generated for use in cybersecurityp. Rossow et al. 22 paint a grim picture of the scientific legitimacy of experimentation in malware research. Hatleback and Spring 10 identify the difficulty of generating sound experimental design when dealing with the engineered mechanisms that are prevalent in computing. This sample of the cybersecuritys literature shows that it would be quite difficult to argue convincingly that the current models in place in cybersecurityp could be characterized as “well-established”. xviii
Concerning (2), the analysis is not as clear-cut. From one perspective—that of the cybersecuritys researchers—it is very clear that there are no illegitimate claims of scientificity for cybersecurityp occurring. This much is evident from the selection of the literature that was cited above, all of which identify the places in which cybersecurityp currently falls short in terms of the qualities articulated by Kott. However, one could argue that the very subset of cybersecurityp researchers who are the targets of that cybersecuritys literature are illegitimately creating the impression that their cybersecurityp models are well-founded. ixx To adjudicate this discrepancy, it is helpful to appeal to a unique feature of the cybersecurity field that makes an important difference. Because the cybersecuritys discipline is so nascent, its researchers are, in fact, still heavily involved in cybersecurityp. Thus, despite the presence of some cybersecurityp researchers who may be illegitimately creating the impression that the models of cybersecurityp are well-founded, it certainly is not the case that, on the whole, the proponents of cybersecurityp are doing so. In this respect, it is the cybersecuritys researchers who are shielding cybersecurityp from a potentially pseudoscientific fate.
So, where does that leave the status of cybersecurityp? Since Hansson’s (1) is satisfied, cybersecurityp is not scientific (presently). However, since Hansson’s (2) is
4 Conclusions
The history of science is a story of disciplines evolving and devolving, finding success and suffering failure. The discipline of cybersecurity is now entering that story. Only hindsight yields the perspective necessary to categorize a discipline accurately. Indeed, as Kuhn observed in the work in which he first defined ‘protoscience,’ I claim no therapy to assist the transformation of a proto-science to a science, nor do I suppose that anything of the sort is to be had…. A sentence I once used when discussing the special efficacy of mathematical theories applies equally well here: “As in individual development, so in the scientific group, maturity comes most surely to those who know how to wait.” Fortunately, though no prescription will force it, the transition to maturity does come to many fields, and it is well worth waiting and struggling to attain. Each of the currently established sciences has emerged from a previously more speculative branch of natural philosophy, medicine, or the crafts at some relatively well-defined period of the past. Other fields will surely experience the same transition in the future. Only after it occurs does progress become an obvious characteristic of a field. (Kuhn,
19
p. 245)
Sometimes, even hindsight is not enough. Renaissance-era alchemy was protoscientific, yet historians remain conflicted about whether it evolved to scientific success in the form of chemistry, or whether it devolved to pseudoscientific failure in the form of a panpsychic, spiritual, or Jungian mysticism. 24 Even now, then, it is not clear whether Renaissance-era alchemy was pre-scientific or pre-pseudoscientific. And even in cases where hindsight reveals an incontrovertible pseudoscience, such as the astrology that was practiced contemporaneously with alchemy, scientific giants like Johann Kepler can be found partaking in the pseudoscience. xx At this early stage of cybersecurity research, then, it seems premature to label cybersecurityp anything but a protoscience; it is a field for which, as Kuhn would surely agree, we cannot yet judge whether its progress will become substantial enough to cross the threshold into the maturity of established science.
The adoption of a model-driven paradigm in the discipline of cybersecurity offers the opportunity for its transition to maturity, particularly if one adopts a model-based definition for science such as the one formulated by Kott. However, for the history of science to include that story, researchers must mark the distinction between cybersecurityp and cybersecuritys. The protoscience of cybersecurityp is capable of transitioning into a science, but it will take the development of cybersecuritys to enable that transition.
Footnotes
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
