Sage Journals: Discover world-class research

Abstract

Extant scholarship has until now relied on informal-theoretic, case study, and interpretative methods to assess patterns of norm development in cyberspace. Ideally, these accounts would be complemented with more systematic cross-national and longitudinal empirical evidence. To address this gap, this article introduces the International Cyber Expression Dataset. The dataset includes a corpus of more than 34,000 official expressions of view by states and their authorized representatives regarding the international politics of cyberspace. The article describes the sources of these data and demonstrates the dataset’s usefulness, with an Online appendix containing an exploratory analysis of norm convergence. Future research can leverage the dataset to empirically test questions of theory and policy. For example, the dataset can be used to study how foundational theories of norm diffusion apply to cyberspace. It can also be paired with existing cyber conflict datasets to study the conditions under which state practice influences cyber discourse, and vice versa.

Keywords

cyber conflict cyber norms dataset discourse international law

Cyber scholarship has, in recent years, embarked on a more empirical turn. Theoretical predictions are ubiquitous, but testing and adjudicating between those predictions are challenging owing to the unique features of cyberspace (Gorwa and Smeets, 2019; Shore, 2022; Whyte, 2018). As Shandler and Canetti (2024) describe, ‘the [cyber] domain is complex, quality data is sparse, [and] affairs are shrouded in secrecy.’ To that end, recent work has attempted to quantify patterns in a number of important issue areas: cyber conflict and escalation (Kostyuk and Zhukov, 2019; Valeriano and Maness, 2015); strategy and capacity (Kostyuk, 2021; Valeriano et al., 2018); proxy warfare (Akoto, 2022; Borghard and Lonergan, 2016; Canfil, 2022; Herzog, 2011; Leal and Musgrave, 2022; Maurer, 2018a); psychology and decisionmaking (Gomez, 2019; Gomez and Whyte, 2022; Gomez and Villar, 2018; Hedgecock and Sukin, 2023; Kostyuk and Wayne, 2021; Shandler et al., 2021, 2022) and more. Despite this trend, the study of cyber norms has not received similar treatment.

To address this gap, this article introduces the International Cyber Expression (ICE) dataset. Just as other articles in this issue that contribute to our understanding of cyber conflict by harnessing new data sources (Kostyuk, 2024; Makridis et al., 2024; Oppenheimer, 2024), the ICE dataset offers a new way for researchers to conduct large-n analyses of discourse and norm evolution in cyberspace. Because raw expression text is included, scholars may also find it useful for locating specific cases for qualitative analyses. Building on existing projects that catalog norm-building activity, it systematically documents sovereign expressions of view on cyberspace issues. The dataset includes $N = 34, 103$ observations at the country-expression level, totaling 60 million words. In addition to the corpus, the dataset includes more than 100 variables derived from both original and pre-existing works. A detailed description of the dataset, as well as its limitations, are discussed here and in the codebook.

In studying how political actors work to advance ‘rules of the road’ in cyberspace (Broeders et al., 2022; Giles, 2012; Mazanec, 2015; Raymond, 2019), scholars have relied primarily on informal-theoretic, case study, and interpretative methods to develop and evaluate theories. Such approaches have offered valuable insights. However, they often have unavoidable cross-national, context-specific, and temporal limitations. The United Nations Group of Governmental Experts (UN GGE) continues to dominate cyber norm scholarship (Maurer, 2011, 2018b). UN GGE proceedings are private, however, meaning that scholars cannot directly observe the norm development process, which Finnemore and Hollis (2019) argue is crucial for understanding how norms develop.¹ As we are reminded, many ‘useful lessons [are] missed if one examines cybernorms only as products’ (Finnemore and Hollis, 2016). Focusing on GGE outcome documents, which are only produced in the event of consensus, also has the unfortunate effect of selecting on success cases. Most importantly, the prevailing focus on the GGE elides the many expressions of view that states have exchanged on cyber issues in other venues.

The ICE dataset remedies this shortcoming by cataloguing sovereign expressions in 57 different institutional settings since 1998. Rather than select on consensus outcomes, it catalogues what might be termed ‘opinio publicae’ – publicly-staked opinions about how the international relations of cyberspace ought to operate. Unlike analyses scoped exclusively on the GGE, researchers can use the ICE dataset to better separate cyber norm candidates from norm successes; track their progress over time; and identify the conditions under which they are contested or accepted by different parties. In so doing, the dataset offers a window into ‘norm nebulas’ – clusters of normative ideas and aspirations that their advocates would like to see become the norm.²

Novel approaches to the empirical study of cyber norms are needed now more than ever. Practitioners such as Admiral Michael S. Rogers, formerly the director of United States (US) Cyber Command (CYBERCOM), have long argued that ‘shared norms are a basic building block for cybersecurity’ (Farrell, 2015). However, GGE member states unexpectedly failed to reach consensus in 2017 (Mačák, 2017), fueling cynicism about the future of cyber norms (Maurer, 2020; Segal and Waxman, 2011; Schmitt and Vihul, 2017). States are now at a ‘crossroads’ (Hoffman et al., 2020) in setting rules of the road for cyberspace conduct (Schmitt, 2020). Scholars and policymakers alike disagree on what it will take to get cyber norms (Healey and Maurer, 2017) and where the road ahead might lead (Demchak, 2018; Ford, 2010; Healey, 2011; Markoff and Kramer, 2009; Maurer, 2020; Maurer and Taylor, 2018; Schmitt, 2013). Despite the importance of norms for regularizing interaction in the cyber domain, ‘failure remains an option’ and ‘may even be the dominant outcome’ (Finnemore and Hollis, 2016).

The prospects for cooperation in cyberspace are thought to be limited because states possess ‘radically incompatible values’; a belief that the USA, which benefits disproportionately from the open internet, is ‘acting in bad faith’ given the Snowden disclosures; and the fact that technology companies and non-governmental organizations have such considerable sway (Farrell, 2015). Others argue that because of enduring domestic disagreements, the USA has struggled to articulate a coherent position on the international stage (Creemers, 2017). Moreover, the attribution problem (Libicki, 2009), asymmetric capabilities (Nye, 2010), and indistinguishability between offense, defense, and espionage (Mazanec, 2015) make norms difficult to enforce, decreasing their credibility. Despite an abundance of cynicism, however, states continue to advocate for norms.

Researchers can use these data for a variety of purposes. For norm researchers per se, potential applications include questions ranging from how conventional norm theories apply to cyberspace to the effectiveness of entrepreneurial norm-building efforts. It can be used to study discourse around specific cyber issues, measure tacit convergence between opposing factions, and the use of similar language by different actors across multiple venues. This could include changes to how states frame their vital interests to international audiences, such as cyber sovereignty (Fung, 2023; Kolton, 2017), or how they speak to benefactors and dependents (see Kostyuk, 2024). Finally, scholars can also use it to empirically test predictions about how norms affect conflict processes, for example by pairing ICE data with existing datasets on cyber operations (e.g. Kostyuk and Gartzke, 2023; Maness and Valeriano, 2016), cyber institutions (e.g. Kostyuk, 2021), or other measures of state practice.

The remainder of this article describes the motivation for the dataset, provides an overview of what it contains, and offers several suggestions for future research applications. An empirical example is given in the Online appendix.

Introducing the dataset

This section provides a brief overview of the International Cyber Expression (ICE) dataset. ICE identifies the country that expressed an opinion pertinent to cyberspace as well as the date (YYYY/MM/DD) on which this opinion was expressed. More than 200 sovereign entities are attributed, including all 193 UN member states. The mean expression length is approximately 2000 words. Including raw text, expression metadata, and links to the original source, the dataset contains 123 variables for user convenience.

ICE uses country-expressions as its default unit of analysis. This means that a row is created whenever a country is identified as having issued, officially endorsed, or publicly subscribed to an expression. Data at the expression level are coarser than paragraph- or sentence-length configurations, but unlike granular approaches, the default configuration permits the study of ‘norm-grafting’ (Finnemore and Hollis, 2016), ‘rhetorical adaptation’ (Fung, 2023), and other discursive devices that holistically pair multiple issues in the same context. Information about relative topical content is included for weighting and subsetting purposes. Researchers may of course disaggregate or re-aggregate the data to fit the desired unit of analysis.

Expressions meeting three criteria were subject to inclusion:

First, the theme of the expression must directly pertain to cyberspace. Keywords were used to identify pertinent data from non-cyber-specific sources.³ In cases where paragraph break information was available, paragraphs that did not include a cyber-related keyword were removed. Data from sources that already select on cyber themes are included in their entirety.

Second, the statement must be official. I define ‘official’ as emanating from government offices with de jure international legal standing. These include ambassadors and foreign ministry representatives, heads of state, heads of government, and their immediate deputies (Damrosch and Murphy, 2014). Whenever a high-level representative fitting this profile is attributed, or when the expression is issued directly by the top executive or foreign ministry, the expression is linked to that country. Accessions to other countries’ expressions are also coded this way.⁴ Expressions by government officials are not attributed if they include a disclaimer.

Third, the expression must be primarily intended for consumption by international audiences. Coding expressions that were made in multilateral or international fora was therefore straightforward. Unilateral expressions are included if they deal directly with issues that countries have actively discussed in international contexts. Legislation and other domestic policies are excluded.

Following these rules, country attributions and inclusion decisions were clear and uncontroversial. In

Table I.

Overview of variables by category

Category	Set size	Examples
Time	4	Date, year, quarter
Forum	1	UN, ITU, OEWG
Geography	4	Country, continent, UN subregion
Participants	4	Co-endorsers, forum co-members
Political bloc	47	NATO, Non-Aligned Movement, Shanghai Cooperation Organization, European Union
UN GGE	4	Current member, Chair, ever a member
Security council	4	P5, E10, Presidency
Internet capacity	13	Broadband connections, internet traffic, servers per capita
Legal system	4	Common law, civil law, customary law
Alignment	5	UN ideal point, faction
NLP	30	Dominant topic, sentiment, Flesch-Kincaid score
Other	–	Text, links, summaries

about 98% of cases, the exact date was readily identifiable. In situations where only the month is recorded, the expression is coded as having been made on the first of that month.

Table I summarizes the other variables in the dataset. They include a number of indicators for: (1) membership in any of 47 blocs; (2) GGE experience; (3) geography; (4) telecommunications capacity; (5) domestic legal tradition; (6) diplomatic alignment based on UN ideal point (UNIP) scores (Bailey et al., 2017); and (7) forum. Additional variables are derived using supervised Natural Language Processing (NLP) techniques. These include expression wordcount, linguistic complexity, and sentiment. The codebook contains a detailed breakdown of the coding scheme.

Finally, the dataset encodes basic information about expression topic content. An expression is coded in the dataset as being ‘about’ a given topic t if the estimated proportion of word tokens assigned to topic, $θ_{t}$ , exceeds that of all topics $\neg t$ . In other words, while expression may invoke many different topics, it is considered most associated with topic t whenever $θ_{t} > θ_{\neg t}$ . We can assume mixed membership is more likely when full expressions, as opposed to paragraphs or sentences, are treated as the unit of analysis. To account for topic mixtures at the default level, information on relative topic membership is also included. Some topics may be more substantively interesting than others. Researchers may use the default topic labels (or a custom procedure) to parse the dataset as desired.

Figure 1.

Geographic distribution of observed expressions

Two limitations are underscored. First, care should be taken when dealing with expressions made outside of a formal institutional setting; namely, many unilateral and bilateral expressions, which were identified via convenience sampling. These undoubtedly suffer from reporting bias, consistent with findings by Makridis et al. (2024) and Oppenheimer (2024) in this issue.⁵ Since unilateral/bilateral expressions are not assumed to be independent and identically distributed, they should be treated with care. These types are included by default, but can easily be removed depending on research aims.

Second, not all expressions are available in the public domain. In some cases, the text of unknown expressions is probably missing at random (including the London Process example); in others, states may have retracted their positions (Wuzhen, perhaps) – or off-the-record candor may have been the point (as with the GGE). The raw text of known expressions was recovered for 96% of the sample.⁶ Whenever possible, these are verbatim. Meeting summaries, press releases, or media statements are relied upon only as a last resort.

Content and utility

Scholars can use the dataset in at least two ways. First, expressions of interest can be surgically isolated for traditional qualitative analyses with greater convenience. Second, researchers interested in using large-n methods to better understand global and longitudinal discursive patterns can now leverage the same computational approaches that scholars are already using to study norm change in other domains, such as environmental politics and human rights (e.g. Arias, 2022; Fariss et al., 2015; Hanania, 2021; Park et al., 2020).

Figure 1 maps the global distribution of expressions captured by the dataset at the country level. Recall that only 44 countries have ever been invited to participate in a GGE meeting. This is less than a quarter of all UN members, and most of these have only participated once. Conversely, we observe from Figure 1 that every UN member has staked out at least some public positions on GGE-related matters. For sovereign states (entities like Montserrat are overseas territories), the lower bound is around 13,000 combined words. There are numerous ways researchers might leverage this information. For example, using this dataset, researchers can employ network models to track country- or system-level engagement patterns.

Figure 2 depicts the share of topical emphasis over time. Topics capture the various cyber-related issues that the international community has prioritized since 1998. The overall constitution of these topics provides an overview of dataset contents. As an illustration, trend lines are disaggregated into country ‘factions,’ obtained by splitting UN ideal point scores, previously compiled by Bailey (2017), into equal-sized tertiles.⁷ Higher UNIP scores correlate with developed democracies, while developing countries and especially authoritarian countries tend to have lower UNIP scores. The relationship between topical emphasis and faction membership is fitted by year.

As is apparent, there is considerable variation in what different types of countries tend to emphasize. However, there is also some surprising convergence; for example, the importance of addressing criminal misuse. For proofs of concept, the Online appendix tracks discourse around

Figure 2.

Relative emphasis across 12 issue spaces by faction

three norm candidates – information operations, military operations, and multilateral/multistakeholder inclusion – in more detail. Researchers are encouraged to make use of this dataset to test theories of their own.

Suggested applications

Implicit in the attention that policymakers and pundits have paid to cyber norms since 1998 is the idea that what states say to one another must somehow matter (Carr, 2015; Mueller, 2013), echoing literature in other domains (Goddard and Krebs, 2015; Krebs and Jackson, 2007; Risse, 2000). Even realists such as Carr (1964) elevate so-called discourse power – ‘power over opinion’ – alongside military and economic power (Kania, 2018b). ‘Cultivating norms requires serious effort,’ write Finnemore and Hollis (2016). States invest significant resources into international discussions. This alone would seem to offer prima facie evidence that states view international conversations not simply as ‘cheap talk,’ but as potentially impactful (Lonergan and Schneider, 2023).⁸

The conventional wisdom has long been that cyberspace is a ‘Wild West’ (Obama, 2015) where international law is purportedly ‘silent’ (Hollis and Sander, 2022; Kello, 2021; Sander, 2019). Many decried the breakdown at the 2017 GGE as the ‘end of the road’ (Henriksen, 2019), the ‘end of an era’ (Korzak, 2017), or the ‘end of cyber norms’ (Grigsby, 2017). However, the GGE has endured, in parallel with a new UN track, the Open-Ended Working Group (OEWG) (Grigsby, 2018). Dialogue is also widespread in other fora, a fact that is often overlooked, and more than half the observations in this dataset occur after the 2017 ‘failure.’ Scholars must recognize that the story has not reached its end (Raymond, 1997), animating the need to study these processes as they unfold.

The ICE dataset can empower researchers to make more ‘bold, theoretically grounded, and empirically tested claims’ like the articles in this issue (Shandler and Canetti, 2024). For one, there are still many unresolved questions about how core International Relations theories apply in cyberspace. Cooperation theorists might wish to examine how states ‘anchor’ or ‘graft’ new ideas onto existing norms (Finnemore and Hollis, 2016), ‘rhetorically adapt’ existing norms for new purposes (Fung, 2023), frustrate norm development through strategic obfuscations disguised as substantive contributions (Kania, 2018a; Lantis and Bloomberg, 2018; Pratt, 2019; Schneiker, 2021), or employ other evasive tactics (Búzás, 2018).

The data could also be used to explore how regional and global norms diffuse, or to make comparisons between directed (Keck and Sikkink, 1998) or cyclic (Sandholtz, 2008) processes. For example, Kostyuk (2024) argues in this issue that spillover between allies helps to explain cyber capabilities. Similarly, scholars might use this dataset to test predictions about how allies, ‘critical states’ (Finnemore and Sikkink, 1998), or highly networked ‘gatekeepers’ (Finnemore and Hollis, 2016) might influence foreign policy expressions. For researchers interested in state practice, the data are pairable with existing cyber conflict datasets. And, in addition to the possibility of being paired with data on cyber institutions (Kostyuk, 2021), the ICE dataset appends 13 measures of cyber capacity.

While the dataset is already useful for these purposes, it can be improved upon. First, future versions of the dataset could prioritize the systematic collection of unilateral/bilateral expressions and expressions at international gatherings where English is not necessarily the dominant language, such as the World Internet Conference in Wuzhen, China. Work in progress by Hulvey (2024) promises to address the latter shortcoming. Second, evidence of state practice – extradition agreements, Mutual Legal Assistance Treaties (MLATs), computer emergency response team (CERT) and Computer Security Incident Response Team (CSIRT) arrangements, court indictments and official attributions, other domestic policies, and technical correspondence – could also be included to study norm internalization, although this would involve a significant undertaking to expand the dataset's scope. Third, cybercrime-specific conversations from the UN Third Committee, which are not included in this version, would be relatively easy to incorporate. Finally, future versions should endeavor to include more 'unofficial' and civil society commentary. In the meantime, researchers can use the dataset to better understand cyber norms processes as they unfold in the most prominent fora.

Footnotes

Replication data

All analyses were conducted using R. The dataset, replication code, codebook, and Online appendix can be found at https://www.prio.org/journals/jpr/replicationdata. Updated versions will be posted to .

Acknowledgments

The author thanks Daphna Canetti, Michael Fischerkeller, Erik Gartzke, Richard Harknett, Jason Healey, Lucas Kello, Nadiya Kostyuk, Erik Lin-Greenberg, Jon Lindsay, Andrew Nathan, Jon Penney, Evan Perkoski, Mike Poznansky, Mark Raymond, Ryan Shandler, Brandon Valeriano, Chris Whyte, Josephine Wolff, JD Work, Thomas Zeitzoff, and others for helpful comments on earlier versions of this work. Thanks also to participants at the Center for Peace and Security Studies (cPASS), Columbia SIPA, the Digital Issues Discussion Group, and the University of Haifa for feedback on earlier versions. Gustaf Ahdritz and Hao Zhu provided excellent research assistance.

Funding

This research was supported in part by the National Science Foundation (NSF 19-612 2017591) and the Carnegie Corporation of New York.

ORCID iD

Justin Key Canfil

Notes

References

Akoto

(2022) Accountability and cyber conflict: Examining institutional constraints on the use of cyber proxies. Conflict Management and Peace Science 39(3): 311–332.

Arias

(2022) Who securitizes? Climate change discourse in the United Nations. International Studies Quarterly 66(2): sqac020.

Bailey

Strezhnev

Voeten

(2017) Estimating dynamic state preferences from United Nations voting data. Journal of Conflict Resolution 61(2): 430–456.

Bailey

(2017) East African states adopt China’s playbook on internet censorship. Available at: https://freedomhouse.org/article/east-african-states-adopt-chinas-playbook-internet-censorship (accessed 11 November 2020).

Borghard

Lonergan

(2016) Can states calculate the risks of using cyber proxies? Orbis 60(3): 395–416.

Broeders

de Busser

Cristiano

, et al. (2022) Revisiting past cyber operations in light of new cyber norms and interpretations of international law: Inching towards lines in the sand? Journal of Cyber Policy 7(1): 97–135.

Búzás

(2018) Is the good news about law compliance good news about norm compliance? The case of racial equality. International Organization 72(2): 351–385.

Canfil

(2022) The illogic of plausible deniability: Why proxy conflict in cyberspace may no longer pay. Journal of Cybersecurity 8(1): tyac007.

Carr

(1964) The Twenty Years’ Crisis, 1919–1939: An Introduction to the Study of International Relations. New York: Harper and Row.

10.

Carr

(2015) Power plays in global internet governance. Millennium: Journal of International Studies 43(2): 640–659.

11.

Creemers

(2017) Can the world agree upon cyberspace norms? China-US Focus, 14 August. Available at: https://www.chinausfocus.com/peace-security/can-the-world-agree-upon-cyberspace-norms(accessed 31 May 2020).

12.

Damrosch

Murphy

(2014) International Law, Cases and Materials (6th edn). St. Paul, MN: West Academic Publishing.

13.

Demchak

(2018) Three futures for a post-Western cybered world. Military Cyber Affairs 3(1): Article 6.

14.

Fariss

Linder

Jones

, et al. (2015) Human rights texts: Converting human rights primary source documents into data. PLoS One 10(9): e0138935.

15.

Farrell

(2015) Promoting norms for cyberspace. Technical report. Council on Foreign Relations. Available at: https://www.cfr.org/report/promoting-norms-cyberspace. (accessed 14 March 2023).

16.

Finnemore

Hollis

(2016) Constructing norms for global cybersecurity. American Journal of International Law 110(3): 425–479.

17.

Finnemore

Hollis

(2019) Beyond naming and shaming: Accusations and international law in cybersecurity. SSRN Scholarly Paper ID 3347958.

18.

Finnemore

Sikkink

(1998) International norm dynamics and political change. International Organization 52(4): 887–917.

19.

Fisher

Klein

(2020) FOCUSdata project. Technical report. Available at: https://focusdataproject.com/about/(accessed 29 March 2023).

20.

Ford

(2010) The trouble with cyber arms control. New Atlantis 29(Fall): 52–67.

21.

Fung

(2023) China’s use of rhetorical adaptation in development of a global cyber order: A case study of the norm of the protection of the public core of the internet. Journal of Cyber Policy 7(3): 256–274.

22.

Giles

(2012) Russia’s public stance on cyberspace issues. In: 4th International Conference on Cyber Conflict (CYCON 2012), Tallinn, Estonia, 5–8 June 2012, pp. 1–13. ISSN: 2325-5366.

23.

Goddard

Krebs

(2015) Rhetoric, legitimation, and grand strategy. Security Studies 24(1): 5–36.

24.

Gomez

Villar

(2018) Fear, uncertainty, and dread: Cognitive heuristics and cyber threats. Politics and Governance 6(2): 61–72.

25.

Gomez

Whyte

(2022) Cyber uncertainties: Observations from cross-national war games. In: Cavelty

Wenger

(eds) Cyber Security Politics. London: Routledge, 111–127.

26.

Gomez

MAN

(2019) Sound the alarm! Updating beliefs and degradative cyber operations. European Journal of International Security 4(2): 190–208.

27.

Gorwa

Smeets

(2019) Cyber conflict in political science: A review of methods and literature. Available at: https://osf.io/preprints/socarxiv/fc6sg/ (accessed 21 October 2022).

28.

Grigsby

(2017) The end of cyber norms. Survival 59(6): 109–122.

29.

Grigsby

(2018) Unpacking the competing Russian and US cyberspace resolutions at the United Nations. Council on Foreign Relations, 29 October. Available at: https://www.cfr.org/blog/unpacking-competing-russian-and-us-cyberspace-resolutions-united-nations (accessed 3 November 2019).

30.

Hanania

(2021) The humanitarian turn at the UNSC: Explaining the development of international norms through machine learning algorithms. Journal of Peace Research 58(4): 655–670.

31.

Healey

(2011) The five futures of cyber conflict and cooperation. Georgetown Journal of International Affairs 2011: 110–117. Available at: https://www.jstor.org/stable/43133819 (accessed 9 November 2020).

32.

Healey

Maurer

(2017) What it’ll take to forge peace in cyberspace. Christian Science Monitor. Available at: https://www.csmonitor.com/World/Passcode/Passcode-Voices/2017/0320/What-it-ll-take-to-forge-peace-in-cyberspace (accessed 2 January 2018).

33.

Hedgecock

Sukin

(2023) Responding to uncertainty: The importance of covertness in support for retaliation to cyber and kinetic attacks. Journal of Conflict Resolution 67(10): 1873–1903.

34.

Henriksen

(2019) The end of the road for the UN GGE process: The future regulation of cyberspace. Journal of Cybersecurity 5(1): tyy009.

35.

Herzog

(2011) Revisiting the Estonian cyber attacks: Digital threats and multinational responses. Journal of Strategic Security 4(2): 49–60.

36.

Hoffman

Hollis

Ruhl

(2020) Cyber norms processes at a crossroads. Lawfare, 26 February. Available at: https://www.lawfareblog.com/cyber-norms-processes-crossroads (accessed 26 October 2020).

37.

Hollis

Sander

(2022) International law and cyberspace: What does state silence say? In: Azaria

(ed.) State Silence Across International Law. Oxford: Oxford University Press (2023).

38.

Hulvey

(2024) Mobilizing for Sovereignty: How China's Vision of Order Gains Followers. PhD Dissertation, University of Pennsylvania.

39.

Kania

(2018a) China’s strategic ambiguity and shifting approach to lethal autonomous weapons systems. Lawfare, 17 April. Available at: https://www.lawfareblog.com/chinas-strategic-ambiguity-and-shifting-approach-lethal-autonomous-weapons-systems (accessed 31 March 2021).

40.

Kania

(2018b) The right to speak: Discourse and Chinese power. Center for Advanced China Research, 27 November. Available at: https://www.ccpwatch.org/single-post/2018/11/27/the-right-to-speak-discourse-and-chinese-power (accessed 13 July 2023).

41.

Keck

Sikkink

(1998) Activists beyond Borders: Advocacy Networks in International Politics (1st edn). Ithaca, NY: Cornell University Press.

42.

Kello

(2021) Cyber legalism: Why it fails and what to do about it. Journal of Cybersecurity 7(1): tyab014.

43.

Kolton

(2017) Interpreting China’s pursuit of cyber sovereignty and its views on cyber deterrence. Cyber Defense Review 2(1): 119–154.

44.

Korzak

(2017) UN GGE on cybersecurity: The end of an era? The Diplomat, 31 July. Available at: https://thediplomat.com/2017/07/un-gge-on-cybersecurity-have-china-and-russia-just-made-cyberspace-less-safe/ (accessed 14 March 2023).

45.

Kostyuk

(2021) Deterrence in the cyber realm: Public versus private cyber capacity. International Studies Quarterly 65(4): 1151–1162.

46.

Kostyuk

(2024) Allies and diffusion of state military Cybercapacity. Journal of Peace Research 61(1): 44–58.

47.

Kostyuk

Gartzke

(2023) Fighting in cyberspace: Internet access and the substitutability of cyber and military operations. Journal of Conflict Resolution 68(1): 80–107.

48.

Kostyuk

Wayne

(2021) The microfoundations of state cybersecurity: Cyber risk perceptions and the mass public. Journal of Global Security Studies 6(2): ogz077.

49.

Kostyuk

Zhukov

(2019) Invisible digital front: Can cyber attacks shape battlefield events? Journal of Conflict Resolution 63(2): 317–347.

50.

Krebs

Jackson

(2007) Twisting tongues and twisting arms: The power of political rhetoric. European Journal of International Relations 13(1): 35–66.

51.

Lantis

Bloomberg

(2018) Changing the code? Norm contestation and US antipreneurism in cyberspace. International Relations 32(2): 149–172.

52.

Leal

Musgrave

(2022) Cheerleading in cyberspace: How the American public judges attribution claims for cyberattacks. Foreign Policy Analysis 18(2): orac003.

53.

Libicki

(2009) Cyberdeterrence and Cyberwar. Santa Monica, CA: RAND.

54.

Lonergan

Schneider

(2023) The power of beliefs in US cyber strategy: The evolving role of deterrence, norms, and escalation. Journal of Cybersecurity 9(1): tyad006.

55.

Mačák

(2017) From cyber norms to cyber rules: Re-engaging states as law-makers. Leiden Journal of International Law 30(4): 877–899.

56.

Makridis

Maschmeyer

Smeets

(2024) If it bleeps it leads? Media coverage on cyber conflict and misperception. Journal of Peace Research 61(1): 72–86.

57.

Maness

Valeriano

(2016) The impact of cyber conflict on international interactions. Armed Forces & Society 42(2): 301–323.

58.

Markoff

Kramer

(2009) US and Russia differ on a treaty for cyberspace. New York Times, 28 June. Available at: https://www.nytimes.com/2009/06/28/world/28cyber.html (accessed 11 January 2018).

59.

Maurer

(2011) Cyber norm emergence at the United Nations—An analysis of the UN’s activities regarding cyber-security. Discussion paper. Science, Technology, and Public Policy Program, Belfer Center. Available at: https://www.belfercenter.org/publication/cyber-norm-emergence-united-nations-analysis-uns-activities-regarding-cyber-security (accessed 2 January 2018).

60.

Maurer

(2018a) Cyber Mercenaries: The State, Hackers, and Power. Cambridge: Cambridge University Press.

61.

Maurer

(2018b) Cyber proxies and their implications for liberal democracies. Washington Quarterly 41(2): 171–188.

62.

Maurer

(2020) A dose of realism: The contestation and politics of cyber norms. Hague Journal on the Rule of Law 12(2): 283–305.

63.

Maurer

Taylor

(2018) Outlook on international cyber norms: Three avenues for future progress. Just Security, 2 March. Available at: https://www.justsecurity.org/53329/outlook-international-cyber-norms-avenues-future-progress/ (accessed 10 May 2018).

64.

Mazanec

(2015) The Evolution of Cyber War: International Norms for Emerging-Technology Weapons. Lincoln, NE: University of Nebraska Press.

65.

Mueller

(2013) Networks and States: The Global Politics of Internet Governance. Cambridge, MA: MIT Press.

66.

Nye

(2010) Cyber power. Working paper. Belfer Center for Science and International Affairs. Available at: https://apps.dtic.mil/docs/citations/ADA522626 (accessed 15 May 2020).

67.

Obama

(2015) Obama: Cyberspace is the new ‘Wild West’. NPR, 13 February. Available at: https://www.npr.org/sections/thetwo-way/2015/02/13/385960693/obama-to-urge-companies-to-share-data-on-cyber-threats (accessed 21 January 2019).

68.

Oppenheimer

(2024) How the process of discovering cyberattacks biases our understanding of cybersecurity. Journal of Peace Research 61(1): 28–53.

69.

Park

Greene

Colaresi

(2020) Human rights are (increasingly) plural: Learning the changing taxonomy of human rights from large-scale text reveals information effects. American Political Science Review 114(3): 888–910.

70.

Pratt

(2019) Norm transformation and the institutionalization of targeted killing in the US. European Journal of International Relations 25(3): 723–747.

71.

Raymond

(1997) Problems and prospects in the study of international norms. Mershon International Studies Review 41(2): 205–245.

72.

Raymond

(2019) Social Practices of Rule-Making in World Politics (1st edn) . New York: Oxford University Press.

73.

Risse

(2000) ‘Let’s argue!’: Communicative action in world politics. International Organization 54(1): 1–39.

74.

Sander

(2019) The sound of silence: International law and the governance of peacetime cyber operations. In: 11th International Conference on Cyber Conflict: Silent Battle (eds Minárik

Alatalu

Biondi

, et al.), Tallin, Estonia, 28–31 May, pp. 361–382. NATO CCD COE.

75.

Sandholtz

(2008) Dynamics of international norm change: Rules against wartime plunder. European Journal of International Relations 14(1): 101–131.

76.

Schmitt

Vihul

(2017) International cyber law politicized: The UN GGE’s failure to advance cyber norms. Available at: https://www.justsecurity.org/42768/international-cyber-law-politicized-gges-failure-advance-cyber-norms/ (accessed 31 May 2020).

77.

Schmitt

(2013) Tallinn Manual on the International Law Applicable to Cyber Warfare (Reprint edn). Cambridge and New York: Cambridge University Press.

78.

Schmitt

(2020) Taming the lawless void: Tracking the evolution of international law rules for cyberspace. Available at: https://tnsr.org/2020/07/taming-the-lawless-void-tracking-the-evolution-of-international-law-rules-for-cyberspace/ (accessed 15 July 2020).

79.

Schneiker

(2021) Norm sabotage: Conceptual reflection on a phenomenon that challenges well-established norms. International Studies Perspectives 22(1): 106–123.

80.

Segal

Waxman

(2011) Why a cybersecurity treaty is a pipe dream. Available at: https://www.cfr.org/node/160209 (accessed 11 January 2018).

81.

Shandler

Canetti

(2024) Special issue introduction: Cyber-conflict – moving from speculation to investigation. Journal of Peace Research 61(1): 3–9.

82.

Shandler

Gross

Canetti

(2021) A fragile public preference for cyber strikes: Evidence from survey experiments in the United States, United Kingdom, and Israel. Contemporary Security Policy 42(2): 135–162.

83.

Shandler

Gross

Canetti

(2022) Cyberattacks, psychological distress, and military escalation: An internal meta-analysis. Journal of Global Security Studies 8(1): ogac042.

84.

Shore

(2022) Data incoming: How to close the cyber data gap. Available at: https://warontherocks.com/2022/10/data-incoming-how-to-close-the-cyber-data-gap/ (accessed 20 October 2022).

85.

Valeriano

Maness

(2015) Cyber War Versus Cyber Realities: Cyber Conflict in the International System. Oxford: Oxford University Press.

86.

Valeriano

Jensen

Maness

(2018) Cyber Strategy: The Evolving Character of Power and Coercion. New York: Oxford University Press.

87.

Whyte

(2018) Dissecting the digital world: A review of the construction and constitution of cyber conflict research. International Studies Review 20(3): 520–532.

Until consensus: Introducing the International Cyber Expression dataset

Abstract

Keywords

Introducing the dataset

Content and utility

Suggested applications

Footnotes

Replication data

Acknowledgments

Funding

ORCID iD

Notes

References