Sage Journals: Discover world-class research

Abstract

In organizational theory, institutionalists generally make predictions of corresponding context and policy outcome based on structural processes. Psychoanalytic theory, in contrast, focuses on the rhetorical framing rather than the environment of a policy for predictive outcomes. This study aims to explore the debate over policy prediction by developing a supervised machine learning model to predict for policy success and context in the United Nations (UN). Through data collected with a python web scraper on all UN meetings in the General Assembly (GA) and Security Council (SC) between 1994 and 2020, we parse motions, policies, and conflict indicators, before passing meeting records through the Linguistic Inquiry and Word Count (LIWC) psycholinguistic algorithm. Next, we build 12 different machine learning models to predict for policy passage and context using preprocessed motion and LIWC data; results demonstrate that the psychoanalytic models better predicted for both context and policy outcomes than the institutionalist models, suggesting that the classical political axiom, “actions speak louder than words,” may not be supported by the empirical evidence.

Keywords

United Nations parliamentary procedure artificial intelligence psycholinguistic institutionalism

In 1955, US Secretary of State Dean Rusk first observed that, as contrasted to the 19th century bilateral “backroom deals,” large international institutions were rapidly coming to resemble the multilateral factionalism of domestic parliaments (Fiott, 2011, p. 1; Rosenbaum, 1967, pp. 218–9); as a consequence, Rusk noted, new formalized methods that “equalized” the decision-making power of actors in adjudicating policy disputes were changing traditional diplomacy into “an exercise in building and managing the coalitions required to secure the minimum number of votes” (Smith, 2007, pp. 3–4). Since then, theorists on UN decision-making have generally sorted into two schools: first, institutionalists argue that the processes of the UN, by governing interaction and policy drafting, act as gatekeepers for determining which policies pass and under what circumstances (Berliner & Prakash, 2014, pp. 218–20; Monteleone & Oksamytna, 2020; Smith, 2007, p. 278; Jessup, 1957). In contrast, political psychologists, rather than focusing on the available tools or interests of such actors, pivot to a constructivist viewpoint in emphasizing how rhetoric and cultural pretexts shape the tools, policies, and appraised contexts of a given situation (Bode, 2015, pp. 16–9; Kertzer & Tingley, 2018; Winther & Lindegaard, 2020, pp. 232–3233).

Regardless, the existing literature of both schools tends to analyze voting patterns (Voeten, 2013, pp. 55–8). Others such as Kaufmann (1988) and Smith (2007) focus on a qualitative review of historical cases that represent actors’ manipulation of the process to further specific policy goals.¹ Notably, both methods result in context-orientated case studies that while attractive in addressing pressing issues such as “why the UN did nothing in Rwanda” fail to empirically demonstrate systemic phenomena (Braun et al., 2018; Holsti, 2016). And finally, in contrast to the traditional “realist” or “liberal” approaches, which deem decision analysis as a “black box” that adds little definitive knowledge to international politics thereby justifying a pivot to analyzing vague state intentions, this paper aims to tangibly identify approaches within the institution that can affect real life policy making (Mearsheimer, 1994; Finel, 2001). In this paper, we build on early research by providing a parameter of inputs between 1994 and 2020, rather than a series of individual case studies, to illustrate systemic phenomena. Next, we compare our psycholinguistic model to the traditional institutionalist model to determine which better predicts for policy outcome and contexts. As a result, we make three contributions: first, we employ a new machine learning methodology to test for predictions based on large sets of variables, which allows for large sets of data to be more accurately modeled (Marsland, 2015; Montavon, et al., 2018). Second, we engineer 12 models—six for the GA and six for the SC—to compare and contrast predictions and determine the most accurate model. Third, we implement an empirical strategy focused on identifying meaningful distinctions in policy: rather than recording only votes, we account for the Kaufmann (1988) “death by committee” strategies by also recording events in which votes are not taken (Freixas & Zwicker, 2003; Rasch, 2009). As our conflict indicators further illustrate, we aim to determine when the UN is likely to tangibly intervene, rather than just issue vague statements or discords. As a result of both, we demonstrate that the psycholinguistic model is a stronger predictor than suggested by the current literature.

Literature Review

General Assembly and Security Council

Founded in part out of the failure of League of Nations and the lessons of the Nazi atrocities from the Nuremberg Trials, the United Nations (UN) was a structural compromise between two competing worldviews: a liberal conception of mutual cooperation symbolized by the equality of members under the General Assembly (GA), and a more realist consideration identified in the UN Security Council (SC), where the “P-5 nations”—the most powerful global hegemons such the United States—held veto powers (Paulson, 1975, pp. 134–7; Waltz, 1979). In brief, Table 1 examines the basic differences between each body as described by the United Nations (Luck, 2016; Peterson, 2006, pp. 90–112; United Nations, 2021).

Table 1.

Differences in the General Assembly and Security Council.

	General Assembly	Security Council
Voting Methods	Equality; one vote for each member.	Equality, except the P-5 can veto.
Size	193	15
Jurisdiction	General authority over issues of security, the UN budget, elections to committees, and recommendations.	Primarily related to matters of immediate concern to global security. Is able to issue sanctions, peacekeepers, and peacemakers.
Procedures	Largely formalized—meetings tend to be public and debate regular.	Largely informalized—meetings have multiple formats and only sometimes public.
Committee System	Standing committees that have continual function and jurisdictions.	Special committees only delegated for specific tasks.

In terms of important distinctions, much academic discourse tends to focus on the SC’s veto power; the effect of this difference, however, is empirically complicated. As Kim and Rusett (1996) notes, the actual use of the veto has changed little since the 1970s, with Thalakada (1997) similarly observing that the actual use of the veto has been reserved since the 1960s. More contemporarily, scholars have noted that the use of veto power is more implicit, being instead utilized to threaten a block of policy, forcing lengthy debates that result in a de facto “veto” (Zaccaria, 2021). Finally, even those scholars who do allege the veto power has been used extensively to the detriment of UN agree that the actual use of the veto is less apparent than its threat (Smith, 1999); and even under these cases, certain “norms” have emerged that minimize its impact for certain cases that threaten international security (Wenaweser & Alavi, 2020, pp. 65–6).

The Institutionalist Model

An institutionalist view of the UN, as summarized by Smith (2007), argues that actors use procedural and institutional means to influence policy outcomes and appraise the relative importance and consensus of a given context (p. 12; Monteleone & Oksamytna, 2020). More contemporary institutional process-based scholars tend to view institutional operations as a function of the cultural and social environments that either coerce or normatively induce mimetic change in response to changing governing ethics (Keohane, 1986); that is, how a specific international organization interacts or changes with the environment often mirrors institutional processes (Berliner & Prakash, 2014, pp. 218–20). In the United Nations, the tools of interaction are referred to as “motions” which dictate how delegates use procedures to formally prescribe an action. These motions greatly vary, from how to refer a proposal to a committee to raising procedural objections (Haftendorn et al., 2004). Moreover, each possible procedural action taken by a delegate can be sorted into a class of motion, which signifies precedence and ordering, as presented by Table 2 (Robert, 2001, 2013).²

Table 2.

Characteristics of Motion Classes.

Motion Class	Definition	Characteristics RONR (11th ed.), pp. 62–79
Main (M)	“A motion whose introduction brings business before the assembly.” RONR (11th ed.), p. 62, ll. 19–21.	1. Takes precedence of nothing. 2. Can be applied to no other motions. 3. Is out of order when another has the floor. 4. Must be seconded. 5. Is debatable. 6. Is amendable. 7. Requires a majority vote, with certain exceptions. 8. Can be reconsidered.
Subsidiary (S)	“… motions [that] assist the assembly in treating or disposing of a main motion.” RONR (11th ed.), p. 62, ll. 34–5.	1. Applied to another motion that is pending (being actively considered). 2. Applied to any main motion. 3. Such motions fit into an exact order of precedence. 4. Out of order when another has the floor. 5. In order while the main motion in question is pending.
Privileged (P)	“… motions that do not relate to the pending business, but have to do with special matters of immediate and overriding importance…” RONR (11th ed.), p. 66, ll. 29–33.	1. Does not assist in consideration of a pending motion. 2. Deals with matters of immediate importance. 3. Fit into an order of precedence. 4. Take precedence over motions of any other class.
Incidental (I)	“… motions [that] relate… to the pending business or to business otherwise at hand… but none of them possess all five of the characteristics listed… As a class, incidental motions deal with questions of procedure…” RONR (11th ed.), p. 69, ll. 7–13.	1. A question of process. 2. Not substantive. 3. Applicable only to specific circumstances.
Motions that bring a question back before the assembly (B)	“… motions that bring a question again before the assembly enable the assembly, without violating the above principles, to reopen a completed question…” RONR (11th ed.), p. 75, ll. 20–4.	1. Brings back consideration of a question that has already been considered. 2. Adheres to specific limitations of application.

Amongst these classes, as Table 3 provides an example of, there are numerous strategies that institutionalists allege can be used to influence the policy results (Demeter, 1950; Kaufmann, 1988; Smith, 2007).

Table 3.

Methods of Procedural Manipulation.

Method	Description	Citation
Move a main motion to support a popular idea despite knowing it will fail: “Black Peter”	Delegate’s operating under the “Black Peter” strategy openly support an idea, understanding that an opponent will be forced to reject it.	Kaufmann, 150.
Postpone Definitely, Indefinitely, or Table—“Waiting for Godot”	Delegates openly support the idea, but argue that the idea must be postponed, because “the time is not right.” And the time, for these delegates, is never right.	Smith, 270. Demeter, 29. Kaufmann, 150.
Adjourn the Meeting or Recess—“The Slow Veto”	Delegates attempt to endlessly recess or adjourn, in an attempt to delay a vote.	Smith, 270. Kaufmann, 149. Demeter, 29.
Refer to a Committee—“Ping Pong”	Delegates will argue that the current body is not specialized enough to handle a task, or that another body is already handling it. And when that body is referred to, the delegate then argues the same and refers to it again, seeking further “professional” advice.	Kaufmann, 150. Smith, 270. Demeter, 27.
End Debate—“Censor the Opposition”	Delegates will often attempt to preliminarily close debate, so as to prevent the other side from securing voters or changing minds.	Kaufmann, 149. Demeter, 29.
Reconsider everything—“Change the Swing Voters.”	A delegate will endlessly attempt to reconsider every action to forestall some inevitable conclusion.	Demeter, 28.
Amend, Amend, Amend—“The Filibuster”	A delegate, knowing there are time restrictions, submits an endless number of substantive proposals to restart time restrictions.	Kaufmann, 150. Smith, 268–70. Demeter, 30.

In essence, institutionalists propose that policy outcomes and context predictions are a function of the ordering of an institution, which in turn modifies the transaction cost of taking action (Wood & Bohte, 2004). Moreover, in contrast to output orientated theories of institutional policy, such as a realist analysis of voting for a specifically beneficial resolution, institutionalism observes that even in an environment of selfish actors, policies are often “debilitated” rather than “defeated” through a filter of processes that select for actual outcomes (Kaufmann, 1988, p. 143; Smith, 2007).

Finally, for institutionalists the type of deliberative body matters: indeed, while delegates in the GA have equal voting rights, the “veto power” in the SC introduces inequality among members, which in turn differentiates the cost of transacting policy in either body (Bourantonis, 2010; Smith, 2007). Institutionalists predict, as a result, that the perception of a policy outcome is dependent on the processes used to reach it (Haftendorn et al., 2004); for example, institutionalists would point to use of numerous amendments to “filibuster” and the corresponding defeat of the Razali SC reforms in the GA as evidence of institutional process affecting outcomes (Smith (2007), p. 272).³ For this reason, institutionalists would predict that not only does accuracy differ in both the GA and SC, but that ordering of the institution, and corresponding use by delegates, better predicts policy than what those actors allege to politically and normatively represent.

Psycholinguistic Models

In many democratic institutions, as Smith (2007) notes, policy construction is simply a reflection of voting coalitions. How these coalitions form, and under what criteria they change, is subject to decades of research with various psycholinguistic explanations; Chong and Druckman (2007), for instance, suggest that disputes can often simply be resolved by a contextual view of an argument’s merits. Other scholars suggest that the relative strength of arguments can be traced to broader cultural themes, cognitive biases, and heuristics built into certain rhetorical arguments, both at the domestic and international level (Bayes et al., 2020; Peterson & Arceneaux, 2020, 1129). The UN, moreover, highlights a particularly interesting case study because convincing arguments must transcend traditional regional or cultural borders to establish international representations of norms (Shannon et al, 2014). As a result, the general theses of psycholinguistic models emphasize how arguments for a policy are framed, rather than just what tools are used to further a policy (Manstead, 2001); in effect, this framing modifies how individual delegates appraise a policy or context (Almohammad, 2016, pp. 8–15; Jones, 2020, pp. 649–52). In particular, numerous empirical studies have indicated the validity of “dictionaries,” or built-in algorithms of certain psychological phrases, in transcribing verbal expressive emotions from text documents (Kahn et al., 2007; Mcdonnell et al., 2020). More specifically, as Hacker (1996) points out, linguistic analysis excels at providing evidence for the deliberate use of language to manipulate or coerce a desired outcome. In totality, the psycholinguistic model suggests that the syntax and diction that an individual uses affects the psychological state of both the subject engaged in the discourse and those listening, in turn reflecting their corresponding opinion on policy and context (Gagné, 2011; Martynyuk & Ponikaryova, 2018).

Neural Networking and Classification

To solve the problem of sorting through enormous sets of data, this study employs machine learning models to predict outcomes (Hindman, 2015, pp. 48–52). Functionally, a deep neural network (DNN) algorithm creates a network of layers. Each unit in a layer is called a “neuron.” Each multilayer network of neurons creates numerous nonlinear relationships to defined parameters, similar to the human brain (Grill-Spector et al., 2018). In doing so, as illustrated by Figure 1, the algorithm attempts to generate nonlinear functions that most efficiently select for the given criteria demanded by the output parameter (Hussain & Jeong, 2016, pp. 1–3; Sullivan 2019, pp. 2–11).

Figure 1.

Deep neural network (DNN) model.

The above model shows a simple DNN model designed to portray three concepts: First, the inputs from a defined “training” set of data relay information to the first layers of neurons. Second, the first layer of neurons computes the input data using an activation function designed to calculate a desired output, which is then transmitted to the next layer of neurons. Moreover, each input to the second layer is signaled by a weight given to each variable, allowing the neurons to slowly prioritize or “learn.” Deep neural network models then continue to pass the corresponding weighted values through each layer, slowly “learning” how to prioritize the importance of variables, similar to biological neurons. Finally, the last layer of neurons produces an output function which is then tested against a series of the data not used in the training set, which can then be translated into a number of usable metrics.

In political science specifically, DNN models have become increasingly important for interpreting large sets of data (Hindman, p. 49). Rapid advances in DNNs promise not only to automate technical functions but to augment traditional human interpretations of data (Li et al., 2017, pp. 12–4). While the proprietary nature of these algorithms often prevented their review by scholars, their impact is nevertheless self-evident by their successful implementations. In the UN specifically, as Sovrano et al. (2020) notes, artificial intelligences have already been able to classify UN resolutions according to related Sustainable Development Goals, effectively monitoring state compliance. As a result, DNNs allow for researchers to not only theorize but make tangible predictions for a massive range of factors, and are well suited for previously inaccessible comparisons of psycholinguistic models that utilize hundreds of interacting variables (Kratzwald et al., 2018, p. 24).

Methodology

The architecture of our proposed framework to predict policy passage operates in five central units, as illustrated by Figure 2. First, the web scraper downloads and mines the UN meeting documents, either through a fuzzywuzzy program for motions, policy, and context, or the Linguistic Inquiry and Word Count (LIWC) API (Hu et al., 2021; United Nations, 2021). Second, the database is stress-tested, with finalized data preprocessed before being returned to the database for analysis. Finally, the data is fed to the tensorflow neural network to compute the probability of the labels; that is, the probability of policy passage and the presence of conflict indicators.⁴

Figure 2.

Framework: An architectural view of the data pipeline.

Variables

First, all meetings are sorted by type of assembly: namely, the GA or SC. These two bodies have several distinctive procedural differences, such as the number of motions, voting rights (i.e., presence of veto), and jurisdictions.

Second, meetings were categorized by conflict prevalence (CP), as recorded by a dummy variable (no conflict = 0, conflict = 1). In this paper, CP refers to the presence of forceful intervention in any given topic as defined by Chapter VII of the UN Charter. This includes three potential situations: “peacekeeping,” “peacemaking,” and “sanctions.” These situations conform with the 2005 “Responsibility to Protect” (R2P) doctrine and UN charter (526) as outlined by Burkle (2018).

Third, the number of politics passed in any given meeting was counted. For this paper, the definition of a “policy” is a resolution passed or rejected by a given set of rules governing a main motion. To measure this, our queries in the GA searched for “The draft resolution was adopted.” In the SC, our queries searched for “The draft resolution has been adopted.”⁵

Fourth, the total number of motions per meeting is recorded. These motions are then sorted into the aforementioned five ordinal classes: main motion (M), subsidiary motion (S), privileged motion (P), incidental motion (I), and motions that bring a question before the deliberative assembly (B).⁶ These classes represent every possible procedural action that can occur in a deliberative assembly. Each of these motions holds a specific characteristic that infers an interaction with another motion (Appendixes A and B).

And finally, fifth, the LIWC API categories offered numerous psycholinguistic variables, mined directly from meeting files. As illustrated by Table 4, this includes four central summary dimensions as functions of specific diction and syntax associated with 93 different psychological cues. In effect, these variables aim to demonstrate both the perception of a topic by delegates (emotional tone and authentic) along with individual tactics, strategies, and rhetorical devices used by delegates (analytic and clout). These variables have been shown to relate to personality, rhetoric, and perception (Hirsh & Peterson, 2009).

Table 4.

Linguistic Inquiry and Word Count (LIWC) API Categories.

Dimension	Category
Summary Dimension	Analytic
Summary Dimension	Clout
Summary Dimension	Authentic
Summary Dimension	Emotional Tone

Data Collection

This study will review a population of United Nations (UN) meeting records in the GA and SC between 1994 and 2020.

First, a python web scraper took UN records from online databases and converted the files to a portable document format (PDF). The files were then converted into readable text files and sorted by date.

Second, the documents were analyzed both through a fuzzywuzzy analyzer and LIWC. For fuzzywuzzy, key phrases for policy passage and conflict indicators were run against the text files computing similarity of statements through the levenshtein distance formula at 95% confidence. Fuzzywuzzy data was then stress tested for accuracy.⁷ Thereafter, the program records each query and sums the total frequency of queries per text file. For LIWC, every document was run and results recorded. The queries for both are then placed into a csv file for preprocessing.

Analysis

Our neural network models the transformed data to accurately predict either the probability of policy passing or the presence of conflict indicators. We used the Tensorflow python environment to simulate models (Dillion et al., 2017). For both the GA and SC, the models were divided into three categories:

1. Psychology: the probability of a label given only LIWC input variables as features.

2. Motion: the probability of a label given only motion input variables as features.

3. Mixed: the probability of a label given both LIWC and motion input variables as features.

For both the GA and SC, all parameters, including density, testing and training data allocation, dropouts, and epochs were tuned for optimal accuracy and to fortify against overfitting (Hssayni, 2020). All variables were set as features, with the exception of “Policy Passed” and “Conflict Indicators,” which were set as the labels depending on the model tested. To tune, we simply used the trial and error method, as noted by Bielza et al. (2013).

In both the SC and GA, the model first splits the data into two categories: training and test (test proportion: GA = .2; SC = .15). Second, using a sequential function, hidden layers were added using the Rectified Linear Unit (ReLU) activation function $(f (x) = m a x (x, 0))$ to maximize performance while minimizing vanishing gradients (Hara et al., 2015). Third, to maximize our result for classification and to fit the data to a logistic model, we used a sigmoid activation function for the final output

S (x) = \frac{1}{1 + e^{- x}}

Fourth, we used Adam for the optimizer function, as it provided an output with high accuracy (Zhang, 2018). Fifth, we used binary crossentropy to measure loss (Silaparasetty, 2020, p. 341). Tables 5 and 6 illustrate each tuned parameter for the GA and SC, respectively.

Table 5.

Tuned Hyperparameters and Neutral Network Layers for the General Assembly.

	Psychology Model				Motion Model				Mixed Model
	Policy Passage		Conflict Indicators		Policy Passage		Conflict Indicators		Policy Passage		Conflict Indicators
Layer	Filters	Param	Filters	Param	Filters	Param	Filters	Param	Filters	Param	Filters	Param
Input	164	15,416	128	12,032	144	1008	142	992	114	11,400	134	13,400
Hidden	86	14,190	64	8256	64	9280	68	9724	28	3220	68	9180
Hidden	40	3480	32	2080	16	1040	18	1242	16	464	32	2208
Hidden	16	656	16	528
Output	1	17	1	17	1	17	1	19	1	17	1	33
Dropout	.1, .15		.2, .2		.2		.18, .10		.2, .2		.25, .2
Epochs	50		30		30		30		50		75
Total Trainable Parameters	33,759		22,913		11,345		11,979		15,101		24,821

Table 6.

Tuned Hyperparameters and Neutral Network Layers for the Security Council.

	Psychology Model				Motion Model				Mixed Model
	Policy Passage		Conflict Indicators		Policy Passage		Conflict Indicators		Policy Passage		Conflict Indicators
Layer	Filters	Param	Filters	Param	Filters	Param	Filters	Param	Filters	Param	Filters	Param
Input	164	15,416	140	13,160	124	744	224	1344	126	12,474	248	24,552
Hidden	70	11,550	64	9024	64	8000	124	27,900	64	8128	120	29,880
Hidden	32	2272	36	2340	16	1040	24	3000	32	2080	60	7260
Hidden	16	528	16	592							26	1586
Output	1	17	1	17	1	17	1	25	1	33	1	27
Dropout	.25, .1		.2, .1		.15, .2		.1, .1		.2, .15		.15, .1
Epochs	50		30		30		30		50		30
Total Trainable Parameters	29,783		25,133		9801		32,269		22,715		63,305

Finally, to simulate reproducibility in the stochastic environment of Tensorflow, we modeled the average metrics of 30 models. Due to data imbalance, label classes were weighted for.

Results

As illustrated by Tables 7 and 8, the range, mean, and standard deviation of each variable in a respective body per meeting was reported. For LIWC, we used Table 4 to summarize the variables.

Table 7.

General Assembly Descriptive Statistics (GA, N = 10,211).

	Policy Passage				Conflict Indicators
	Policy Did Pass		Policy Did Not Pass		Conflict Indicators		No Conflict Indicators
	Mean	Std.	Mean	Std.	Mean	Std.	Mean	Std.
Analytic	98.243	1.049	98.182	1.092	98.122	.950	98.248	1.228
Clout	65.652	6.054	67.075	7.015	67.728	6.883	66.158	6.970
Authentic	13.035	7.111	12.876	7.328	13.589	6.769	12.563	7.922
Tone	54.782	17.113	57.023	18.41	61.190	18.790	53.854	17.442
Class M	.077	.287	.026	.218	.037	.249	.034	.199
Class S	.046	.219	.003	.059	.010	.100	.011	.104
Class I	.341	1.114	.141	.809	.151	.704	.209	1.050
Class P	.144	.508	.037	.256	.056	.292	.061	.321
Class B	.058	.243	.016	.137	.020	.140	.023	.173

Table 8.

Security Council Descriptive Statistics (SC, N = 5087).

	Policy Passage				Conflict Indicators
	Policy Did Pass		Policy Did Not Pass		Conflict Indicators		No Conflict Indicators
	Mean	Std.	Mean	Std.	Mean	Std.	Mean	Std.
Analytic	98.491	.877	98.025	1.348	97.717	1.374	98.586	.930
Clout	71.183	3.897	73.740	5.458	73.198	4.751	72.781	5.550
Authentic	16.160	4.995	17.950	6.459	18.589	5.059	16.310	6.793
Tone	70.830	16.644	73.216	21.437	72.943	21.535	72.100	18.770
Class M	1.000	.063	.901	.335	.953	.280	.909	.291
Class S	.048	.634	.071	.480	.103	.675	.027	.336
Class I	.016	.130	.048	.257	.070	.308	.008	.095
Class P	.011	.105	.086	.288	.111	.317	.019	.154

Model Performance

We trained all models under the established hyperparameters from Tables 5 and 6. Table 9 shows predictive outputs for the GA.

Table 9.

Average Prediction Outputs for the General Assembly.

	Psychology Model				Motion Model				Mixed Model
	Policy Passage		Conflict Indicators		Policy Passage		Conflict Indicators		Policy Passage		Conflict Indicators
Metric	Mean	Std. Dev.	Mean	Std. Dev.	Mean	Std. Dev.	Mean	Std. Dev.	Mean	Std. Dev.	Mean	Std. Dev.
Accuracy	.736	.100	.652	.018	.136	.000	.455	.035	.697	.125	.682	.009
Loss	.543	.090	.609	.014	.697	.001	.693	.000	.563	.133	.586	.006
Precision	.333	.081	.602	.031	.136	.000	.399	.135	.303	.079	.637	.027
Recall	.653	.147	.772	.081	.999	.000	.900	.305	.721	.143	.746	.079
F1 Score	.400	.032	.663	.018	.235	.000	.547	.185	.393	.045	.676	.020
AUC	.803	.004	.734	.002	.522	.000	.500	.000	.803	.007	.760	.004

For both policy passage (AUC = .803) and conflict indicators (AUC = .734), the returns demonstrated the model was more accurate than randomization. In contrast, the motion model scores were, for policy passage (AUC = .522), barely better than randomization, and for conflict indicators, no better (AUC = .5). Finally, the mixed model scores displayed at the right, with policy passage (AUC = .803) and conflict indicators (.760) more accurate than randomization.

Next, results demonstrated convergence of the training and validation data, inferring generalization was applicable. Figures 3(A)–3(C) illustrate convergence of training loss and validation scores, relatively recorded for each of the 30 models over time. For each figure, the data was recorded using a smooth loss function (loss on the y-axis and time on the x-axis). Using tensorboard, we smoothed the loss function at .6. Each model was checked for convergence, demonstrating that accuracy results were optimized to control against underfitting and overfitting. Results can therefore be generalized.

Figure 3.

Figures 3(A) to 3(C) represent the General Assembly Psychology Loss Model, Motion Loss Model, and Mixed Loss Model, respectively. Likewise, 3(D) to 3(F) represent the Security Council Psychology Loss Model, Motion Loss Model, and Mixed Loss Model, respectively. Convergence represents validation and training meeting.

Next, Table 10 records outputs for the SC.

Table 10.

Average Prediction Outputs for the Security Council.

	Psychology Model				Motion Model				Mixed Model
	Policy Passage		Conflict Indicators		Policy Passage		Conflict Indicators		Policy Passage		Conflict Indicators
Metric	Mean	Std. Dev.	Mean	Std. Dev.	Mean	Std. Dev.	Mean	Std. Dev.	Mean	Std. Dev.	Mean	Std. Dev.
Accuracy	.871	.029	.795	.009	.658	.002	.487	.012	.854	.015	.803	.008
Loss	.349	.044	.447	.004	.690	.120	.693	.000	.356	.029	.417	.006
Precision	.807	.088	.765	.014	.030	.092	.417	.166	.802	.057	.788	.023
Recall	.732	.034	.836	.034	.999	.305	.867	.346	.680	.023	.820	.053
F1 Score	.752	.030	.765	.013	.046	.141	.560	.223	.720	.016	.797	.016
AUC	.908	.004	.874	.002	.510	.000	.5	.000	.892	.008	.892	.002

For both policy passage (AUC = .908) and conflict indicators (AUC = .874), the returns demonstrated the model was more effective than randomization. The motion models, however, performed significantly worse, with policy passage (AUC = .510) barely better than randomization, and conflict indicators (AUC = .5) performing no better than randomization. Finally, the mixed model scores displayed at the right, had differing results, with policy passage (AUC = .892) better than randomization but weaker than the corresponding psychology only model. In contrast, the mixed conflict model (AUC = .892) was the stronger of the series.

Next, results demonstrated convergence of the training and validation data, inferring generalization was applicable. Figures 3(D)–3(F) illustrate convergence of training loss and validation scores, relatively recorded for each of the 30 models over time. For each figure, the data was recorded using a smooth loss function (loss on the y-axis and time on the x-axis). Using tensorboard, we smoothed the loss function at .6. Each model was checked for convergence, demonstrating that accuracy results were optimized to control against underfitting and overfitting.

Discussion

In general, the results suggest that psychological indicators are stronger predictors for policy passage and context than the actual actions of the delegates to either implement or pass a policy. Moreover, while institutionalists may be inclined to connect rhetorical acts to institutional pressures that predetermine the probability of policy or context, these same pressures do not relate to the actual procedural actions of delegates (Wood & Bohte, 2004). Indeed, the actual action of delegates, ranging from committee hearings, to amendments, to procedural debates was a significantly weaker predictor than the rhetoric those delegates employed, regardless of the type of deliberative body they were based in. The specific results of the study, furthermore, aimed to identify how predictions manifested: namely, whether features better predicted for presence or absence of policy passage or certain contexts. As our results indicated, modeling in both the GA and SC overwhelmingly predicted better for a lack of policy passage or conflict indicators, suggesting that when the UN does pass policy or use a form of coercion in such policy discussions there is a weaker link to rhetoric and action. For this reason, when faced with debates on pressing crises—such as genocide—other factors, such as unannounced state interests, may project a stronger influence.

In the GA specifically, policy predictions for the psychology model were more accurate than the motion model; moreover, because even the mixed model, which analyzed both the rhetoric and actions of delegates in predicting policy passage, was not significantly more accurate than just the psychology model, rhetoric used by delegates was a stronger predictive factor. However, while accuracy was relatively high for predicting policy results, the low precision scores weighed down the overall F1 score, inferring that the model was poor at correctly predicting the presence of policy passage. Likewise, an amplified result of the same trend emerged with predicting for conflict indicators, with the psychology model producing stronger predictions than the motion model. Furthermore, similar to the policy models, the conflict indicator models had poor precision but high recall, indicating the models were better at predicting when sanctions or peacekeeping would not be utilized, rather than when they would be.

Rhetoric in the SC, while following a similar pattern to the GA, was a stronger predictor; indeed, for policy passage, the psychology model was likewise a stronger predictor than the motion model, the latter of which could not even surpass random selection. Both the psychological and mixed models, while with the latter slightly less accurate than the former, demonstrate strong predictability for policy. The SC, however, differentiates largely from the GA in terms of precision scoring across the psychology and mixed models, indicating that predicting the probability of policy passage was more accurate in the former. This supports a more transparent view of the SC: when a delegate says something, it more strongly relates to the actual passing of a given policy. Institutional differences between the SC and GA, moreover, may contribute to this difference; indeed, as the literature suggests, delegates have less reason to use “behind the scene” manipulation when the veto is always an alternative (Kaufmann, 1998). Thus, despite arguments from Hovell (2009) and Wenaweser and Alavi (2020) that procedures in the SC incentivize policies to fail, it may also paradoxically induce transparency, thereby lending to broader range of rationalizations for policy amendments and subsequent passage (Norris, 2017); in layman’s terms, delegates that are more open may be better able to compromise and adjust positions, wherein GA procedures—which value each individual vote—may also make individual delegates harder to discern, and thereby decrease transparency (Kaufmann, 1998). The size of both institutions may also reflect the transparency of rhetoric on either policy or context, given that scholars have already noted an increased pool of interests may obfuscate the actual positions of the majority (Smith, 2007, pp. 162–5); that is, because even a few dissenting delegates in the SC may prove fatal to a policy, and thus bump predictability, the same situation in the GA may be overcome by a “silent majority” (Panke, 2013, pp. 15–22, 120–5).

The trend of policy success as a function of psychological response persists across the spectrum of policy contexts. That is, for both the GA and SC, the presence of sanctions, peacekeeping, and peacekeeping could be accurately predicted by the models. Both bodies, however, had varying degrees of conflict indicators explained by the model; in the GA, precision was lower than recall, inferring that the model was relatively better at predicting the lack of a policy context. The SC mirrored a similar trend, albeit similar to actual policy success, held both high recall and precision, inferring the model, while relatively worse at predicting for a specific policy context, was a strong fit regardless. As a result, the psychology and mixed models were able to, especially in the SC, both predict for the probability of a policy successfully passing and the type of policy that would pass, given only the input of rhetoric used in the meeting. This suggests, in contrast to prior supported expectations by Kaufmann (1998) and Smith (2007), that the type of policy passed could be controlled for when making predictions about policy success.

Limitations and Future Research

The present study had several important limitations. First, while fuzzy analysis allows for more efficient processing of large data, string-matching that is too complex can result in disjunctions between actual motions and recorded motions, increasing the probability of type II error (Noh et al., 2015, pp. 4348–51). Alternatively, strings that are too short, as in some cases the query selected for class B motions, can yield false positives and increase the probability of type I error (4349). Until public libraries of fuzzy algorithms are able to interpret contextual use of certain terms, the very nature of data mining will contain error. This error, however, is mitigated by the presence of a large sample size (4348).

Second, as already noted, DNN models are extremely complicated, lending to a lack of interpretability for two reasons: first, machine functions fail to report on how it arrived at a conclusion, and second, DNN models are inherently stochastic, making reproduction difficult (Wang & Shang, 2014). While the models can accurately predict policy outcomes and context at a rate far stronger than traditional methods, it is nearly impossible to exactly know the reasoning of how it arrived at a conclusion (Hicks et al., 2018, pp. 363–4). While researchers could, hypothetically, examine functions individually to derive the model’s reasoning, doing so would require investigating hundreds of thousands of intricate relationships, many of which are codependent (Reimers et al., 2020). Second, the models, along with all machine learning models, are stochastic—that is, the model generation is non-deterministic due to random initialization of layer weights, shuffling of data when splitting, and CPU multi-threading (Chen et al., 2016; Huang et al. 2016, pp. 646–8). While some research has suggested that machine learning models can be reproduced through seeding, this equally undermines the very strength of machine learning: computation optimization (Dahl et al., 2013). We still preferred this method, however, because direct human involvement necessarily detracts from engineering efficient algorithms that can explain relationships between the over 100 variables used in our study (Yalcin, 2021). Conclusions drawn on the models, while accurate in probability of prediction, must therefore be carefully tailored to a difficulty in applying extensive theoretical implication (Li, 2017).

Third, LIWC has several limiting functions: first, because the LIWC algorithm simply parses phrases rather than learns from the use of words in context, researchers should not interpret strong variables to mean that such sentiment causes certain outcomes; indeed, words associated with “anger,” for example, can be applied to various contexts not related to actual anger (Crespo & Fernández-Lansac, 2015). Second, the LIWC algorithm cannot detect all forms of language, such as sarcasm, and thus lacks a truly encompassing view of the human language. While the literature generally supports a consensus that rhetoric and norms within the United Nations limits the aforementioned expressions, coded language, such as Baton’s (1991) qualitative observation of racial rhetoric embedded in discussions of previous colonies, will need further investigation in future studies (Wiseman, 2011).

Conclusion

Institutionalists often adopt an “actions speak louder than words” paradigm when predicting policy; but as our results suggest, such an insight is unfounded in actually predicting policy passage. The model outcomes support the academic hypothesis that the psychological effects on appraisal, rather than substantive barriers provided by structural features, best predicts for policy outcomes. This conclusion may not, however, be mutually exclusive with the institutionalist worldview; Kaufman, Smith, and Demeter’s “Black Peter” and “Waiting for Godot” strategies (Table 3), for example, seem presuppose the necessity of a psychological response in frustrating structural outcomes. In context, this often means using procedural blocks to wear down an opponent’s morale or broad consensus. As a result, the institutionalist view is still potentially useful in prescribing actions to affect a vote, even if not accurate in predicting the actual outcome of the vote. Furthermore, given the inherent difficulty of understanding how a DNN comes to conclusions, institutionalism’s alternative—to manipulate the where and when speeches will be given—may be easier for policymakers to use.

The algorithm simply represents the underlying variables that best predict for policy outcomes. In the SC and GA, the empirical evidence demonstrates both policy passage and the conflict indicators can be better predicted given the rhetoric of delegates, rather than the actual substantive action of any delegate; our mixed models, moreover, suggests that substantive action only played a minor role on predicting for context. Thus, the evidence tends to support the psycholinguistic view of policy analysis, with speech generally predicting for corresponding policy outcome.

Ultimately, further work will be needed to properly implement the program to a workable interface. This may manifest, for example, in easily accessible application programming interfaces (API) that can be fed speeches to produce predictions. Our models, furthermore, can provide policymakers a valuable insight into the strength of the opposition given the rhetoric of individual delegate speeches.

Footnotes

Acknowledgments

We extend our thanks to Dr. Sereseres, whose general guidance allowed us to pursue this study. We would also like to thank Keoni Gandall, who assisted in the programming. Finally, we extend our thanks to the University of California, Irvine, Undergraduate Research Opportunities Program (UROP), who funded this study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is funded by Undergraduate Research Opportunities Program, GF11210.

Notes

General Assembly Motion Queries.

Motion Class	Standardized Motion	UN Rule	Query
M	Adoption of the agenda	13	May I take it that the assembly wishes to take note of those items that remain open for consideration
M	Adoption of the Agenda (general order)	21	The agenda was adopted
S	Amend	22	Amend the agenda to
M	Adopt, accept, or agree to a report	64	I have been authorized to make the following statement on behalf of the assembly
M	Committee, to refer a matter that is not pending	65	Refer the matter
M	Adopt Special Rules of Order	66	Report of the Committee
I	Point of Order	71	Point of Order
I	Point of Order	73	Right of reply
S	To postpone indefinitely	74	To adjourn the debate
S	Previous Question	75	May I take it that it is the wish of the General Assembly to conclude its consideration
P	Recess	76	To suspend the meeting
P	Adjournment	76	To adjourn the meeting
S	Amendment	78	To introduce a draft amendment to the draft resolution
M	Main Motion	78	The Assembly will now take a decision on draft resolution
I	Objection to consideration of the question	79	Objection to consideration of the question
I	Withdraw or modify a motion, to grant maker permission to, after motion has been stated by the chair	80	To withdraw
B	Reconsideration	81	Reconsideration of the
M	Main Motion	96,100	Appoint a Committee
I	Division of the Assembly	87,127	A recorded vote has been requested
I	Division of a question	129	A paragraph-by-paragraph vote on the draft resolution

Security Council Motion Queries.

Motion Class	Standardized Motion	UN Rule	Query
M	Adoption of the Agenda (general order)	9	The agenda was adopted
M	Main motion or question	28	appoint a committee
M	Adopt, accept, or agree to a report	29	I have been authorized to make the following statement on behalf of the committee
M	Adopt, accept, or agree to a report	29	May I take it that the draft report as corrected is adopted by the council
I	Point of Order	30	point of order
S	Amend a pending motion	31	amendment
I	Division of a Question	32	A paragraph-by-paragraph vote on the draft resolution
S	Commit, Refer, or Recommit a pending question	33	refer the matter
S	Postpone definitely	33	postpone discussion to
S	Postpone indefinitely	33	postpone discussion indefinitely
P	Recess, to take a, if moved while business is pending	33	The meeting was suspended
P	Adjourn, ordinary case in societies	33	the meeting be adjourned
I	Withdraw or modify a motion	35	To withdraw the draft resolution
M	Adopt Special Rules of Order	37	In conformity with the usual practice, I propose, with consent of the council, to invite those representatives to participate in discussion
S	Previous Question	S/2010/507 and other interpretations.	It is my understanding that the security council is ready to vote on the draft resolution before it

Author Biographies

Kimo Gandall is a practicing Professional Registered Parliamentarian (PRP) and serves as the Vice President of the California State Association of Parliamentarians (CSAP). During his time at the University of California, Irvine, he served as Chairman of the Parliamentary Society, Parliamentarian of the Student Senate, and a board member of Model United Nations. Gandall graduated from the University of California, Irvine with a BA in Political Science. Gandall’s research primarily deals with the intersection of new technologies, especially artificial intelligence, and political institutions.

Juliana Chhouk graduated from the University of California, Irvine with a BA in Psychology, with a particular interest in decision making and sentiment analysis. During her time at UCI, Chhouk worked at the Lee Bayesian Modeling Lab, focusing on decision making in consumer behavior. In the fall of 2022, she will pursue a masters of Clinical Psychology at Pepperdine University.

Alex Wang is a UROP Fellow and the prior President of the Student Senate. He will graduate from UCI in 2023, with a BS in Computer Engineering. Wang’s primary research interest is in the application of artificial intelligence.

Logan Knight is a UROP Fellow and LSAT instructor. He will graduate June 2022 from the University of California, Irvine with a BA in Political Science. Knight intends to begin study at Harvard Law School in the Fall of 2022. Knight’s research interests include constitutional law, the Supreme Court, and artificial intelligence.

References

Almohammad

A. H.

(2016). Toward a theory of political emotion causation. SAGE Open, 6(3), 215824401666210. https://doi.org/10.1177/2158244016662106

Banton

(1991). Racial rhetoric at the United Nations. International Journal of Politics, Culture and Society, 5(1), 5–18. https://doi.org/10.1007/bf01390111

Bayes

Druckman

J. N.

Goods

Molden

D. C.

(2020). Druckman, When and how different motives can drive motivated political reasoning. Political Psychology, 41(5), 1031–1052. https://doi.org/10.1111/pops.12663

Berliner

Prakash

(2014). The United Nations global compact: an institutionalist perspective. Journal of Business Ethics, 122(2), 217–223. https://doi.org/10.1007/s10551-014-2217-5

Bielza

Fernández del Pozo

J. A.

Larrañaga

(2013). Parameter control of genetic algorithms by learning and simulation of bayesian networks - a case study for the optimal ordering of tables. Journal of Computer Science and Technology, 28(4), 720–731. https://doi.org/10.1007/s11390-013-1370-0

Bode

(2015). Individual agency and policy change at the united nations: the people of the United Nations. Routledge.

Bourantonis

(2010). The history and politics of un security council reform. Routledge Chapman & Hall.

Braun

Schindler

Wille

(2018). Rethinking agency in international relations: Performativity, performances and actor-networks. Journal of International Relations and Development, 22(4), 787–807. https://doi.org/10.1057/s41268-018-0147-z

Burkle

F. M.

(2018). United Nations charter, chapter VII, article 43: Now or never. Disaster Medicine and Public Health Preparedness, 13(4), 655–662. https://doi.org/10.1017/dmp.2018.43

10.

Chen

Zhang

Xing

(2016). Deep networks with stochastic depth for acoustic modelling. In 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Jeju, Korea (South), 13–16 December 2016, https://doi.org/10.1109/apsipa.2016.7820692

11.

Chong

Druckman

(2007). Framing Theory. Annual Review of Political Science, 10.

12.

Crespo

Fernández-Lansac

(2015). Memory and narrative of traumatic events: A literature review. Psychological Trauma: Theory, Research, Practice, and Policy, 8(2), 149–156. https://doi.org/10.1037/tra0000041.supp

13.

Dahl

G. E.

Sainath

T. N.

Hinton

G. E.

(2013). Improving deep neural networks for LVCSR using rectified linear units and dropout. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, 26–31 May 2013. https://doi.org/10.1109/icassp.2013.6639346

14.

Demeter

(1950). Demeter’s manual of parliamentary law and procedure: for the conduct of business in all deliberative assemblies (p. 4). Bostonia Press.

15.

Dillion

Langmore

Tran

Brevdo

Vasudevan

Moore

Patton

Alemi

Hoffman

Saurous

(2017, November). Tensorflow distributions. arXiv:1711.10604.

16.

Finel

B. I.

(2001). Black box or pandora’s box: state level variables and progressivity in realist research programs. Security Studies, 11(2), 187–227. https://doi.org/10.1080/714005331

17.

Fiott

(2011). On the Value of Parliamentary Diplomacy. Madariaga Paper, 4(7).

18.

Freixas

Zwicker

W. S.

(2003). Weighted voting, abstention, and multiple levels of approval. Social Choice and Welfare, 21(3), 399–431. https://doi.org/10.1007/s00355-003-0212-3

19.

Gagné

C. L.

(2011). Psycholinguistic perspectives. Oxford Handbooks Online. https://doi.org/10.1093/oxfordhb/9780199695720.013.0013

20.

Grill-Spector

Weiner

K. S.

Gomez

Stigliani

Natu

V. S.

(2018). The functional neuroanatomy of face perception: From brain measurements to deep neural networks. Interface Focus, 8(4), 20180013. https://doi.org/10.1098/rsfs.2018.0013

21.

Grovogui

S. N’Z.

(2006). Beyond eurocentrism and anarchy memories of international order and institutions. Palgrave Macmillan.

22.

Hacker

(1996). Political linguistic discourse analysis. State University of New York.

23.

Haftendorn

Keohane

Wallander

(Eds.), (2004). Imperfect Unions: Security Institutions over Time and Space. Oxford University Press.

24.

Hara

Saito

Shouno

(2015). Analysis of function of rectified linear unit used in deep learning. In 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland, 12–17 July 2015. https://doi.org/10.1109/ijcnn.2015.7280578

25.

Hicks

Riegler

Pogorelov

Anonsen

K. V.

de Lange

Johansen

Jeppsson

Ranheim Randel

Losada Eskeland

Halvorsen

(2018). Dissecting deep neural networks for better medical image classification and classification understanding. In IEEE 31st International Symposium on Computer-Based Medical Systems (CBMS), Karlstad, Sweden, 18–21 June 2018.

26.

Hindman

(2015, May). Building better models: Prediction, replication, and machine learning in the social sciences. ANNALS, AAPSS 659.

27.

Hirsh

J. B.

Peterson

J. B.

(2009). Personality and language use in self-narratives. Journal of Research in Personality, 43(3), 524–527. https://doi.org/10.1016/j.jrp.2009.01.006

28.

Holsti

(2016). The problem of change in international relations theory. In International relations theory, foreign policy analysis, history of international order, and security studies. Springer. https://doi.org/10.1007/978-3-319-26624-4_5

29.

Hovell

(2009). The Deliberative Deficit: Transparency, Access to Information, and UN Sanctions. Sanctions, Accountability and Governance in a Globalised World, 92–122. https://doi.org/10.1017/cbo9780511776847.007

30.

Hssayni

E. H.

(2020). Regularization of deep neural networks with average pooling dropout. Journal of Advanced Research in Dynamical and Control Systems, 12(SP4), 1720–1726. https://doi.org/10.5373/jardcs/v12sp4/20201654

31.

Sun

Rowlands

Beck

Posma

. (2021, January). Uto-CORPus: automated and consistent Outputs from research publications. BioRxiv.

32.

Huang

Sun

Liu

Sedra

Weinberger

(2016). Deep networks with stochastic depth. European Conference on Computer Vision.

33.

Hussain

Jeong

(2016). Efficient deep neural network for digital image compression employing rectified linear neurons. Journal of Sensors.

34.

Jessup

P. C.

(1957). International parliamentary law. American Journal of International Law, 51(2), 396–402. https://doi.org/10.2307/2195716

35.

Jones

(2020). Towards an emotional geography of diplomacy: Insights from the United Nations security council. Transactions of the Institute of British Geographers, 45(3), 649–663. https://doi.org/10.1111/tran.12371

36.

Kahn

J. H.

Tobin

R. M.

Massey

A. E.

Anderson

J. A.

(2007). Measuring emotional expression with the linguistic inquiry and word count. The American Journal of Psychology, 120(2), 263. https://doi.org/10.2307/20445398

37.

Kaufmann

(1988). Conference Diplomacy: an Introductory Analysis. Nijhoff.

38.

Keohane

R. O

(1986). Neorealism and its critics. Columbia University Press.

39.

Kertzer

J. D.

Tingley

(2018). Political psychology in international relations: beyond the paradigms. Annual Review of Political Science, 21(1), 319–339. https://doi.org/10.1146/annurev-polisci-041916-020042

40.

Kim

SY.

Rusett

(1997) (In press). The New Politics of Voting Alignments in the General Assembly. In The Once and Future Security Council. New York: St. Martin’s Press.

41.

Kratzwald

Ilić

Kraus

Feuerriegel

Prendinger

(2018). Deep learning for affective computing: text-based emotion recognition in decision support. Decision Support Systems, 115, 24–35. https://doi.org/10.1016/j.dss.2018.09.002

42.

, et al. (2017). Understanding Error Propagation in Deep Learning Neural Network (DNN) Accelerators and Applications. Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. https://doi.org/10.1145/3126908.3126964

43.

Luck

(2016). UN security council: Practice and promise. Routledge.

44.

Manstead

A. S. R.

(2001). Attitudes and behavior. In International encyclopedia of the social & behavioral sciences (pp. 909–913). Elsevier. https://doi.org/10.1016/b0-08-043076-7/01759-9

45.

Marsland

(2015). Machine learning: An algorithmic perspective. Chapman and Hall/CRC.

46.

Martynyuk

Ponikaryova

(2018). Syntactic rhythm of political speeches as a psycholinguistic tool of suggestive influence. Psycholinguistics, 24(2), 181–197. https://doi.org/10.31470/2309-1797-2018-24-2-181-197

47.

Mcdonnell

Owen

J. E.

Bantum

E. O. C.

(2020). Identification of emotional expression with cancer survivors: Validation of linguistic inquiry and word count (Preprint). https://doi.org/10.2196/preprints.18246

48.

McKelvey

R.D.

Ordeshook

P.C.

(1984). An Experimental Study of the Effects of Procedural Rules on Committee Behavior. The Journal of Politics, 46(1), 182–205. https://doi.org/10.2307/2130439

49.

Mearsheimer

J. J.

(1994). The false promise of international institutions. International Security, 19(3), 5. https://doi.org/10.2307/2539078

50.

Montavon

Samek

Müller

K.-R.

(2018). February). Methods for interpreting and understanding deep neural networks. Digital Signal Processing, 73, 1–15.

51.

Monteleone

Oksamytna

(2020). Liberal Institutionalism. United Nations peace operations and International Relations theory. https://doi.org/10.7765/9781526148889.00008

52.

Noh

Lee

(2015). Keyword Selection and Processing Strategy for Applying Text Mining to Patent Analysis. Expert Systems with Applications, 42(9), 4348–4360. https://doi.org/10.1016/j.eswa.2015.01.050

53.

Norris

(2017). Electoral transparency, accountability, and integrity. Oxford Scholarship Online. https://doi.org/10.1093/acprof:oso/9780190677800.003.0011

54.

Panke

(2013). Unequal actors in equalising institutions: Negotiations in the united nations general assembly. Palgrave Macmillan.

55.

Paulson

(1975). Classical Legal Positivism at Nuremberg. Philosophy & Public Affairs, 4(2).

56.

Petersen

M. B.

Arceneaux

(2020). An intuitionist theory of argument strength in politics: How intuitive cognitive biases produce universally strong arguments. Political Psychology, 41(6), 1113–1131. https://doi.org/10.1111/pops.12668

57.

Peterson

M. J.

(2006). The UN General Assembly. Routledge.

58.

Rasch

M. B.

(2009). Coherence Of EU member states’ decision-making-voting behaviour in the UN general assembly: Introduction and methodological considerations (pp. 203–217). The European Union at the United Nations. https://doi.org/10.1163/ej.9789004167148.i-360.29

59.

Reimers

Runge

Denzler

(2020). Determining the relevance of features for deep neural networks. Computer Vision - ECCV 2020, 2020, 330–346. https://doi.org/10.1007/978-3-030-58574-7_20

60.

Robert

H. M.

(2001). Parliamentary law. Irvington Publishers, Ardent Media.

61.

Robert

H. M.

III (2013). Robert’s rules of order (11th ed.). Da Capo Press.

62.

Rosenbaum

(1967). Cyprus and the United Nations: An appreciation of parliamentary diplomacy. The Canadian Journal of Economics and Political Science, 33(2), 218. https://doi.org/10.2307/139773

63.

Shannon

V. P.

Kowert

P. A.

Anstee

(2014). Psychology and constructivism in international relations an ideational alliance. University of Michigan Press.

64.

Silaparasetty

(2020). Deep learning projects using tensorflow 2: Neural network development with Python and Keras. Apress.

65.

Smith

C. B.

(2007). Politics and process at the United Nations: The global dance. Viva Books Private.

66.

Smith

C. B.

(1999). The politics of global consensus building: A comparative analysis. Global Governance: A Review of Multilateralism and International Organizations, 5(2), 173–201. https://doi.org/10.1163/19426720-00502003

67.

Sovrano

Palmirani

Vitali

(2020). Deep learning based multi-label text classification of UNGA resolutions. In Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance, Online, 23–25 September 2020. https://doi.org/10.1145/3428502.3428604

68.

Sullivan

(2019). Understanding from machine learning models. The British Journal for the Philosophy of Science, 73(1). https://doi.org/10.1093/bjps/axz035

69.

Thalakada

(1997) (In press). China’s Voting Pattern in the Security Council, 1990-1995. In The Once and Future Security Council. St. Martin’s Press: New York.

70.

United Nations . (2021b). United Nations, Main Body, Main Organs, General Assembly. United Nations. https://www.un.org/en/ga/about/background.shtml (Accessed May 18, 2021).

71.

United Nations . (2021a). United Nations, Main Body, Main Organs, General Assembly. United Nations. https://www.un.org/en/ga/documents/pvsr.asp (Accessed January 14, 2021).

72.

United Nations . (2021c). United Nations, Security Council. United Nations. https://research.un.org/en/docs/sc/quick/meetings/2021 (Accessed January 14, 2021).

73.

Voeten

(2013) (In press). Data and Analyses of Voting in the United Nations. In Routledge Handbook of International Organization. Routledge.

74.

Waltz

(1979). Theory of international politics.

75.

Wang

Shang

(2014). A new active labeling method for deep learning. In 2014 International Joint Conference on Neural Networks (IJCNN). Beijing, China. 6–11 July 2014. https://doi.org/10.1109/ijcnn.2014.6889457

76.

Wenaweser

Alavi

(2020). Innovating to restrain the use of the Veto in the United Nations security council. Case Western Reserve Journal of International Law 52(1): 65–72.

77.

Winther

B. Z.

Lindegaard

L. B.

(2020). In the name of democracy: UNSC reform at the intersection of discourse and covernmentality. Discourse & Society, 32(2), 231–253. https://doi.org/10.1177/0957926520970382

78.

Wiseman

(2011). Norms and diplomacy: The diplomatic underpinnings of multilateralism. In The new dynamics of multilateralism. Routledge.

79.

Wood

B. D.

Bohte

(2004). Political transaction costs and the politics of administrative design. The Journal of Politics, 66(1), 176–202. https://doi.org/10.1046/j.1468-2508.2004.00147.x

80.

Yalcin

O. G.

(2021). Deep Learning and Neural Networks Overview. In Applied Neural Networks with tensorflow 2: API oriented deep learning with python.

81.

Zaccaria

(2021). The United Nations at 75: An institutional appraisal. Italian Journal of International Affairs, 56(1).

82.