Abstract
Although information and communication technologies (ICT) have created hope for a shared pluralistic world, democratic principles are far from being respected in the public digital environment, and require a detailed knowledge of the laws by which they are governed. Von Foerster's conjecture is one of the early theoretical results that could help to understand these laws. Although neglected for a long time, the advent of the overlying layer of recommendation and ranking systems which is progressively occupying the web has given empirical evidences of this conjecture, which predicts the consequences of increasing inter-individual influences on social dynamics and the susceptibility of these latter to manipulation. With both von Foerster's conjecture and the Snowden revelations in the background, we analyse the impact of ICT on human societies and their governance, in view of the fact that they have a massive impact on the way in which people influence each other in their tastes and actions.
Keywords
In 1976, at Cuernavaca, Heinz von Foerster, founder of the second-order cybernetics and precursor in the field of complex systems, intervened during a seminar given by Ivan Illich, a thinker in political ecology. According to the analysis made by the latter of his notion of counter-productivity – auto-deregulation and auto-disorganisation of a system, which becomes foreign to the elements from which it is made up
1
– von Foerster made a visionary conjecture: What you are trying to describe is the relationship of circular causality between the whole (a human community for example) and its parts (the individuals from which it is comprised). On the one hand, individuals are related to each other, and on the other hand they are related to the whole. The bonds between individuals can be more or less “rigid” – the technical term I use is “trivial”. The more trivial they are, by definition the less the behaviour of one of them provides information to the observer who already knows the behaviour of the others. I will make the following conjecture: the more trivial the inter-individual relationships, the more the behaviour of the whole will appear to the individual elements from which it is made up as having its own dynamics which escape their control. I am aware that this conjecture is paradoxical, however it is important to understand that it has a meaning only because, here, we adopt the point of view internal to the system, of the elements concerning the whole. For an observer outside the system, it is obvious that, on the contrary, the triviality of the relationships between these parts promotes conceptual control, in the form of a model. When the individuals are related trivially (as the consequence of mimetic behaviour for example) the dynamics of the system are predictable, but the individuals feel powerless to steer or redirect its course, even though the behaviour of the whole continues to be simply the result of individual reactions to the predictions of this same behaviour. The whole appears to become autonomous with respect to its conditions of emergence, and its development to be immobilised as its destiny.
Three years after 1984, which had not delivered on its literary promises, few individuals were inclined to accept that a mathematical theorem could account even to a small degree for social phenomena. Still today, some affirm that such generalities in the social domain cannot be founded, first of all because the notion of experimentation at the scale of a society is itself problematic. We will however show that information and communication technologies (ICT), which have become ubiquitous in our societies, provide us with an example of what von Foerster called “rigid relationships”, a terrain for experimentation and an empirical validation of von Foerster's intuition. The implications of this conjecture are numerous, and in particular make it possible to shed original light on the recent revelations of Edward Snowden, which provide an indisputable demonstration of the fact that access to an “external” view of our digital societies has become a strategic issue for many actors.
Pólya's urn and the illusory destiny
Von Foerster's conjecture would no doubt warrant being contextualised by a detailed account of the fundamental concepts of second-order cybernetics and “complex system” approaches: self-organisation, self-reference, positive/negative feedback, etc. For reasons of brevity, we will however restrict this analysis by providing a small number of examples, necessary for its comprehension, and referring the reader wishing to acquire an exhaustive understanding of the founding document (Dumouchel and Dupuy, 1983), as well as some more recent publications (Bourgine et al., 2008, Smith et al., 2009).
We begin with a deliberately schematic thought experiment. Let us imagine that two social networks A and B – such as Facebook and Google+ – appear the same day on the market, with identical functionalities and one single user each. Let us also imagine that each platform acquires new users using the following approach: on a regular basis, one user from one of these two platforms decides to invite a person who is not yet familiar with social networks to come and register in order to become the user's “friend”, and that these persons always accept such an invitation. If at any moment, any user, whatever platform he/she uses, has the same probability of making an invitation, what will happen? Following the first invitation, for example from A, there will be two users for A and just one for B. Thereafter, there will therefore be a two times greater chance that an invitation is made by a user from A than from B. And after that?
As shown in Figure 1, the computer simulation of this process shows that the market share of platform A varies initially in a relatively unpredictable manner, before stabilising in the vicinity of 80%. This convergence towards a stable market share appears to be inescapable (since the fluctuations become progressively smaller), despite the random nature of the initiatives made by the users', who can belong to one platform or the other. An analyst would therefore be tempted to make a retrospective interpretation of the difference in popularity between the two social networks as reflecting an intrinsic value of the two platforms, since after a certain length of time, nothing seems to change the balance of forces, despite the constant flow of new users. This analyst would however be mistaken, since the platforms are strictly equivalent.
Simulation of the variation in market share of platform A, after 1000 enrolments.
The process we have just described is referred to in mathematics as the Pólya Urn model (Johnson and Kotz, 1977). Its dynamics can be described very accurately. Using the terms of the preceding description, it is shown that the final distribution of market share held by the two platforms is entirely contingent: another experiment would converge to a different value. Furthermore, all values of market share for platform A lying between 0% and 100% have the same probability of being observed.
This phenomenon is a key notion in the study of dynamic systems, called path dependency: although the law of variation of the system does not change over time, the set of possible states of the system – the distribution of market shares in our case – contracts over time. During an experiment, from the point of view of the users (inside the system), it is thus possible to be under the illusion that an objective logic guides its evolution towards a given distribution of market shares. However, an external observer, aware of the enrolment process, would only see contingency, stabilized by positive feedback. 2 In more technical terms, the asymptotic proportion of market shares is called an attractor, and systems conforming with the Pólya Urn model have the particularity of having an infinite number of equi-probable attractors.
This abstract experiment, although very simple, illustrates important concepts of second-order cybernetics:
Dynamic systems may have a large number of stable attractors, especially in the presence of positive feedback-inducing path dependencies, The selection of one of these attractors during the history of the system can be totally contingent, and bootstrapped by random fluctuations, The interpretation of the attractor selection process is observer-dependent. In particular, an observer inside the system, e.g. one of the platforms' users, will tend to interpret it's dynamics as the emergence of a self-organized order (stable distribution of market shares), whereas an observer outside the system, observing it's global state without interfering with its dynamics, will see the contingent selection of one of the possible attractors.
The era of recommendations
The preceding thought experiment makes it possible to apprehend an important notion of von Foerster's conjecture, that of “rigid” or “trivial” relationships: the action of each agent (enrolling as a new user) is clearly determined by the choice of one of his/her acquaintances. We could state that this is the most trivial of recommendation systems. However, there is an extremely recent societal phenomenon, which could not have been imagined at the time of von Foerster's conjecture that is a powerful “triviality” factor in human relationships: the overlying layer of recommendation and ranking systems which is progressively occupying the web. Their finality is precisely to suggest actions or ideas to an Internet user, in accordance with the history of his/her actions (from which his/her preferences are inferred), the actions of other users, and more recently with the history of the actions of the members of his/her social network.
We note that in the space of 15 years, Google, which captures 65% of information searches in the world and has become one of the world's major companies, established its notoriety with the Page Rank Algorithm, which hypothesises that the best way to find information on the web is to use the links left by Internet users when recommending sites. However, the mechanisms of recommendation have led to the success of other Internet giants such as Amazon or Netflix, which propose products to their clients which they will probably like, according to their buying behaviour, and more recently the buying behaviour of their friends. The performance of these recommendation systems and their ability to predict an Internet user's behaviour have long been widely demonstrated, and the use of online social networks has become the panacea of these practices. Nowadays, neither an e-commerce site, nor a social network platform would go without a system for recommending new friends, new products or a collaborative filtering system. This is such a strategic component for these web actors that a company such as Netflix, which offers the downloading of films to approximately 25 million subscribers, did not hesitate to offer $1,000,000 to any person able to introduce a 10% improvement to the predictive performance of its recommendation system. To give the reader an idea of the “rigidity” introduced by such systems, it is important to be aware that 70% of Internet users trust the online recommendations made by people they don't know, and that this level of trust rises to 90% when they know the person. 3 With a site such as Netflix, 60% of purchases arise following the publication of a recommendation.
It is important to note that in the case of Netflix, this has nothing to do with a recommendation for conventional consumer products, for which we could imagine that this simply provides an update of objective information, but is relevant to cultural products, the evaluation of which is considered to be relatively subjective. One could then query the impact of recommendation mechanisms or social dynamics ranking in fields as diverse as the choice of cultural property (literature, music, cinema), the choice of travel destinations, friendship networks, and social diversity via social networks and dating (or even sex-dating) sites, or opinion dynamics (in particular in politics, where the influence of search engine manipulation has already been documented (Epstein and Robertson, 2015), and one can observe the appearance of platforms using the principle of online social networks to organise public debate). In a world submerged by enormous quantities of data, recommendation and ranking systems have become indispensable means of support, whose cultural impact will perhaps one day become as significant as state initiatives in the field of education.
During a recent Technology, Entertainment, Design (TED) conference, 4 Kelvin Slavin defended the idea that new actors have appeared in the co-evolution between Humanity and Nature: algorithms. He convincingly shows that the choices made by algorithms in various spheres of human activities have measurable and significant societal and environmental effects. With von Foerster's conjecture in the background, we can indeed wonder about the impact on human societies and their governance, of recommendation and ranking algorithms, and more generally ICT, in view of the fact that these have a massive impact on the way in which people influence each other in their tastes and actions.
In the academic sphere, the sciences of management and marketing were, understandably, the first to invest in the field of recommendation. One of the questions initially posed by these knowledge communities was to ask whether the recommendation mechanisms found on the web promote the diversification of consumption or, on the contrary, the emergence of star products and blockbusters. Empirical studies have provided various answers to this question (Brynjolfsson et al., 2011; Fleder and Hosanagar, 2009; Zhou et al., 2010) and it is not surprising to learn that formal approaches, based on the Pólya Urn model, have succeeded in showing that both outcomes are possible, depending on the parameter settings of the algorithms implemented (Fleder and Hosanagar, 2009). The choice of the type of algorithm, in fine of the diversification of the users' choice and indirectly of their tastes (including the so-called “cultural” type of good), is then subjected to the company's commercial strategy. An example is given by this recommendation taken from a management publication, following an analysis of various lessons learned: “If people buy your products individually, choose a conservative recommendation mechanism (which tends to reduce diversity); if they like you enough to pay for a monthly subscription, they will probably be open to recommendation mechanisms, which will provide them with pleasant surprises” (Davenport and Harris, 2009). Will the extent of the general knowledge of the population in a given country one day depend as much on the commercial strategy of major retailers as on the choice of its educational institutes?
Empirical validation of von Foerster's conjecture
Some publications with a broader scope look beyond the field of management science, and provide a highly interesting perspective of the influence of inter-individual connections on social dynamics. A paper published in Science (Salganik et al., 2006) thus designed one of the first large-scale experiments to study the effect of social influence on cultural dynamics. Their starting point is the observation that blockbusters in the realm of cinema, literature or music often inspire analysts with the idea that they are qualitatively different from other productions, despite the fact that experts regularly fail to predict their success.
To better understand this paradox, the authors created an artificial music market, with which 14 000 participants enrolled. From a platform prepared for the experiment, these participants were able to download music from previously unknown groups. Each participant had the possibility of choosing a music title, listening to it, rating it on a scale from 1 to 5, and then eventually downloading it. The participants were assigned to groups, in accordance with different experimental conditions characterised by the presence or absence of information on the number of downloads made by the other users, and depending on the more or less obvious nature of this information (the titles were ranked according to the number of downloads, or listed randomly). The authors thus determined three different degrees of social influence: nil, weak and strong. Finally, each experimental condition resulted in several experiments with independent user cohorts, which allowed the variability of changes related to the different protocols to be tested.
Although the authors of this study were probably unaware of von Foerster's conjecture, their conclusions
5
are remarkably similar: (…) our findings suggest that social influence exerts an important but counter-intuitive effect on cultural market formation (…) On the one hand, the more information participants have regarding the decisions of others, the greater agreement they will seem to display regarding their musical preferences; thus the characteristics of success will seem predictable in retrospect. On the other hand, looking across different realizations of the same process, we see that as social influence increases, which particular products turn out to be regarded as good or bad becomes increasingly unpredictable, whether unpredictability is measured directly or in terms of quality.
However, this provides insight into only one aspect, i.e. that of the system's internal perspective. The picture was completed few years later by a paper adopting the external perspective (Ormerod and Glass, 2010). Using the data produced during the experiments of Salganik et al. (2006) (all of the digital trails left by the participants in these experiments), the authors adopted the external point of view of the data scientist, who seeks to predict the system's evolution, on the basis of an exhaustive scrutiny of its successive states. The participant and his/her restricted vision of the system are thus replaced by the omniscient agent, who at all times has access to the complete history of the system's states. Once again, the authors' conclusions are strikingly similar to von Foerster's intuitions. The authors were thus able to show that in a situation of strong social influence, once the experiment had been initiated, in the light of a small portion of the downloading history, it is possible to predict its outcome. Although they find that “social influence decreases the ex ante predictability of the ensuing social dynamics” they also show that “these same social forces can increase the extent to which the outcome of a social process can be predicted very early in the process”, which corresponds exactly to the second part of von Foerster's contribution.
The conclusions drawn by comparing these two empirical studies represent, more than 30 years after the conjecture was made, an initial, experimentally confirmed, prediction of von Foerster's theorem, highlighting the consequences of the growth in interpersonal influences in human relationships – or in the words of von Foerster, “rigidity”. From the perspective of the actors, inside the system, social dynamics appear to be independent in comparison with the agents who create them (emergence of obvious collective behaviours, ex ante unpredictability of outcomes); whereas from the external point of view, the social dynamics appear, on the contrary, to be more straightforward to predict.
Insights from second-order cybernetics
It is not insignificant that the results published by Salganik et al. insist on the role of social influence in the process of establishing preferences. It should be mentioned that this interpretation is contrary to the currently adopted hypotheses found in most studies of neoclassical economics. Is the quality of a musical composition one of its properties, a property of its mode of dissemination, or a social construct? Although their study did not intend to provide an explicit answer to this question, it nevertheless suggests that the distribution of preferences expressed for this or that title is not just the expression of an objective value. This distribution depends on the configuration of the interactions, which in this experiment were mediated by an online platform.
Second-order cybernetics allows us to think of this phenomenon through a constructivist conception of cognition. Today, the most common definition, arising from cognitivism, envisages cognition as a manipulation of representations, dealing with the objects in our environment. Von Foerster, on the contrary, considered that there is no objective environment outside cognition. He defines cognition as the emergence of neuronal activities specific to the observer, called eigenbehaviours, 6 resulting from his/her interaction with the environment (Von Foerster, 1981). These “nervous activities are internally perceived as thought and will, or are externally perceivable as speech and movement” (Varela, 1981). Consequently, objects are not defined as entities with objective properties; but rather, for us as observers, an object is anything that “presents tokens for eigenbehaviours which we can establish” (Smith et al., 2009).
Von Foerster's conjecture is thus closely related to the more general and fundamental question of the origin of the values that we attribute to the various objects and situations we encounter. Two opposite concepts dominate social sciences, depending on whether these values are thought of as the objects' intrinsic properties, or as the reflection of social norms that transcend individuals. The motor of social dynamics is then sought, either within the individuals, at the level of motivations and preferences pre-dating their social participation, or on the contrary within institutions and social events constraining or even determining the behaviour of individuals. This is the classical opposition between methodological individualism, at the centre of most approaches in economics, and holism, a widely adopted point of view in sociology.
An intermediate approach is however defended in various branches of social science: Tardian sociology (Tarde, 1890), Girardian anthropology (Girard, 1977), the economy of conventions (Orléans, 2011), or the economy of institutions (Dolfsma, 2004), to mention just a few. All of these share the affirmation that the agents' characteristics (depending on discipline: values, motivations, desires or preferences, etc.) are co-constructed during their social interactions, with this co-construction possibly being influenced by individual predispositions or pre-existing social structures.
The conception of cognition advocated by second-order cybernetics lies within this intermediate category, with the concept called complex methodological individualism (Dumouchel and Dupuy, 1983): societies are complex systems within which the configurations and quality of interactions reveal, as much as they constrain, the eigenbehaviours of the cognitive beings from which they are composed. According to their form of expression and the characteristic time scales of the objects studied by social sciences, these eigenbehaviours correspond to behaviours, values, preferences, agents' motivations, etc.
This leads to the importance of the notion of the rigidity and degree of triviality of relationships: when the quality and quantity of inter-individual interactions are modified, the very essence of the individuals who make up the social fabric is likely to be modified. Von Foerster furthermore defined the concept of trivialization, several years previously (von Foerster, 1972), as an intervention on a system, designed to establish a “one-to-one relationship between its input (stimulus, cause) and its output (response, effect)”, in order to make it predictable. This was associated with the following warning, which takes on its full meaning in the present context: “while our preoccupation with the trivialization of our environment may be in one domain useful and constructive, in another domain it is useless and destructive. Trivialization is a dangerous panacea when man applies it to himself.”
By affirming that the “eigenbehaviours” expressed by the members of a society are more or less differentiated (distributions that are either dispersed or restricted to a small number of values), depending on whether the relationships between individuals are more or less trivial, von Foerster's conjecture is above all a prediction dealing with the relationship between the dynamics of socio-cultural values and the nature of social relationships. In von Foerster's epistemology, the trivialization of relationships between humans is equivalent to the trivialization of human beings themselves.
The epistemology of second-order cybernetics thus unavoidably changes the nature of the questions raised by ICTs. Taking the example given by Salganik et al., as often seen in this type of study, the question is no longer “what type of infrastructure will allow users to discover the best musical compositions, in view of their preferences?”, but rather “what influence do the various types of infrastructure have on their preferences, and in what manner will new preferences be satisfied?”. Here, it is not a matter of technical structures providing support for the determination of preferences, but of recognizing, on the one hand the changeable nature of preferences, and on the other hand the overlapping of individual wishes and social interaction configurations (mediatized or not, through the use of technology) in the formation of preferences.
The latter question is of acute interest for all of the technical infrastructures governing our relationships with peers and society. Recommendation systems, and in particular those implemented within social networks, are of particular interest, since they expand the possibilities for the trivialization of human relationships, by offering judgements and actions for which the ease of integration or implementation are proportional to their standardisation (ratings, likes, etc.). These effects become even more significant when they introduce new positive feedback from the whole towards the constituent parts.
Unavoidably, as a consequence of their ubiquity, ICTs lead to social changes that affect the nature of social dynamics itself.
Towards predictive social sciences
From “flash mobs” to the Arab revolutions of 2011, the advent of ICT in our daily lives has already profoundly modified our societies, in the sense of von Foerster's intuition. It has reinforced the interdependence between individuals, well beyond traditional spheres of influence, thus enabling the emergence of new types of self-organised collective behaviour. In parallel, the existence of a point of view “external” to human interactions, which was unimaginable during the pre-Internet era of von Foerster, continues to be materialised as an ever increasing number of human activities leaves digital tracks (e.g. stock exchange transactions, e-commerce, online publication, etc.) and the users of new services provided by ICT relinquish entire spheres of private life. Whether it takes place in scientific, economic, political, cultural or industrial fields, the large-scale collection of societal data is the key to major stakes, and is becoming one of the most flourishing sectors of the digital economy.
In the digital era, can be massively analysed: text (e.g. emails, blogs, press, sms, etc.), audio data (thanks to the progress achieved in voice recognition and Internet telephony), centres of interest (e.g. analysis of search engine usage, web navigation profiles, user profiles), networks for family, friends, dating and professional activities (via the analysis of address books and all types of social network), preferred living spaces and travel destinations (e.g. via GPS or cell phone data), and images (through the use of face recognition and meta-data it is possible to know who is with who, when and where). This list is far from being exhaustive, and this type of data is ever less frequently considered by its owners to be personal. The rights to access personal data, required by the simple smartphone ‘app' of a public transport provider (i.e. RATP Paris, version 2014), are food for thought on the erosion of the notion of “private life”. In order to take advantage of a service as simple as the suggestion of subway itineraries, the user must authorise access to the status and identifier of the telephone, to its GPS position, to its address book, to data storage (reading and writing), to all networks, and must authorise the creation or suppression of user accounts, control of the telephone's vibrator and sleep-mode system, as well as its synchronisation parameters.
In this context, it is not surprising that the question of behavioural and societal prediction has become one of the new Eldorados of the digital economy. Whereas IBM sells crime prediction systems to American municipal police departments, Google has acquired the social prediction start-up Behavio. Its services, such as Google Now, aim to predict your behaviour in order to improve your information and awareness. On the other hand, in 2014 Amazon filed a patent for predictive delivery (delivery before you have even ordered). Whilst academic or private research is starting to imagine how a “social climate prediction” tool could work, we are witnessing an effervescence of studies dealing with the exploitation of digital trails for predictive applications. Cultural markets are of course addressed (Mishne and Glance, 2006), with the question of blockbuster prediction, this time in real life, being moreover well positioned as a demonstrator for the on-request predictive service of Google. However, one can also consider the prediction of the stock exchange index (Bollen et al., 2011), election outcomes (Tumasjan et al., 2010), crimes 7 (Vlahos, 2012), or events such as changes in political regime (Leetaru, 2011).
Although the search for an “external” point of view of our societies now represents a strategic incentive for most major Internet companies, the recent revelations of the ex-NSA agent Edward Snowden add a political dimension to this (Greenwald, 2014). The new posture of the NSA: “Partner it All, Sniff it All, Know it All, Collect it All, Process it All, Exploit it All”, expressed in one of the top-secret documents he divulged, leaves little doubt as to the agency's ambitions. By spying on the main worldwide information highways, whilst negotiating with large ICT companies for exhaustive access to their users' data (Microsoft, Google, Apple, Facebook, Yahoo, Skype, etc.), the NSA has created a hitherto unimaginable database on the state of human activities, which is updated almost in real time. This database, together with the methods that will be developed to harness its information, perhaps embody the ultimate horizon of what an “external” viewpoint of our societies could be. The extent of the NSA's spying activities created a shock, since it indiscriminately reduced to ashes any notion of private life for both American residents and the rest of the world. However, the most perturbing fact is perhaps that the means used by the NSA appear to be disproportionate, or even inadequate, in view of their justification. The latter states that it requires all of these data in order to prevent a second September 11. In order to find a needle in a haystack, access to the entire haystack would be required. However, this affirmation has been seriously questioned, in particular in a 2013 8 report produced for the White House by a panel of experts, entitled “Report of the Review Group on Intelligence and Communications Technologies”. This report affirms that the NSA counterterrorism programme for the collection and analysis of digital trails produced on a global scale “was not essential to preventing attacks” and that much of the evidence it did turn up “could readily have been obtained in a timely manner using conventional (court) orders.”
If it is not already the case, for reasons of frustration or expediency, weak signal detection, early warnings of social change and social prediction could undoubtedly become one of the NSA's principal objectives in this massive data collection exercise. The stakes reach far beyond those of counter-terrorism, and are in a certain sense far more strategic. What state would not be motivated by the ability to predict, on its territory and abroad, forthcoming revolutions or future protest movements, the next economic crisis, or what voters will be most receptive to during the next elections? If major web companies, which nevertheless have only a limited view when compared to that of the NSA, consider social prediction to be a strategic axis, it would be astonishing for the latter organisation not to adopt the same point of view. Furthermore, from the methodological point of view, the prediction (in terms of probability) of collective behaviour represents a considerably more fitting use for these massive volumes of data than the detection of potential terrorists. Indeed, the error rate in the detection of potential terrorists by these means is probably several orders of magnitude higher in proportion than the proportion of the total population represented by these individuals.
However, as pointed out by von Foerster more than 40 years ago (1972), to the extent that the ability to implement social predictions depends on the rigidity of human relationships, the field of application of such predictions is restricted to social activities that are sufficiently trivialized for them to be immune to change: In order to protect society from the dangerous consequences of change, not only a whole branch of business has emerged, but also the Government has established several offices that busy themselves in predicting the future by applying the rules of the past. These are the Futurists. Their job is to confuse quality with quantity, and their products are “future scenarios” in which the qualities remain the same, only the quantities change: more cars, wider highways, faster planes, bigger bombs, etc. While these “future scenarios” are meaningless in a changing world, they have become a lucrative business for entrepreneurs who sell them to corporations that profit from designing for obsolescence.
One may nevertheless conjecture that the introduction of macroscopes, which will certainly become more common in coming years, will have the consequence of strengthening the stability of social systems (more stable attractors, more pronounced polarization effects), whilst at the same time increasing the amplitude of extreme phenomena (more radical social change involving a greater proportion of humanity).
The application of von Foerster's conjecture
The migration of certain social activities towards digital platforms, through the addition of new dimensions to inter-personal relationships, has revealed the visionary nature of von Foerster's conjecture. This also establishes new relationships between the human community and some of its constituents.
Whereas, just a few years ago, the web may have appeared to some to be a virtual universe, relatively disconnected from the real world, the digital world is now one of the territories we explore every day. There, as the transmitters and receivers of information, or members of social networks, a significant proportion of our social life takes place. This new territory nevertheless conserves the properties of its platform, in other words its artificiality. In the dialectics of the internal and external points of view, the question arises as to the degree of reality expressed by the digital world, and the influence of artefacts created with the intention of influencing the course of its history.
Although it would probably have been inconceivable at the time of von Foerster, ICTs currently allow, by means of artificial amplification, any entity to influence social dynamics by taking advantage of the rigidity of inter-personal relationships. Once again, the marketing sector was a pioneer in this adventure. The creation of fake comments by dummy users, concerning this or that product in order to improve their sales, has now become common practice, although it is illegal in many countries.
This practice is now widely used, well beyond the realm of consumption. Online services now propose, for a few hundred dollars, 10,000 ‘likes' or additional followers for your profile on any given social network. The social network industry is moreover beginning to react to this type of practice, by regularly “purging” dummy behaviours from its user base. Thus, in 2014 it was observed that on the Instagram portal most users lost a strong proportion of their followers, and as many as several million were lost by some celebrities, thereby generating an epidemic of very real existential crises. 9
Even those organisations that are expected to regulate these practices find that they are in their interest. Whilst the preparation of a cyber-war is on the agenda of most of the world's leading powers, various secret services are creating fake users on the web – “sock puppets” in jargon – to influence opinions. In March 2011, the Guardian (Fielding and Cobain, 2011) thus revealed a call for tender from the American army for the development of software able to create multiple persona, i.e. fake users, having a history of online activity (blog posts, comments, tweets, etc.) and a presence in cyber-space that would be sufficiently coherent from the technical, geographic and cultural point of view to be taken for humans. This army of fake users, controlled by a small number of individuals armed with this “persona” management software, is designed to infiltrate many different social networks and web 2.0 platforms, and when the time comes to use its influential skills to disseminate ideas suited to the required objectives (discrediting an individual, introducing a reform, supporting a candidate, etc.).
Officially developed to manipulate public opinion in enemy countries, the American army is suspected of using this technology against its own citizens, as was done in the case of the rupture of the levees caused by hurricane Katrina, in order to discredit the local authorities who claimed negligence on the part of the army's engineers. 10 Other governments are also believed to use this type of technology, as for example in the case of the Iranian government, which claims to have the world's second largest cyber-army, and whose cyber-department Paydari infiltrates Facebook in order to join pro-government discussions with other users. Whereas traditional propaganda made massive efforts to deliver messages via reputable individuals, the cyber-space offers the avenue of a complementary strategy. Taking literally the Keynesian vision, according to which “one cannot beat the wisdom of the crowd” (Orléan, 1986), the aim is to create the illusion of a crowd voicing the same opinion, in order to be more convincing.
This practice consisting in the creation of the illusion of a consensus, or a population's spontaneous support for a given cause, has been given the Anglo-Saxon name of “astroturfing”, from the name of a brand of artificial lawn. Although it already existed in the field of marketing, industrial lobbying, or politics, before the arrival of social networks on the Internet, in recent years it has taken on such proportions that it regularly plays a role in important political decisions or the results of elections. For example, researchers in the field of complex systems and data-mining have recently developed information flow reconstruction techniques on the Twitter micro-blogging platform, allowing astroturfing activities to be detected. They thus revealed several cases of opinion manipulation through the use of fake Twitter accounts in the context of elections in the United States (Ratkiewicz et al., 2010). By providing any Internet user with a reconstruction of informational dynamics on Twitter, their platform “Truthy” 11 allows deviant usage to be identified and thus contributes to the foiling of astroturfing practices. This type of macroscope can contribute to the limitation of the effects of astroturfing practices. However, as the latter improve in efficiency when social ties become more rigid, it is above all the education of citizens which will allow their effects to be reduced: learning to select sources of information whilst maintaining a critical point of view of these and of the recommendations they project.
Conclusion
By following the breadcrumb trail sketched 40 years ago by von Foerster, we have been able to perceive the deep modifications experienced by our informational environment in recent years. Although ICT have created hope for a shared pluralistic world, democratic principles are far from being respected in the public digital environment, and require a detailed knowledge of the laws by which they are governed.
Firstly, by becoming aware that the manner in which these spaces are populated by public expression is strongly constrained by their design and the functionalities they propose, such as recommendation systems. These latter has not only the potential to influence socio-economic dynamics through the change of agent's knowledge (Hayek's “division of knowledge” issue – 1937) but also to influence preference formations within a population. Secondly, by ascertaining the change in scale induced by the extent to which information circulates and the improvement in the coordination of individuals. Society tends to function increasingly like an integrated system, whose levers allow whoever is able to use them to make a significant impact on its development.
In this context, whereas the race for the reconstruction and prediction of social dynamics is open, the possibilities for massive “low level” intervention on social systems, or on the contrary exclusive access to an “external” point of view of society, will provide those with such access with the power to dominate, which is contrary to our most fundamental ethical principles (see Epstein and Robertson, 2015 for an empirical illustration). Von Foerster's theorem provides insight into clues to the mitigation of these effects, for example by implementing recommendation systems that rely on known “real life” 12 persons, and by educating populations to monitor the degree of rigidity of their social interactions. Most importantly, we should pay attention to the ethics surrounding von Foerster's research. These define a healthy society as one whose members “perceive others as autonomous, non trivial beings” (von Foerster, 1972). In such a society, “education is neither a right nor a privilege: it is a necessity”. It is each individual's responsibility to determine his/her way forward.
In any event, the fact that organisations such as the NSA are able to jointly conduct programs designed to establish an “external” point of view, in the sense of von Foerster, and programs which, in the words of the NSA (Greenwald, 2014), are designed to use “online techniques to make something happen in the real or cyber world” via “the 4 D's: Deny/Disrupt/Degrade/Deceive”, should motivate us to adopt the highest possible level of vigilance with respect to the expansion of their ability to influence community dynamics. These types of program threaten not only fundamental rights, violating without discrimination the private sphere of the world's population, but also the democratic functioning of our societies, by opening the way for social engineering capable of misappropriating its mechanisms. With the advent of Big Data and the unprecedented improvement of data collection and processing techniques, there is every reason to believe that, in terms of intelligence, we are approaching the threshold of counter-productivity theorised by Illich (1973): “When an empowered activity exceeds a threshold defined by an ad hoc scale, it firstly turns against its own objectives, and then threatens the entire social fabric with destruction.”
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was partially supported by the Algopol grant funded by ANR (ANR-12-CORD-0018) and the ARESOS Mastodons project (CNRS MI).