Abstract
The rapidly increasing deployment of AI raises societal issues about its safety, reliability, robustness, fairness and moral integrity. This paper reports on a declaration intended as a code of conduct for AI researchers and application developers. It came out of a workshop held in Barcelona in 2017 and was discussed further in various follow up meetings, workshops, and AI schools. The present publication is a matter of historical record and a way to publicize the declaration so that more AI researchers and developers can get to know it and that policy makers and industry leaders can use it as input for governance. It also discusses the rationale behind the declaration in order to stimulate further debates.
Motivation
The AI summer is here
It can no longer be denied that Artificial Intelligence is having a fast growing impact in many areas of human activity. It is helping humans to communicate with each other – even beyond linguistic boundaries, find relevant information in the vast information resources available on the web, solve challenging problems that go beyond the competence of a single expert, enable the deployment of autonomous systems, such as self-driving cars or other devices that handle complex interactions with the real world with little or no human intervention, and many other useful things. These applications are perhaps not like the fully autonomous conscious intelligent robots that science fiction stories have been predicting, but they are nevertheless very important and useful, and most importantly they are real and here today.
The growing impact of AI has triggered a kind of ‘gold rush’: we see new research laboratories springing up, new AI start-up companies, and very significant investments, particularly by big digital tech companies, but also by transportation, manufacturing, financial, and many other industries. Management consulting companies are competing in their predictions how big the economical impact of AI is going to be and governments are responding with strategic planning to see how their countries can avoid staying behind.
Clearly most of the activity is in the US [18] and China but there are also signs of enhanced AI activity in Europe and anouncements of action plans by various European governments and the European Commission. The Macron 1.5 billion Euro strategic plan for stimulating AI in France [26] is one example. Although European strategic proposals are today (i.e. in 2018) mostly still in the phase of promises, European AI researchers, developers and entrepreneurs hope that they will provide structural funding for AI in the near future and that AI becomes recognized in upcoming European framework programs as a research field with a clear economic impact and hence in need of significant structural funding.
Clouds on the horizon
Although all this is positive news, it cannot be denied that the application of AI comes with certain risks. Many people (including luminaries such as Bill Gates, Elon Musk, or Stephen Hawking) believe that the main risk of AI is that its deployment would get out of hand. Machines that can learn, reconfigure themselves, and make copies of themselves may one day outrun the human race, become smarter than us and take over. To researchers in the field this risk seems far-fetched. But they see other risks, which are already upon us and need urgent remediation. Here are some examples: AI algorithms, particularly those embedded in the web and social media, are having an important impact on who talks to whom, how information is selected and presented, and how facts (justified or fake) propagate and compete in public space. Critics point out that these AI algorithms are now held (at least partly) responsible for allowing the emergence of a post-truth world, highjacking democratic decision processes, and dangerously polarizing society. Polarization is making it much more difficult to deal with the big issues facing our society, such as climate change mitigation, diminishing pollution, achieving economic prosperity for an exploding world population, avoiding violent conflicts due to ethnic, nationalistic or religion diversity, coping with massive migration, etc. They all require determined collective action and therefore a political consensus. AI should (and could) help to support consensus formation rather than destroy it. Many applications use deep learning or other forms of statistical inference to great advantage. For many of these applications, such as speech recognition or machine vision, this technique is the most effective one found so far. But the applications of deep learning to domains that involve rule-governed behavior and human issues, such as financial decision making, human resource management, or law enforcement, has been shown to be problematic from a humanistic point of view. Job seekers report frustration to get through the machine learning based filters which reinforce gender and class and focus on keywords or features of a cv that are not essential nor fair [12]. The use of AI in decisions on parole has caused an outcry because the basis of these decisions is obscure due to the black box nature of deep learning and biased in ways that are unacceptable [6]. All of this raises growing questions about the robustness, explainability, reliability and accountability of AI systems based on deep learning. Self-driving cars act upon their own decisions, unavoidably leading to risks to human life. More generally, do we need to put limits on autonomous artificial intelligence systems? Who is responsible when something goes wrong? And what about other applications of autonomous AI such as autonomous weapons. The AI community is already speaking out against their use1 See for example https://futureoflife.org/open-letter-autonomous-weapons/.
Several initiatives have been taken in recent years to understand better the risks of AI deployment and come up with legal frameworks, codes of conduct, and value-based design methodologies. Examples are the Alomar principles for beneficial AI,2
the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems,3 the technology industry consortium ‘Partnership on AI to benefit people and society’,4 or the EU GDPR regulation which includes the right for an explanation [10]. There is also a rapidly growing literature on the risks of AI and how to handle it (see for example [3–5,17,21]). hybrid Against this background, Luc Steels and Ramon Lopez de Mantaras organised in march 2017 a debate in CosmoCaixa Barcelona under the auspices of the Biocat and l’Obra Social la Caixa with support from ICREA, the Institut de Biologia Evolutiva (UPF/CSIC) and the Institut d’Investigacio en Intel-ligencia Artificial (CSIC). More information about this event is found here: http://www.bdebate.org/en/forum/artificial-intelligence-next-step-evolutionThe event assembled a number of top experts in Europe who are concerned with the benefits and risks of AI – particularly, but not exclusively, in the domain of web and social media – and to discuss strategies to deal with these risks. The Barcelona initiative is complementary to other ongoing efforts because (i) it intends to stimulate the debate within Europe whereas other initiatives are primarily in the Anglo–American sphere, and (ii) give a voice to European AI developers and researchers, whereas most of the discussion on ethical AI so far has been dominated by social scientists, legal experts and business consultancy firms.
The Barcelona meeting featured a small-scale workshop on March 7th with two sessions followed by discussion:
The second day was open to the public. It featured the following presentations, recorded and available through: http://www.bdebate.org/en/videos
Part A. Advances in knowledge-based AI:
1. How is the semantic web transforming information access. Guus Schreiber (Network Institute, Vrije Universiteit Amsterdam)
2. How are advances in language processing helping to bring order in cyberspace. Walter Daelemans (Computational Linguistics, University of Antwerp).
Part B. Advances in machine learning:
1. How do recent developments in deep learning increase its power and scope of application. Joan Serra (Telefonica I+D, Barcelona),
2. Why is the industrial impact of machine learning growing so fast? Francisco Martin (BigML, Corvallis Oregon US).
How do AI algorithms influence the selection of media content. Cornelius Puschmann (Hans Bredow Institute for Media Research, Hamburg)
The complex dynamics of rumour and fake news spreading. Walter Quattrociocchi (Ca’Foscari University of Venice)
Best practices for the development and deployment of AI. Francesca Rossi (University of Padova, Italy) Technologies for the democratic city. Francesca Bria. (Comisionada de TecnologÌa e Innovacion Digital, Ayuntamiento Barcelona)
The symposium concluded with a panel discussion and a presentation by Luc Steels of the ‘Barcelona Declaration for the Proper Development and Usage of Artificial Intelligence in Europe’. The declaration was then signed by most of the participants and from then on has become accessible for signature and discussion on the web.
Main points of the Barcelona Declaration
This section reprints the complete text of the declaration. A summary is provided in Table 1.
We distinguish between knowledge-based AI and data-driven AI.
Knowledge-based AI has shown to be most successful in intellectual tasks, such as expert problem solving, whereas data-driven AI is most successful in tasks requiring intuition, perception, and robotic action. The full potential of AI will only be realized with a combination of these two approaches, meaning a form of
We believe that AI can be a force for the good of society, but that there is a sufficient danger for inappropriate, premature or malicious use to warrant the need for raising awareness of the limitations of AI and for collective action to ensure that AI is indeed used for the common good in safe, reliable, and accountable ways.
Barcelona, 8 March 2017
The list of current signataries is available through the website:
https://www.iiia.csic.es/barcelonadeclaration/
It is still possible to sign the declaration through the same site.
After the event in Barcelona, the declaration was spread through various AI research channels and public media. It was integrated in various discussions on the future governance of AI in Europe, for example, at a hearing in Brussels of the EU political Strategy Center.6
The declaration was also discussed at various AI schools and fora, such as the 2017 edition of the Interdisciplinary College IK in Guenne, Germany.In general, the declaration contributed to raise awareness and has given additional impetus to initiatives by governments and law makers in many European countries, such as the Netherlands [20], Belgium [23], Denmark [24], the UK [1], a.o. In some cases, the recommendations of the declaration were explicitly referred to in parliamentary hearings [13]. In addition, the European Commission initiated in the spring of 2018 a High-level Expert Group on Artificial intelligence7
as a steering group of the newly formed European AI alliance,8 which includes stakeholders ranging from industry to policy makers and academics.So, although the landscape of AI in Europe is rapidly changing through all these discussions and activitites, the issues raised in the declaration remain highly relevant. The remainder of this section highlights some of them.
When AI is interpreted too broadly like this, there is a risk, for the field of AI itself, to be blamed for malicious applications, business practices or societal phenomena, such as fake news, hate speech or cyber crime, even if no AI is involved at all. The declaration therefore proposed to focus only on ethical issues as related to AI in the narrow sense. And even if we maintain this restriction, we need to be more precise whether we are talking about knowledge-based AI or data-oriented machine learning, because the legal and ethical issues for both approaches are quite different. For example, the topic of explainability was already an important component of knowledge-based systems built in the 1980’s and adequate approaches have been developed and used extensively [16], whereas explanation is highly problematic for current machine learning techniques such as deep learning and it is still very unclear how it could be achieved [2].
All of these initiatives are very welcome but they are statements of intent and concrete actions with a direct impact on the deployment of AI, or, just as important, on the creation of stable funding for AI research and education in Europe are still rare. One positive sign was a first 20 mi EU call within the European H2020 framework program (with deadline march 2018) that explicitly targeted the stimulation of European AI research in line with what was proposed in the Barcelona declaration, namely the creation of a European ecosystem and a platform in which European actors could share resources in the form of machine learning algorithms, data sets, knowledge bases, ontologies, lexicons, grammar models, etc. The AI4EU consortium has been selected and activity is planned to start in January 2019. Moreover Cecile Huet from the Future and Emergent Technologies office (DG CNECT) outlined the Artificial Intelligence Strategy for Europe at IJCAI-2018 in Stockholm, which foresees an important increase of opportunities for AI research (
A typical example is a recent hype episode about an experiment carried out by Facebook researchers on the acquisition of skills in negotiation, with the acquisition of language skills as a secondary needed competence [15]. The paper was published on arXiv which ensures rapid dissemination in the machine learning community, and on a company blog,10
which ensures that the experiment is picked up by the media. So far so good.However, a report in the media, on the website Fast Company, did not discuss the negotiation experiment itself but focused entirely on the acquisition of language skill with the headline: “AI Is Inventing Languages Humans Can’t Understand. Should We Stop It?”, commenting “Researchers at Facebook realized their bots were chattering in a new language. Then they stopped it.”11
Indeed, non-English dialogs started to be produced such as this one:Bob: you i everything else....
Alice: balls have a ball to me to me to me to me to me to me to me.
Although the researchers never mentioned anything about stopping the experiment for this reason.
The phenomenon of novel language emergence is in itself interesting, particularly to those in the AI research community that have been studying for decades the cultural evolution of language through a wide range of agent-based experiments, including with embodied robots [22]. But then, web media, blogs, and newspapers picked up the theme of self-generated language and elaborated only on the potential dangers with stories that became progressively more and more scary. For example, the UK tabloid The Sun reported: “Facebook shuts off AI experiment after two robots begin speaking in their OWN language only they can understand” and quotes Kevin Warwick as “anyone who thinks this is not dangerous has got their hand in the sand”. The Sun adds: “The incident closely resembles the plot of The Terminator in which a robot becomes self-aware and starts waging a war on humans.”12
Newspapers all over Europe picked up the story, all adding their own twists and exagerations and confronting AI researchers with this supposed step towards AI disaster and urging politicians to stop this madness.The Facebook researchers involved surely did not intend this media storm but these stories are the ones that stick into the public understanding of AI. Clearly much greater care needs to be taken in communicating AI experiments. Otherwise the lack of prudence will without doubt lead to a new AI winter as the expectations and scare stories currently being created by overzealous media are impossible to fulfill and they overpower the more reasonable statements that most AI researchers tend to make.
All this work builds up possible experience that can lead to certification procedures but there is still a considerable road to travel before this will be as uncontroversial as certifying a new refrigerator or new medicine.
Here is a recent example where this recommendation has been violated. In may 2018 Google demonstrated a speech understanding system called Google Duplex that is claimed to be able to hold a conversation over the phone for ordering a reservation in a restaurant or make similar appointments for services [14]. This result is quite interesting although the boundaries of the system’s performance are not very clear and some have even questioned whether the demo was in real circumstances. But the fact that Google Duplex tried to con humans into believing that the conversation was not with a machine, created an immediate backlash and a promise that in the future the system would identify itself.15
Moreover some observers were quick to realise the abuse that could be made of this technology. Here is for example a typical reaction (posted on the Google Blog anouncing the Duplex demonstration).A couple of problems spring immediately to mind. First, the use of embedded “uh”s and other artifacts to try fool the listener into believing that they are speaking to a human may well engender blowback as these systems are deployed. My sense is that humans in general don’t mind talking to machines so long as they know that they’re doing so. I anticipate significant negative reactions by many persons who ultimately discover that they’ve been essentially conned into thinking they’re talking to a human, when they actually were not. It’s basic human nature – an area where Google seems to have a continuing blind spot. Another problem of course is whether this technology will ultimately be leveraged by robocallers (criminal or not) to make all of our lives even more miserable while enriching their own coffers.
A possible response to avoid these problems is to legally require that any AI system should make it explicitly clear upfront that it is an artificial system, so that human users can also shield themselves from calls by such systems.
On the other hand, the discussion on automation and employment is erroneously centered on only the number of jobs. Instead it should be focused on the changing nature of work. Some tasks have been, and will continue to be, automatized but the number of jobs where the majority, or all, of its tasks can be automatized is not as large as some studies say. There are studies in different European countries that show how robotization has in fact increased the global number of jobs, creating jobs with higher quality and better paid. On example is a study in Catalonia done at the Catalan Open University with several hundred SMEs from 2002 to 2014 [25].
A similar study done by the Centre for European Economic Research in Mannheim, Germany, also shows that automation resulted in an overall increase in (better paid) jobs in Germany between 2011 and 2016. This does not mean that we should not be concerned by the effects of AI and automation in general on employment (not only robots in manufacturing but also clerical work and professional services) but we should focus our concerns and find solutions to the problem of the changing nature of jobs, train workers to face this challenge and ensure that automation does not increase inequality in society.
This paper contributes to ongoing discussions in Europe related to the ethical issues of AI. We focused on the ‘Barcelona Declaration for the Proper Development and Use of AI’, which was launched in the spring of 2017, and discussed some of its ramifications. Given the public interest in AI and the eagerness of many organisations, both private companies and governmental institutions, to develop applications that affect people in their daily lives, it is important that the AI community, encompassing application developers as well as researchers, engages in open discussions, partly to avoid over-expectations with an unavoidable backlash later and partly to avoid improper usage of AI that causes unneeded negative side effects and undue human suffering. At the same time, we must realize that no set of rules or in-built technological constraints can ever avoid malicious use by unscrupulous actors. The ultimate responsibility always lies with humans, both as designers and as users, and they should be held accountable.