Big Data: From modern fears to enlightened and vigilant embrace of new beginnings

Abstract

In The Black Box Society, Frank Pasquale develops a critique of asymmetrical power: corporations’ secrecy is highly valued by legal orders, but persons’ privacy is continually invaded by these corporations. This response proceeds in three stages. I first highlight important contributions of The Black Box Society to our understanding of political and legal relationships between persons and corporations. I then critique a key metaphor in the book (the one-way mirror, Pasquale’s image of asymmetrical surveillance), and the role of transparency and ‘watchdogging’ in its primary policy prescriptions. I then propose ‘relational selfhood’ as an important new way of theorizing interdependence in an era of artificial intelligence and Big Data, and promoting optimal policies in these spheres.

This article is a part of special theme on The Black Box Society. To see a full list of all articles in this special theme, please click here: https://journals.sagepub.com/page/bds/collections/revisitingtheblackboxsociety

Introduction

In The Black Box Society: The Secret Algorithms Behind Money and Information, Frank Pasquale digs deep into three sectors that are at the root of what he calls the black box society: reputation (how we are rated and ranked), search (how we use ratings and rankings to organize the world), and finance (money and its derivatives, whose flows depend crucially on forms of reputation and search). Two of the meanings of ‘black box’ – a persistent monitoring device, on the one hand, and the node of a system that prevents an observer from identifying the link(s) between input and output, on the other hand – serve as apt metaphors for today’s emerging Big Data environment. Data is gathered intensively, but its processing and use is opaque.

Algorithms and Big Data in reputation assessment, media, and finance often exhibit a disconnection with human judgment or control. This disconnection can transmogrify them into blind zombies, opening new risks, affordances, and opportunities. We are far from the ideal representation of algorithms, as support for (rather than replacement of) human decision-making. All too often, decision-making has been taken over by algorithms, and there is no ‘invisible hand’ ensuring that profit-driven corporate strategies will deliver fairness or improve the quality of life.

All these points are either in, or in harmony with, Pasquale’s approach. Where Pasquale does not go, but which I believe is very important to the analysis, is a broader temporal framework. We must reconsider Modern referential frameworks, both solipsistic and individualistic, which hinder our ability to attend to changing human needs, desires, and expectations in this emerging hyperconnected era. That does not mean going back to the Middle Ages, as feared by some, but instead stepping firmly into this new era that is coming to us.

To develop this argument, this review proceeds in three stages. I first highlight important contributions of The Black Box Society to our understanding of political and legal relationships between persons and corporations. I then critique a key metaphor in the book (the one-way mirror, Pasquale’s image of asymmetrical surveillance), and the central role of transparency and ‘watchdogging’ in its set of policy prescriptions. I then propose ‘relational selfhood’ as an important new way of theorizing interdependence in an era of artificial intelligence and Big Data and promoting optimal policies in these spheres.

The personification of corporations

One of the red threads unifying The Black Box Society’s treatment of numerous technical subjects is unveiling the oddness of the comparative postures and status of corporations and people. As nicely put by Pasquale, ‘corporate secrecy expands as the privacy of human beings’ contracts’ (Pasquale, 2015: 26) and, in the meantime, the divide between government and business is narrowing (Pasquale, 2015: 206). It is my view that these oddnesses go along with what I would call a sensitive inversion (Dewandre and Gulyás, 2018). Corporations, which are functional beings, are granted sensitivity as if they were human beings, in policy-making imaginaries and narratives, while men and women, who are sensitive beings, are approached in policy-making as if they were functional beings: consumers, job-holders, investors, bearer of fundamental rights, but never personae per se. The granting of sensitivity to corporations goes beyond the legal aspect of their personhood. It entails that corporations are the one whose so-called needs are taken care of by policy makers, and those who are really addressed to, qua persona. Policies are designed with business needs in mind, to foster their competitiveness or their ‘fitness’. People are only indirect or secondary beneficiaries of these policies.

Pasquale’s critique of the hypostatization of corporations and reduction of humans has many theoretical antecedents. Looking at it from the perspective of Hannah Arendt’s The Human Condition illuminates the shortcomings and risks associated with considering corporations as agents in the public space and understanding the consequences of granting them sensitivity. Action is the activity that flows from the fact that men and women are plural and interact with each other: ‘the human condition of action is plurality’. Plurality is itself a ternary concept made of equality, uniqueness, and relationality. Equality is what we grant to each other when entering into a political relationship. Uniqueness refers to the fact that what makes each human a human qua human is precisely that who s/he is, is unique. The third component of plurality is the relational and dynamic nature of identity. For Arendt, the disclosure of the who ‘can almost never be achieved as a wilful purpose, as though one possessed and could dispose of this ‘who’ in the same manner he has and can dispose of his qualities’ (Arendt, 1958: 159).

For Arendt, the essence of politics is freedom and is grounded in action, not in labour and work. The public space is where agents coexist and experience their plurality: the fact that they are equal, unique, and relational. So, it is much more than the usual American pluralist (i.e., early Dahl-ian) conception of a space where agents worry for exclusively for their own needs by bargaining aggressively. In Arendt’s perspective, the public space is where agents, self-aware of their plural characteristic, interact with each other once their basic needs have been taken care of in the private sphere. As highlighted by Seyla Benhabib in The Reluctant Modernism of Hannah Arendt, ‘we not only owe to Hannah Arendt’s political philosophy the recovery of the public as a central category for all democratic-liberal politics; we are also indebted to her for the insight that the public and the private are interdependent’ (Benhabib, 2003: 211).

It is easy to imagine that there can indeed be no freedom below satiety, and that ‘sheer togetherness’ would just be impossible among agents below their satiety level or deprived from having one. This is however the situation we are in, symbolically, when we grant corporations the status of persona while considering efficient and appropriate that they care only for profit-maximisation. For a business, making profit is a condition to stay alive, as for humans, eating is a condition to stay alive. However, in the name of the need to compete on global markets, to foster growth and to provide jobs, policy-makers embrace and legitimize an approach to businesses as profit-maximisers, much beyond supporting their need to be, more simply, profit-makers (Bold, 2018; Stout, 2012). So, the condition for businesses to deserve the status of persona in the public space is, no less than for men and women, to attend their whoness and honour their identity, by staying away from behaving according to their narrowly defined interests. It means also to care for the world as much, if not more, as for themselves.

Some might find a book like The Black Box Society, which offers a bold reform agenda for numerous agencies, to be too idealistic. But in my view, it falls short of being idealistic enough: there is a missing normative core to the proposals in the book, which can be corrected by democratic, political, and particularly Arendtian theory. If a populace has no acceptance of a certain level of goods and services prevailing as satiating its needs, and if it distorts the revelatory character of identity into an endless pursuit of a limitless growth, it cannot have the proper lens and approach to formulate what it takes to enable the fairness and fair play described in The Black Box Society.

The limits of the one-way mirror metaphor and watchdogging prescription

Frank Pasquale is well aware of and has contributed to the emerging critique of transparency, and he states clearly that ‘transparency is not just an end in itself’ (Han, 2015: 8). However, there are traces of the Modern reliance on transparency as regulative ideal in The Black Box Society. One of them is when he mobilizes the one-way mirror metaphor. He writes:

We do not live in a peaceable kingdom of private walled gardens; the contemporary world more closely resembles a one-way mirror. Important corporate actors have unprecedented knowledge of the minutiae of our daily lives, while we know little to nothing about how they use this knowledge to influence the important decisions that we—and they—make. (Pasquale, 2015: 9)

I refrain from considering the Big Data environment as an environment that ‘makes sense’ on its own, provided someone has access to as much data as possible. In other words, the algorithms crawling the data can hardly be compared to a ‘super-spy’ providing the data controller with an absolute knowledge. Another shortcoming of the one-way mirror metaphor is that the implicit corrective is a transparent pane of glass, so the watched can watch the watchers.

This reliance on transparency is misleading. I prefer another metaphor that fits better: a change of ‘social atmosphere’ or ‘social gravity’. For centuries, we have slowly developed social skills, behaviours, and regulations to strike a balance between accountability and freedom, in a world where ‘verba volant and scripta manent’, where human interactions took place in an ‘atmosphere’ with a defined and understandable ‘social gravity,’ reminiscent of standard 1 g gravitational force on earth. They were evanescent by default and action had to be taken to register them. Now, with all interactions leaving a trace by default, and each of us going around with our digital shadow, we are drifting fast towards an era where the ‘social atmosphere’ will be of heavier gravity, say ‘10 g’. The challenge is huge and will require a lot of collective learning and adaptation to develop the literacy and regulatory frameworks that will recreate and sustain the balance between accountability and freedom for all agents, human, and corporations.

The heaviness of this new data density is orthogonal to the two phantasms of bright emancipatory promises of Big Data or frightening fears of Big Brother. Because of this social hypergravity we, individually and collectively, have indeed to be cautious about the use of Big Data. This heavier atmosphere opens to increased possibilities of hurting others, notably through harassment, bullying and false rumours (Pasquale, 2012). The advent of Big Data does not, by itself, provide a ‘license to fool’ nor does it free agents from the need to behave and avoid harming others.

Another ‘leftover’ of the Modern conceptual framework that surfaces in The Black Box Society is the reliance on watchdogging for ensuring proper behaviour by corporate agents. Relying on watchdogging for ensuring proper behaviour nurtures the idea that it is all right to behave badly, as long as one is not seen doing do. This puts the entire burden on the watchers and no burden whatsoever on the doers. Far from empowering the watchers, this enslaves them to waste time monitoring actors who should be acting in much better ways to start with. If, instead, proper behaviours are witnessed, then the watchers are bound to praise the doers. In both cases, watchers are stuck in a passive, reactive and specular posture, while all the glory or the shame is on the side of the doers. I don’t deny the need to have watchers, but I warn against the temptation of relying excessively on the divide between doers and watchers to police behaviours, without engaging collectively in the formulation of what proper and inappropriate behaviours are.

A new era of hyperconnectivity

I believe with Paul Dourish and Genevieve Bell (2011) we are not merely anticipating, but entering into, the ubiquitous computing era. We live in an environment which is increasingly reactive as a result of the intricate mix between off-line and online universes. Human interactions are also deeply affected by this new socio-technico-natural compound, as they are or will soon be ‘sticky’, i.e. leave a material trace by default, and this for the first time in history. The era of untraceability and anonymity may not be over, but technology and its constant implementation are continually limiting the scope of such immunities to inspection and recording. These new affordances and constraints destabilize profoundly our Modern conceptual frameworks, which rely on distinctions that are blurring. It is time to give a proper name to this new era we are stepping into, and hyperconnectivity may be such a name.

When it comes to addressing the challenges described by The Black Box Society, it is important to mention the epistemological stance that has been spelled out brilliantly by Susan H Williams: ‘the connection forged in Cartesianism between knowledge and power’ (Williams, 2004: 35). Before encountering Williams’s work, I came to refer to this stance less elegantly with the expression ‘omniscience-omnipotence utopia’. Williams writes that ‘this epistemological stance has come to be so widely accepted and so much a part of many of our social institutions that it is almost invisible to us’ and that ‘as a result, lawyers and judges operate largely unself-consciously with this epistemology’ (Williams, 2004: 32). To Williams’s ‘lawyers and judges’, we should add policy-makers and stakeholders. This Cartesian epistemological stance is eminently Modern, as it grounds a conviction that the world can be elucidated in causal terms, that knowledge is about prediction and control, and that there is no limit to what men can achieve provided they have the will and the knowledge. In this Modern worldview, men are considered as rational subjects and their freedom is synonymous with control and autonomy. They demand transparency above all, as a ground for exercising autonomy and control. Despite a mounting body of empirical research undermining these assumptions, policy-making continues to rely heavily on Modern conceptual frameworks. This not only from the policy-makers’ point of view, but more widely from all those engaging in the public debate.

The concept of the rational subject was forged to erect Man over nature. Nowadays, the problem is not so much to distinguish men from nature, but rather to distinguish men – and women – from artefacts. Robots come close to humans and even outperform them if we continue to define humans as rational subjects. The figure of the rational subject is torn apart between ‘truncated gods’ – when Reason is considered as what brings eventually an overall lucidity – and ‘smart artefacts’ – when reason is nothing more than logical steps or algorithms.

This is what happens in the Kafkaesque and ridiculous situations where a thoughtless and mindless approach to Big Data is implemented, and this from both stances, as workers and as consumers. This mindless use of automation is only the last version of the way we have been thinking for the last decades: that progress means rationalisation and de-humanisation across the board. The real culprit is not algorithms themselves, but the careless and automaton-like human implementers and managers who act along a conceptual framework according to which rationalisation and control is all that matters. More than the technologies, it is the belief that management is about control and monitoring that makes these environments properly in-human.

With her concept of plurality, Arendt offers an alternative to the rational subject for defining humanness: that of the relational self (Dewandre, 2015, 2018). The relational self, as it emerges from the Arendtian’s concept of plurality (Ess, 2015) is the man, woman or agent self-aware of their plurality: the facts that (i) they are equal to their fellows; (ii) they are unique as all other fellows are unique; and (iii) their identity as a revelatory character requiring to appear among others in order to reveal itself through speech and action.

The relational self, as arising from Arendt’s plurality, combines relationality and freedom. It resonates deeply with the vision proposed by Susan H Williams, the relational model of truth and the narrative model to autonomy, in order to overcome the shortcomings of the Cartesian and liberal approaches to truth and autonomy without throwing the baby – the notion of agency and responsibility – out with the bathwater (Williams, 2004).

Adopting the relational self as the canonical figure of humanness instead of the rational subject’s one puts under the light the direct relationship between the quality of interactions and the quality of life. In contradistinction with transparency and control, which are meant to empower non-relational individuals, relational selves are self-aware that they are in need of respect and fair treatment from others, instead. It also makes room for vulnerability, notably the vulnerability of our attentional spheres, and for saturation: the fact that we have a limited attention span and are far from making a ‘free choice’ when clicking on ‘I have read and accept the Terms & Conditions’. Instead of transparency and control as policy ends in themselves, the quality of life of relational selves and the robustness of the world they construct together and that lies between them depend critically on being treated fairly and not being fooled.

In a hyperconnected era, one can see clearly why the recommendations Pasquale offers in his final two chapters ‘Watching (and Improving) the Watchers’ and ‘Towards an Intelligible Society,’ are so important. Indeed, if watchdogging the watchers has been criticized earlier in this review as an exhausting labouring activity that does not deliver on accountability, improving the watchers goes beyond watchdogging and strives for a greater accountability, more meaningful and relevant than transparency.

What I appreciate in Pasquale’s call for intelligibility is that it calibrates the needs of relational selves to interact with each other, to make sound decisions and to orient themselves in the world. Intelligibility is different from omniscience-omnipotence. It is about making sense of the world, while keeping in mind that there are different ways to do so. Intelligibility connects relational selves to the world surrounding them and allows them to act with other and move around. In the last chapter, Pasquale mentions the importance of restoring trust and the need to nurture a public space in the hyperconnected era. He calls for an end game to the Black Box. I agree that conscious deception inherently dissolves plurality and the common world, and needs to be strongly combatted, but I think that a lot of what takes place today goes beyond that and is really new and unchartered territories and horizons for humankind. With plurality, we can also embrace contingency in a less dramatic way that we used to in the Modern era. Contingency is a positive approach to un-certainty, accounting for the openness of the future.

The advent of Big Data can be looked at in two ways: as the endpoint of the materialisation of all the promises and fears of Modern times, or as a wake-up call for a new beginning. By making obvious the absurdity or the price of going all the way down to the consequences of the Modern conceptual frameworks, it calls on thinking on new grounds about how to make sense of the human condition and make it thrive. The former makes humans redundant, is self-fulfilling, and does not deserve human attention and energy. Without any hesitation, I opt for the latter: the wake-up call and the new beginning. Let’s engage in this hyperconnected era bearing in mind Virginia Woolf’s ‘Think we must’ (Woolf, 1966) and, thereby, shape and honour the human condition in the 21st century.