Big Data is a big lie without little data: Humanistic intelligence as a human right

Abstract

This article introduces an important concept: Transparency by way of Humanistic Intelligence as a human right, and in particular, Big/Little Data and Sur/Sous Veillance, where “Little Data” is to sousveillance (undersight) as “Big Data” is to surveillance (oversight). Veillance (Sur- and Sous-veillance) is a core concept not just in human–human interaction (e.g. people watching other people) but also in terms of Human–Computer Interaction. In this sense, veillance is the core of Human-in-the-loop Intelligence (Humanistic Intelligence rather than Artificial Intelligence), leading us to the concept of “Sousveillant Systems” which are forms of Human–Computer Interaction in which internal computational states are made visible to end users, allowing users (but not requiring them) to “jump” into the computational feedback loop whenever or wherever they want. An important special case of Sousveillant Systems is that of scientific exploration: not only is (big/little) data considered, but also due consideration must be given to how data is captured, understood, explored, and discovered, and in particular, to the use of scientific instruments to collect data and to make important new discoveries, and learn about the world. Science is a domain where bottom-up transparency is of the utmost importance, and scientists have the right and responsibility to be able to understand the instruments that they use to make their discoveries. Such instruments must be sousveillant systems!

Introduction

Surveillance ¹ (oversight, i.e. being watched) and sousveillance ² (undersight, i.e. doing the watching) can both be thought of in the context of control theory and feedback loops. ³

In particular, McStay considers surveillance in this way, i.e. in regard to the form of privacy that is inherently violated by profiling, and related closed-loop feedback systems that manipulate us while monitoring us (McStay, 2011). Ruppert considers the interplay between surveillance and public space, through a case study of Toronto’s Dundas Square (Ruppert, 2006), where security guards prohibit the use of cameras while keeping the space under heavy camera surveillance. This surveillance without sousveillance (Mann, 2002a) creates a lack of integrity, i.e. surveillance is a half-truth without sousveillance (Mann et al., 2015).

The intersection of Sousveillance and Media was pioneered by Bakir, i.e. sousveillance as not merely a capture or memory right, but also sousveillance as a disseminational (free-speech) right. This gave rise to two important concepts: sousveillance cultures and sousveillant assemblage (Bakir, 2010), analogous to the “surveillant assemblage” of Haggerty and Ericson (2000).

Surveillance has strong connections to Big Data, where states and other large organizations, especially in law enforcement, collect data secretly, or at least maintain some degree of exclusivity in their access to the data (Newell, 2013).

Two important concepts have been proposed to help mitigate this one-sided nature of Big Data: (1) “giving Big Data a social intelligence” (Ruppert et al., 2015) and (2) the concept of “Personal Big Data” (Gurrin et al., 2014) which might more properly be called “little data”. Both of these concepts embody Big Data’s sensory counterpart that corresponds more to sousveillance than surveillance (Mann, 2016a).

The veillance divide is justice denied

A good number of recent neologisms like: “Big Data”, “Internet of Things” (“IoT”), “Artificial Intelligence” (“AI”), etc., describe technologies that aim to grant the gift of sight, or other sensory intelligence, to inanimate objects. But at the same time these inanimate objects are being bestowed with sight, that very same sight (ability to see, understand, remember, and share what we see) is being taken away from humans. People are being forbidden from having the same sensory intelligence bestowed upon the things around them.

Indeed, we’re surrounded by a rapidly increasing number of sensors (Cardwell, 2014) feeding often closed and secretive “Big Data” repositories (Pasquale, 2015). Entire cities are built with cameras in every streetlight (Miao, 2015; Spielman, 2015). Automatic doors, handwash faucets, and flush toilets that once used “single-pixel” sensors now use higher-resolution cameras and computer vision (Iott and Nelson, 2005) (and with time-of-flight camera in US20150268342; see also gesture-sensing shower, US20080256494).

Surveillance is also widely used without regard to genuine privacy, i.e. with only regard to Panoptic-privacy. In Alberta, for example, the Privacy Commissioner condones the use of surveillance cameras in the men’s locker rooms of Calgary’s Talisman Centre where people are naked (David Fraser, 2007) as long as only the police (or other “good people”) can see the video. Westside recreation Centre also in Calgary, Alberta, also uses surveillance cameras in their men’s (but not women’s) locker rooms (Frakes, 2016) (Figure 1).

Figure 1.

Westside Recreation Centre in Alberta, Canada, uses surveillance cameras in their men’s (but not women’s) shower/locker/changerooms (Edwardson, 2016), while they also have a “no cell phone” policy allegedly to “protect the privacy” of their guests.

While surveillance (oversight) is increasing at an alarming rate, we’re also seeing a prohibition on sousveillance (undersight).

Martha Payne, a 9-year-old student at a school in Scotland, was served disgusting school lunches that lacked nutritional value. So in 2012 she began photographing the food she was served (Payne and Payne, 2012). When she began sharing these photographs with others, she generated a tremendous amount of online discussion on the importance of good school nutrition. And she brought about massive improvements in the nutritional value of school lunches around the world. She also raised considerable money for charity, as a result of her documentary photography. But, in part due to the notoriety of her photo essays, she was suddenly banned from bringing a camera to school, and barred from photographing the lunches she was served by her school.

While schools begin the process of installing surveillance cameras, students are increasingly being forbidden from having their own cameras. And for many people on special diets, or with special health needs, using photography to monitor their dietary intake is a medical necessity. I proposed the use of wearable sensors (including wearable cameras) for automated dietary intake measurement in 2002 (US Pat. App. 20020198685; Mann, 2002c). This concept is now gaining widespread use for self-sensing and health monitoring (Doherty et al., 2013b, 2013a).

So when people suffer from acute effects like food poisoning or allergic reactions, or from longer-term chronic effects of poor nutrition, like obesity, being forbidden from keeping an accurate diary of what they have eaten is not just an affront to their free speech. It is also a direct attack on their health and safety.

Neil Harbisson, a colorblind artist and musician, has a wearable computer vision system that allows him to hear colors as musical tones. And he wears his camera and computer constantly. Wearable computing and Personal Imaging (wearable cameras) are established fields of research (Mann, 1997), dating back to my augmented reality vision systems of the 1970s. I also wear a computer vision system and visual memory aid. Harbisson and Mann both have cameras attached in such a way as to be regarded as part of their bodies, and thus their passports both include the apparatus, as it is a part of their true selves and likenesses (see Figure 2). And we are not alone: many people now are beginning to use technology to assist them in their daily lives, and in some ways, the transition from a surveillance society to a veillance society (i.e. one that includes both surveillance and sousveillance) is inevitable (Ali and Mann, 2013).

Referring back to Martha Payne, discussed earlier in this article, there is often an hypocrisy of the officials (schools, as well as society in general) wanting to collect more and more data about us, while forbidding us from collecting data about them or about ourselves (like monitoring our own dietary intake, monitoring our exercise, or helping us see and remember what we see). We need to be critical of this hypocrisy because (1) it is a direct threat to our health, wellness, and personal safety, and (2) data obtained under this hypocrisy lacks integrity (integrity is the opposite of hypocrisy). Thus it is with great joy and relief that we learn how Martha Payne fought back and won the right to continue using photography to monitor her dietary intake, not only for the journalistic freedom (in the Bakir sense), but also for the personal safety that such self-monitoring systems can provide.

Surveillance is a half-truth, without sousveillance

Surveillance is the veillance of hypocrisy, in the sense that, as often practiced by security guards, closely monitoring surveillance cameras, these guards tend to observe and object to individuals taking pictures in the surveilled spaces. The opposite of hypocrisy is integrity (Mann et al., 2015).

Surveillance typically tells a story from the side of the security forces. When stories told from other points-of-view are prohibited, i.e. the capturing of evidence to support these other points-of-view is prohibited, the total captured evidence is less than the full truth (Mann et al., 2015). In this sense, surveillance often gives rise to a half-truth.

Justeveillance (fair sight) in AI and machine learning

Much has been written about equiveillance, i.e. the right to record while being recorded (Manders, 2013; Mann et al., 2006; Weber, 2010, 2012a), and Martha’s case is like so many others.

In the context of human–human interaction, the transition from surveillance to veillance represents a “fair” (French “Juste”) sight and, more generally, fair and balanced sensing.

But our society is embracing a new kind of entity, brought on by AI (Artificial Intelligence) and machine learning. Whether we consider an “AI” as a social entity/actor, e.g. through Actor Network Theory (Callon, 1999; Latour, 2005; Munro, 2009; Wood and Graham, 2006), or simply as a device to interact with, there arises the question: “Are smart things making us stupid?” (Morozov, 2013).

Past technologies were transparent, e.g. electronic valves (“vacuum tubes”) were typically housed in transparent glass envelopes, into which we could look to see all of their internals revealed. And early devices included schematic diagrams and parts lists—efforts by the manufacturer to help end users understand how their products worked.

In the present day of computer chips and closed-source software, manufacturers take extra effort not to help people understand how things work, but to conceal functionality: (1) for secrecy; and (2) because they (sometimes incorrectly) assume that their users do not want to be bothered by detail, i.e. that their users are looking for an abstraction and actually want “bothersome” details hidden (Burrell, 2016; Mann, 2016b).

At the same time these technologies are being more concealing and secretive, they are also being equipped with sensory capacity, so that (in the Actor Network Theory (ANT) sense) these devices are evolving toward knowing more about us while revealing less about themselves (i.e. toward surveillance).

Our inability to understand our technological world, in part through secrecy actions taken by manufacturers, and in part through a general apathy, leads to the use of modern devices through magic, witchcraft-like rituals rather than science (Lynn Kaarst-Brown and Robey, 1999). This technopaganism (Stivers and Stirk, 2001) leads people to strange rituals rather than trying to understand how things work. General wisdom from our experts tell us to “reboot” and try again, rather than understand what went wrong when something failed (Robertson, 2011). But this very act of doing the same thing (e.g. rebooting) over and over again, expecting a different result is the very definition of insanity:

“Insanity is doing the same thing, over and over again, but expecting different results.” (Narcotics Anonymous, 1981)

In this sense, not only do modern technologies drive us insane, they actually require us to be insane in order to function properly in the technopagan world that is being forced upon us by manufacturers who conceal its workings.

I propose ⁴ as a solution, a prosthetic apparatus that embodies the insanity for us, so that we don’t have to. I call this app “LUNATIC”. LUNATIC is a virtual personal assistant. The user places a request to LUNATIC and it then “tries the same thing over and over again …” on behalf of the user so that the user does not need to himself or herself become insane. For example, when downloading files, LUNATIC starts multiple downloads of the same file, repeatedly, and notifies the user when the result is obtained. LUNATIC determines the optimum number of simultaneous downloads. Typically this number works out to 2 or 3. A single download often stalls, and the second one often completes before the first. If too many downloads of the same file are initiated, the system slows down. So LUNATIC uses machine learning to detect slowed connections and makes a best guess as to the optimum number of times to repeat the same tasks over and over again. This number is called the “optimum insanity”, and is the level of insanity (number of repetitions) that leads to the most likely successful outcome.

At times the optimum insanity increases without bound, typically when websites or servers are unreliable or erratic. LUNATIC is not performing a denial of service attack, but, rather, a “demand for service”. A side effect is that when large numbers of people use LUNATIC, erratic websites will experience massive download traffic, such that LUNATIC disincentivises insanity.

In this sense, LUNATIC is a temporary solution to technopagan insanity, and ultimately will hopefully become unnecessary, as we transition to the age of Sousveillant Systems.

Sousveillant systems

Humanistic Intelligence is defined as intelligence that arises by having the human being in the feedback loop of a computational process (Minsky et al., 2013).

Sousveillant Systems are systems that are designed to facilitate Humanistic Intelligence by making their state variables observable. In this way machines are “watching” us while allowing us to also “watch” them. See detailed description in Mann (2016a), a figure from which is excerpted here (Figure 3).

Cyborg craft

Sousveillant Systems give rise to a new form of Human–Computer interaction in which a machine can function as a true extension of the human mind and body. Manfred Clynes coined the term “cyborg” to denote such an interaction (Clynes and Kline, 1960), his favorite example being a person riding a bicycle (Clynes, 1996, personal communication). A bicycle is a machine that responds to us, and we respond to it in an immediate way. Unlike modern computers where feedback is delayed, a bicycle provides immediate feedback of its state variables all which can be sensed by the user.

Consider the Computer Numerical Control (CNC) milling machine that extends our capacity to make things. We begin with Computer Aided Design (CAD) and draw something, then send it to the machine, and let the machine work at it. There’s not much real feedback happening here between the human and machine. The feedback is delayed by several minutes or several hours.

David Pye defines “craft” (“workmanship”) as that which constantly puts the work at risk (Pye, 1968). As such, modern CNC machine work is not craft in the Pye sense. Nor is anything designed on a computer in which there is an “undo” or “edit revision history” functionality.

Imagine a CNC machine that gave the kind of feedback a bicycle does. Could we take an intimate experience in craft, like a potter’s wheel, and make a CNC machine that works at that kind of continuous (“undigital”) feedback timescale?

To answer this question, we are developing Haptic Augmented Reality Computer Aided Design (HARCAD) to create a new kind of craft called “Cyborg Craft” (see Figures 4 and 5).

Figure 2.

Harbisson and Mann with passports depicting the physical reality of their bodies as partly computational, both examples of people who are part technological, through the use of camera-based computer vision as a seeing aid. The Veillance Divide (e.g. when surveillance is the only allowable veillance) renders such people under attack as “existential contraband”—contraband by their mere existence.

Figure 3.

Humans have senses and effectors at their informatic inputs and outputs. Machines have sensors and actuators at their informatic inputs and outputs. The signal flow paths that connect them are surveillance (when we’re being watched or sensed by the machine) and sousveillance (when we’re watching or sensing the machine). Humanistic Intelligence (HI) requires all six of these signal flow paths to be present (Minsky et al., 2013). When one of the six is blocked (most commonly, Sousveillance), we have a breakdown in the feedback loop that allows for a true synergy (“cyborg” state). To prevent this breakdown, Sousveillant Systems mandate Observability (Sousveillance).

Figure 4.

Haptics, Augmented Reality, and Computer Aided Design: A Tactile Sequential Wave Imprinting Machine (Mann, 2015, 1992) is formed from a linear actuator connected to an antenna moved in front of a radio transmitter (microwave motion sensor). The actuator is fed by a signal from a lock-in amplifier connected to the moving antenna plus another stationary antenna, so the user can grasp, touch, hold, and feel otherwise invisible electromagnetic radio waves. In an early embodiment (Mann, 1979), radio waves are picked up (or reflected) by the moving metal bar of a pen plotter, and the user grasps the pen to feel the radio waves. A light bulb is also attached to the pen so that the user can also see the radio waves, through Persistence-of-Exposure (PoE) of the human eye, or photographic film. In more modern embodiments, a linear actuator drives an LED light attached to the finger. By wrestling with this robotic device, the user and the device together trace out waves imported into the Autodesk Fusion 360 Cloud-Based 3D CAD Platform for instant collaboration with others. Together with the Metavision Augmented Reality glasses, multiple people use aluminum foil brushes to collaboratively sculpt and shape cloud-based metaveillance waveforms to design buildings, furniture, automobiles, or other curvaceous products.

Figure 5.

Wrestling with robots as a means for achieving Cyborg Craft: Robotic-inspired abakographic light painting (top row). Stephanie, age 10, wrestles with the robot while controlling the spinning of a Potterycraft wheel with her left foot.

Here we have a sensory synergy between human and machine, in which the feedback loop is essentially instantaneous.

Feedback delayed is feedback denied

Let us conclude with a hypothetical anecdote set in the future of musical instruments, which parallels the author’s personal experience with scientific instruments. This points to a possible dystopia not so much of government surveillance, but of machine inequiveillance.

The year is 2067. Ken is the world’s foremost concert pianist, bringing his own Steinway grand piano to each concert he performs in, now that concert halls no longer have real pianos. Software synthesis has advanced to the point where none of the audience members can hear the difference. Steinway stopped making real pianos in 2020. Yamaha and others also switched to digital-only production the following year.

Even Ken has trouble telling the difference, when someone else is playing, but when he plays a real piano himself, he can feel the difference in the vibrations. In essence, the performance of the Steinway Digital is as good as the original, but the vibrotactile user-interface is delayed. The tactuators installed in each key simulate the player’s feeling of a real piano, but there is a slight but noticeable delay that only the musician can feel. And user-interface is everything to a great musical performance.

Ken no longer has access to a real piano now that his Steinway grand piano was water-damaged by a roof leak while he was away last March. He tried to buy a new piano but could not find one. Tucker Music had one in their catalog, for $6,000,000, but when Ken called Jim Tucker, Jim said there were no more left. Jim sold about 50 of them at that price, over the past few years, as he collects and restores the world’s last remaining real pianos, but no more are coming up for sale.

Ken has felt that his musical performances have declined now that he no longer has access to a real piano. Software, AI, and machine learning make better music anyway, so there’s no longer a need for human musicians, anyway.

But there has been no great advancement in music in recent years, now that there are no longer any great musicians still passionate about music for music’s sake. Today’s musician spends most of the time writing grant proposals and configuring software license servers rather than playing music.

The need for antiques to obtain truth in science

The above story depicts a true event, except for a few small changes. My (not Ken’s) instrument that was damaged was not a musical instrument, but, rather, a scientific instrument called a Lock-In Amplifier (Cosens, 1934; Meade, 1982; Michels and Curtis, 1941; Stutt, 1949) made by Princeton Applied Research in the early 1960s. It was easy to understand and modify. I actually did some modifications and parts-swapping among several amplifiers to get some special capabilities for AR (augmented reality) veillance visualizations, such as bug-sweeping and being able to see sound waves, radio waves, and sense sensors and their capacity to sense (Mann, 2015).

The roof leak occurred in March 2016, while the amplifier was running (it takes a few hours to warm up, and since it uses very little electricity it is best to leave it running continuously).

The PAR124A is no longer available, and large organizations like research universities and government labs are hanging on to the few that remain in operation.

It should be a simple matter of purchasing a new amplifier, but none of the manufacturers are willing to make the modifications I require, nor are they willing to disclose their principles of operation to allow me to do so. Neither Princeton Applied Research, nor Stanford Research Systems (nor any other modern maker of lock-in amplifiers) is able to supply me with an instrument I can understand.

One company claims to have equaled the performance of the PAR124A, at a selling price of $17,000:

Since the invention of the lock-in amplifier, none has been more revered and trusted than the PAR124A by Princeton Applied Research. With over 50 years of experience, Signal Recovery (formerly Princeton Applied Research) introduces the 7124. The only lock-in amplifier that has an all analog front end separated, via fiber, from the DSP main unit.

Recent research findings, however, show that the PAR124A from the early 1960s still outperforms any scientific instrument currently manufactured (Wang et al., 2015).

And performance alone is not the only criterion. With a scientific instrument we must know the truth about the world around us. The instrument must function as a true extension of our mind and body, and hide nothing from us. Modern instruments conceal certain aspects of their functionality, thus requiring a certain degree of technopaganism (Stivers and Stirk, 2001) to operate.

Thus we currently live in a world where we can’t do good science without access to antiques.

Imagine a world in which there are no Steinway grand pianos anymore, a world bereft of quality, except old ones in need of restoration. A musician would have to be or hire a restorer and repair technician, and hope for access to one of the few working specimens that remain.

Is this the world we want to live in?

Science demands integrity

Only through Sousveillant Systems and “little data” we can preserve the tradition of science, in the face of technopaganism, surveillance, and “Big-only Data”. A goal of our research is to produce devices that embody sousveillance-by-design, starting with scientific test instruments like lock-in amplifiers, and progressing toward concepts like “little data”. To that end, let us hope that we can build sousveillance into our networked world, starting with instruments.

This commentary is a part of special theme on Veillance and Transparency. To see a full list of all articles in this special theme, please click here: http://bds.sagepub.com/content/veillance-and-transparency