Abstract
Ethics in data science and artificial intelligence have gained broader prominence in both scholarly and public discourse. Much of the scholarly engagements have often been based on perspectives of transparency, politics of representation, moral ethical norms, and refusal. In this article, while the authors agree that there is a problem with the universal model of technology, they argue that what these perspectives do not address is the postcolonial epistemology of the machine. Drawing from Mark Fisher’s science fiction capital, it is posited that data capitalism doesn’t rely on data as a given, but on what data can become; it operates in the future as much as the calculation of probabilities coincides with the predictive extraction of surplus value. The authors argue that in order to address ethical and sociopolitical concerns in artificial intelligence, technosocial systems must be understood in data capitalism. After discussing what they characterize as the three paradigms of prediction, the authors point toward the transformative potential of temporal structures and indeterminacies in automated self-regulating systems. They argue therefore that assumptions of technological determinism that are found in debates about the reproduction of biases in systems of predictive intelligence has nothing to do with the technical machine, but is rather the result of a continuous re-territorialization of the technosocial possibilities of re-inventing epistemological paradigms outside the framework of colonial capital.
If the current global pandemic condition has accelerated the use and dependence on digital technologies, then ethical and political questions of colonialist and capitalist interventions are even more paramount. Ethics in data science and artificial intelligence (AI) have gained broader prominence in both scholarly and public discourse. Much of the scholarly engagements have often been based on perspectives of transparency, politics of representation, moral ethical norms, and refusal. These interventions have often led to solutions that seek to co-opt or re-design in terms of repurposing technology, including the repurposing of technology as a means of resistance. Others have offered technological solutions for problems of transparency, representation, and moral ethical norms. 1 With each of these perspectives, we agree that there is a problem with the universal model of technology and we want to say something about this given the work we’ve been doing on data capitalism and predictive intelligence, as we argue that what these perspectives do not address is the postcolonial epistemology of the machine.
From this standpoint, critiques of automation that propose that machines can be programmed to do what is socially or politically progressive fall in to the risk of assuming that the algorithm is transparent and can be co-opted and repurposed, rather than being a sociotechnical system that can be redesigned. For us, critical design means that we are to diagnose the function of learning and not reduce algorithms to training.
Not only does transparency make white box assumptions about the algorithm but it also fundamentally leaves intact the universality of learning, namely the configuration of how to learn and thus how to know universally determined by a particular epistemology. More specifically, the claim for transparency risks falling back into one model of knowledge, based on the given specific articulation of a universal fairness. If you train the algorithm to be fair and fair for everyone, this relies on communities to argue that every community needs to have access and fair representation, but in fact what will be imposed is merely another universal particular, a know-how that comes from specific views of the world. By arguing that we need to make the algorithm fair for everyone, we are also posing another reiteration of identity politics and not a challenge of epistemology. This claim therefore suits precisely the model of universal reason based on the self-positing judgment that distinguishes transparency from opacity. We claim that this is precisely what data capitalism desires.
Drawing from Mark Fisher’s (2009) science fiction capital, we posit that data capitalism doesn’t rely on data as a given, but on what data can become; it operates in the future as much as the calculation of probabilities coincides with the predictive extraction of surplus value. There is no given proof of one truth in data capitalism, its operations rather require the search mode of induction, namely the operations of trial and error. Here, data is completely malleable. Because of this heuristic model that data capitalism has, its ontology is based not on universal forms, but is open to process. From this standpoint, it is not possible to talk about computational empiricism insofar as the operations of induction merely anticipate and affectively shape future events. As in Brian Massumi’s (2015) ontopower, process can be understood both in terms of process of becoming and epistemological processing of information. Here, ontopower entails a potentiation of value from and through machines, which are there to grant the recursive reconfiguration of the transparency principle, namely the project of the self-determining subject to hide its over-determining subject position (da Silva, 2007). Transparency is therefore a strategy for preserving the post-Enlightenment subject as this continues to impart the hierarchies of gender, race, and class onto a conception of the human that has come to haunt the machine. From this standpoint, what is claimed to be opaque in the black box of machine learning algorithms cannot be disentangled from the normative apparatus that reproduces the transparency of the self-determining subject. As cybernetic networks impose on the social body an infrastructure of seamless communication based on the equivalence of connections, the heuristic medium of governance withdraws into the background, out of sight in the recursive colonialism of the transparent subject. Thus, it is the transparent subject that goes without being challenged in the general debates on AI ethics that push for transparency or the repurposing of technology.
What is disregarded in the dominant discourse in AI ethics is what the “learning” is of machine learning. We want to make a few remarks on predictive intelligence, but first want to turn to Sylvia Wynter to discuss the importance of what can be called “sociogenic prediction.” Far beyond the algorithmic mis-recognition of skin color, the machine learning of the flesh inherits Western histories of Man that enter the constitution of new assemblages in a system of sociopolitical relations. As first coined by Fanon (1967), the sociogenic principle is a concept that Sylvia Wynter (2001, 2007) further developed as a way to account for how the sociopolitical becomes flesh. For Wynter, the sociogenic principle is an ontological account of how the sociopolitical assemblages of Man and the logic of symbolic “difference” become programmed in the body resulting into an ontogenic formation of identity that come to brand the flesh. This sociopolitical assemblage of Man, what Wynter also calls Western Man, has gone through a process of auto-determinations derived from the cosmogonies of human origin. Wynter argues that the current iteration of cosmogony corresponds to a biohumanist homo oeconomicus, constituted by the economic theories of Adam Smith. Here, the correlation between biological and economic survival, through the forces of selection and optimization of survival, defines the epistemological explanation of who is and who is not successful as a species. It is this correlation that consolidates the formation of the sociogenic code and ensures the reproduction of the racialization of the world. This also corresponds to what could be called the sociopolitical constitution of Man, as a fictive (and yet dominant) genealogy that tells the story of being human. For Wynter understands the reproduction of racialization in terms of autopoetic and self-regulatory practices that are imprinted within the flesh and as such enable the ontogenic self-replication of this originary myth. By drawing from neurobiology, Wynter explains how symbolic “difference” materializes in terms of ontologies via neurochemical processes that produce a racialized e/affect, making the materiality of “difference” seem natural and thus granting a monolithic explanation of the human. However, it is our argument that the autopoetic institution of the sociogenic code permeates not just human ontologies but also more-than-human ontologies including the sociotechnical assemblages of data and algorithms (Dixon-Román, 2016).
The sociogenic coding of the other as the negative marker, we claim, is necessary to the recursive loops of the colonial enterprise, whereby the naturalization of the dyadic structure of equivalence between man and the world (self and other) ensures that all remains the same under the Western sun. From this standpoint, it seems not sufficient to claim that the transparency of the self-determining subject must be unveiled by demanding more transparency from the system, and for instance asking to recode a machine learning program in the name of an equality of representation. In other words, the demand for enlarging the normalized category of the human to include excluded differences and shed light on the blindness of the machine does not seamlessly ensure a political overturning of the dyadic pattern of self-recognition.
It is already evident therefore that if ANN (artificial neural networks) will be trained to recognize non-Caucasian features and skin colors, it will do so only by learning to extend the sociogenic commitment to the evolutionary ground of the biological man into the smooth machines of a technical strata. To put it in another way, as it stands, the cybernetic regime of immediate communication allows no possibility of breaking away from what Sylvia Wynter calls the autopoetic self-determination of Man, predicated upon the negative side of the color line (Wynter 12). The predictive intelligence of machines here becomes a sociotechnical assemblage that contains within itself the seeds of an ontological re-origination of a “speciated genre or Mask of being human” (13).
To further explicate Wynter’s sociogenic principle in relation to prediction, we re-work the late statistician Leo Breiman’s paradigms of prediction. In Breiman’s articulation of the cultures of statistics, he talks about two paradigms: data modeling and algorithmic modeling. We agree with this distinction, and yet we aim to add another paradigm, what we are calling “computation modeling.” We argue the main distinction between each of these paradigms is how error, noise, indeterminacy, or the incomputable is accounted for.
For instance, data modeling is characterized by the fitting of parametric statistical models (e.g. logistic or linear regression) to a sample of data of the population. The models are evaluated based on an analysis of the model error, residual variance, and goodness of fit (e.g. R-squared). As an example, data modeling might be used in the context of predicting the risk of someone committing a violent offense and thus falls onto the statistical strategies of predictive policing. Here, the sociogenic has multiple pathways on algorithmic institution, including the operationalization of the variables, what predictors are used, and what and who comprises the sample for parameterizing the model. These are all discursive formations of sociopolitical significance.
If data modeling is based on a known specified parametric model that is fit to the data, then algorithmic modeling is based on an unknown model (i.e. black box) that is determined by the data. Algorithmic modeling is understood to include nonparametric, nonlinear models (e.g. Support Vector Machines, Random Forest, or Neural Nets) that are designed to optimize predictive accuracy; thus, the algorithm is determined by the data, which is not a priori to data processing. Models are evaluated based on prediction accuracy and, as such, an analysis of the error in prediction. Apart from Breiman, our characterization of algorithmic modeling is based on nonautomated processes of model updating. Our contention is that Breiman’s conception of algorithmic modeling is not enough to account for the more profound historical transformation of automation itself, what we characterize as computational modeling below. Applying the above predictive policing example to algorithmic modeling results in other patterns of sociogenic violence. With this paradigm of prediction, the sociogenic institution of the algorithm entails how the response variable is operationalized as well as algorithm programming decisions of cost ratio of false negatives to false positives, the determined patterns learned from the data assemblages, and the performative enactments from feedback loops.
While algorithmic modeling is determined by the data and designed to make more accurate predictions than data modeling, the process of handling indeterminacy or the incomputable is distinct from what we are calling computational modeling. Similarly, while the type of algorithms used in computational modeling do not necessarily differ from those used in algorithmic modeling, there is a minor yet significant difference in deployment that we argue has substantial implications for algorithmic reason and the sociogenic. When the process of algorithmic re-estimation (i.e. learning) is automated, and therefore is left to run the information according to the capacity of the output to overwrite the input, introduces temporality in the algorithmic procedure of re-estimation. Temporality, for instance, is at the core of an automated prediction model that must account for the nonlinear and recursive loops between inputs and outputs. The limit of the finite algorithmic model to account for infinities already sets up the conditions for a reflexive function in the algorithm. The automation of this process enables the prehension of incomputables and forms of thought that are immanent to the algorithm (Parisi, 2013). While the nonautomated process of model re-estimation accounts for the incomputable, it does it in a way that simply adds new data to the existing training data to estimate the algorithmic model. Instead, at the core of computational modeling, there is an ongoing iterable process that continues to build the model based on algorithmic randomness without returning to the original condition.
Employing computational modeling to discuss the predictive policing example will point to similar pathways that concern sociogenic violence, but with a force of alterity. Similar to algorithmic modeling, the sociogenic in-formations of the algorithm via the computational modeling paradigm of prediction is similar: the operationalizing of the response variable, algorithm programming decisions of cost ratio of false negatives to false positives, the determined patterns learned from the data assemblages, and the performative enactments from feedback loops. Under computational modeling, data assemblages and feedback loops are most determinative, as the automated learning of the algorithm is determined from patterns in the data and the incomputable. Thus, while algorithmic modeling engages in performative acts of prediction based on a model that was calibrated at one point in time, computational modeling will continue to automatically re-calibrate estimated parameters based on the processing of new data. This means that the sociogenic constitution of the algorithm is likely to shift or reconfigure as patterns of policing and public behaviors change. While the sociogenic may have similar pathways for algorithmic constitution, it is the incomputable and its prehension over time that creates conditions for alternative configurations of algorithmic thought (Parisi, 2013).
Incomputable probabilities are discrete states of nondenumerable infinities that are incompressible to the finite systems of algorithms. It is the halting probability of a universal free-prefix self-delimiting Turing machine, what mathematician and information theorist Gregory Chaitin (2005) calls Omega. Its binary expansion is an algorithmic random sequence, which is incomputable. The incomputable is that which is not outside the system, it is fundamentally a part of the system. It indicates that the system is an incomplete model of cognition.
Hence, computational modeling is used not simply to build profiles based on pre-fixed sets of algorithms, but to exploit the self-delimiting power of computation, defined by its in/capacity to decide when a program should stop. By transforming nondenumerable infinities into random discrete sets or Omega probabilities, computational modeling manifests random actualities.
When automated procedures become temporal operators of variations and heuristic searchers of results, automation itself—that is the computational procedure—becomes open to the indeterminacy of its own function. In particular, algorithmic iterations, it has been argued, have become opened to the circular looping of time. If we are to draw on more challenging conceptions of automation, inspired for instance by Gilbert Simondon’s (2017) general theorization of the modes of existence of technical objects, our argument for a dynamic view of automation can suggest that temporal processing in automation radically challenges the reproduction of the sociogenic principle in systems of prediction. Simondon (2017) insisted that machine design includes a principle of indetermination, which is to be added to the space of indeterminacy in the human–machine relation. In particular, as Yuk Hui (2019) has recently suggested, there is a possibility of approaching this question of the inorganic time of the machine in terms of recursivity. It is precisely this link between the inorganic time and the inorganic thought of the machine that can allow us to discuss machine thinking away from either an optimization of mechanical functions or simply as an extension of the soul of man.
In particular, Hui (2019) takes Simondon’s proposition of a non-Cartesian form of cognition to challenge the assumption that thinking follows a linear chain of causes and effects, namely where reasoning is confined to a procedure for transporting evidence from one point to another without having any active function to rather change the course of things. According to Hui (2019), Simondon refuses Descartes rationalism by demonstrating that the cybernetic principle of feedback adds a new temporal structure to thinking that is described in terms of a spiral. As Hui (2019) further explains, according to Simondon cybernetics replaces the telos of thought with a self-regulatory process. In particular, insofar as the recursivity of feedback makes the cybernetic system possible, it also impedes the system to become systematic, complete, and simply a reproductive whole. However, since human relations are abstracted and re-integrated into the temporality of machines, which, as we have suggested so far, constitute the engine of algorithmic governmentality, the question of temporality—and thus of recursive temporality in nonorganic machines—is still in need to be further explored.
For Hui (2019), margins of indeterminacy not only describe the recursive temporalities of machines, but more importantly a recursive thinking in machines. This remark suggests that the technical machine is not simply a mirror of the normative apparatus of knowledge reproduction, automation, in terms of the temporal processing of outputs, it can include both contingency and chance within itself because the temporality of the technical object or cybernetic machines precisely admits that errors, incident, and failure are part of the causal process of machine learning. From this standpoint, it is important to transform the conception of the algorithm itself as being not simply pre-determined by its default or setting position. As much as the recursivity of the system allows for a change over time, so too it points to the indeterminate elaboration of an “algorithm without programs.”
If, as we have seen, recursive feedback is at the core of the re-estimation model used in statistics to predict outcomes on the basis of a given data set or of a training data set, then it is important to further re-envision how recursivity works through infinities, that is how what cannot be known in advance becomes a problem of compression or patterning of infinities. From this standpoint, one can argue that the science of statistics can be taken as a techno-scientific instance of epistemological reconfigurations of the problem of the incomputable in the formation of feedback systems of predictive intelligence. What we are arguing therefore is that assumptions of technological determinism that we find in debates about the reproduction of biases in systems of predictive intelligence has nothing to do with the technical machine, but is rather the result of a continuous re-territorialization of the techno-social possibilities of re-inventing epistemological paradigms outside the framework of colonial capital. From this standpoint, data capitalism takes recursive accumulation in statistics to preserve human capital in machines so that feedback procedures of estimation remain anchored to the reproduction of social relations, embedded in the material-historical determination of man. In other words, what is deterministic in statistical procedures of estimation is not the cybernetic principles of feedback or computational patterning, but above all the matrix of representation, where the sociogenic principles are amplified and distributed through the image of the automata that must maintain an objective efficiency.
From this standpoint, incomputables are not simply open to be co-opted because they already constitute or structure the system to define a new order of cosmogony admitting the indeterminacy of knowledge as an inevitable consequence of computational know-hows. This proposition is only one possibility for continuing to question—debunk and construct anew—the implications of techno-scientific epistemology in matters of governance, one that points toward a necessary transformative force or the counter-futures of AI ethics.