In the Image of Injustice: A Critical Examination of AI,Surveillance,and Systemic Racism in the United States

Abstract

Artificial intelligence (AI) is rapidly being integrated into the United States criminal justice system under the promise of efficiency and objectivity. This article challenges that narrative, arguing that AI-powered surveillance technologies, particularly facial recognition and predictive policing, function as new vectors for systemic racism. Through a critical examination of federal studies, seminal journalistic investigations, and recent documented cases of wrongful arrest, this paper demonstrates how these systems are not merely biased but serve to automate and scale discriminatory practices. Landmark studies, such as those by the National Institute of Standards and Technology (NIST), reveal that facial recognition algorithms exhibit significantly higher error rates for racial minorities. Furthermore, case studies of predictive tools like COMPAS show a clear racial disparity in risk assessments. The analysis concludes that these technologies create a dangerous feedback loop: biased data from historically over-policed communities trains algorithms to identify those same communities as high-risk, thereby justifying and intensifying their surveillance. This process launders historical prejudice through a seemingly neutral technological interface, entrenching racial inequality and posing a profound threat to justice.

Introduction

The proliferation of artificial intelligence (AI) within the American criminal justice system marks a pivotal moment in the history of law enforcement and social control. Proponents champion these technologies, from predictive policing algorithms that forecast crime hotspots to facial recognition software that identifies suspects in a crowd, as objective, efficient tools capable of transcending human bias (Police Executive Research Forum, 2024). This narrative of technological neutrality suggests that data-driven systems can correct for the prejudices that have historically plagued policing. However, a growing body of evidence presents a far more troubling reality: instead of eliminating bias, these systems often inherit, amplify, and conceal it, creating new and formidable mechanisms for the perpetuation of systemic racism.

This article provides a critical examination of this phenomenon. It argues that the deployment of AI surveillance in the United States does not represent a break from historical patterns of racial discrimination but rather an evolution of them. The analysis proceeds through an examination of two primary case studies: facial recognition technology and predictive risk algorithms. These cases were selected due to their widespread adoption across United States jurisdictions, their documented influence on policing policy, and the existence of robust empirical audits demonstrating their disparate impact. By analyzing the technical mechanics and real-world impacts of these systems, this paper demonstrates how historical injustices are encoded into the algorithms that now police communities. This process creates a pernicious feedback loop where biased data leads to biased enforcement, which in turn generates more biased data, laundering discrimination through a seemingly objective technological process (Eubanks, 2018).

The Technological Manifestation of Systemic Racism

Systemic racism refers to the ways in which societal structures, institutional practices, and cultural norms combine to create and perpetuate racial inequality, often without overt individual prejudice (Feagin, 2013). Within the justice system, this phenomenon manifests as racial disparities in policing patterns, arrest rates, and sentencing outcomes that are the legacy of historical policies and practices (Alexander, 2010). The introduction of AI into this environment is not a neutral act. Algorithmic systems are trained on historical data, and if that data reflects decades of biased policing, the resulting algorithm will inevitably learn to replicate those biases (O’Neil, 2016).

This dynamic gives rise to what scholars have termed “technological redlining” or the “New Jim Code,” where technology is used to reinforce social hierarchies and discriminatory practices (Benjamin, 2019; Noble, 2018). Unlike the overt racism of the past, algorithmic bias is often invisible to the public and even to the officers using the technology. The decision-making process is obscured within a “black box,” making it incredibly difficult to challenge or scrutinize. This opacity grants the system a veneer of scientific authority that can mask deeply inequitable outcomes, making the resulting discrimination seem not like a product of prejudice, but of objective fact.

Facial Recognition: A Case of Encoded Bias

Facial recognition technology is one of the most widely adopted and controversial AI tools in modern policing. It operates by comparing a “probe” image (e.g., from a surveillance camera) against a database of known faces (e.g., mugshots). The promise is one of rapid and accurate suspect identification. The reality, however, is one of deeply embedded racial bias and operational failure.

The Evidence of Disparity

A landmark 2019 study by the National Institute of Standards and Technology (NIST), a United States government agency, provided definitive evidence of this bias. Testing 189 algorithms from 99 different developers, the study found that the vast majority exhibited significant demographic differentials. Across one-to-one matching algorithms (used to verify a photo against a specific identity), false positive rates were highest in West and East African and East Asian people, and lowest in Eastern European individuals. The study found that top-performing algorithms falsely identified African American and Asian faces 10 to 100 times more often than Caucasian faces (Grother et al., 2019).

While NIST has continued to evaluate these systems, noting improvements in pure algorithmic accuracy in controlled settings, the reaction from law enforcement has been mixed. Some jurisdictions imposed moratoriums based on these findings, yet many agencies have continued deployment, citing vendor claims that newer versions have resolved these disparities. However, reliance on “lab-tested” accuracy masks a critical flaw: the operational environment of policing rarely matches the pristine conditions of a laboratory.

The Human Cost of Algorithmic Error

Operational failures have led to multiple documented cases of wrongful arrests of Black individuals, proving that improvements in code have not translated to the protection of civil liberties. In 2020, Robert Williams was arrested in Detroit after a facial recognition system incorrectly matched his driver's license photo to a grainy surveillance image of a shoplifter (Hill, 2020).

More recent cases demonstrate that these errors are not a thing of the past. In 2023, Randal Quran Reid was arrested in Georgia for a theft in Louisiana, a state he had never visited. A facial recognition match generated a warrant, and despite visible physical differences, officers relied on the algorithm's output (Negussie, 2023). Similarly, in 2023, Porcha Woodruff, a woman eight months pregnant, was arrested in Detroit for carjacking. The facial recognition software matched her to a 2015 mugshot, failing to account for the passage of time or her pregnancy (Hill, 2023).

These cases highlight a critical danger: “automation bias,” where human operators place an undue level of trust in a technology's output. Officers frequently treat a probabilistic “match” as probable cause for arrest, bypassing traditional investigative work (Stark, 2019). Even if an algorithm is 99% accurate, the remaining 1% of error falls disproportionately on Black citizens, and when combined with automation bias, the results are catastrophic.

Predictive Policing and Recidivism Risk: Automating Suspicion

Beyond identifying individuals, AI is also used to predict behavior and allocate police resources. Predictive policing systems analyze historical crime data to forecast where and when future crimes are likely to occur, while recidivism risk algorithms assess an individual's likelihood of reoffending. Both systems have been shown to reproduce and amplify racial bias.

Predicting Crime or Predicting Policing?

Systems like PredPol (now Geolitica) and ShotSpotter are designed to direct police patrols to high-risk areas. The logic appears sound: send officers where crime is most prevalent. However, the data these systems rely on is not a pure measure of criminal activity; it is a measure of where police have made arrests in the past. Given the well-documented history of disproportionate policing in minority communities, these algorithms inevitably learn that these neighborhoods are “high-risk” (Lum & Isaac, 2016). The result is a self-fulfilling prophecy: The algorithm sends more police to a neighborhood, who then make more arrests for low-level offenses, which generates more data confirming the neighborhood as high-risk. Such feedback loops justify over-policing while leaving crime in other areas unaddressed.

The COMPAS Controversy

Perhaps the most scrutinized risk-assessment tool is the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS). Used in courtrooms across the country to inform decisions on bail and sentencing, the algorithm assigns defendants a “risk score.” A 2016 investigation by ProPublica analyzed the COMPAS scores of over 7,000 people arrested in Broward County, Florida. Their findings were stark: the algorithm was demonstrably biased against Black defendants. Black defendants were almost twice as likely as White defendants to be wrongly labeled as having a higher risk of reoffending. Conversely, White defendants were mislabeled as low-risk more often than Black defendants (Angwin et al., 2016).

Although developers argue that these tools are “calibrated” (meaning a score of 7 represents the same risk for Black and White defendants), this mathematical parity ignores the disparate impact of False Positive rates. Recent scholarship continues to support the ProPublica findings, suggesting that structural factors such as housing instability and employment history, proxies for race in a stratified society, feed into these risk scores, ensuring that the algorithm penalizes poverty under the guise of predicting criminality.

The Limits of Algorithmic Fairness

In response to these critiques, a field of “algorithmic fairness” has emerged, proposing technical solutions to “de-bias” these systems. These methods generally fall into three categories: pre-processing (scrubbing data of sensitive attributes like race), in-processing (adding fairness constraints to the algorithm's learning function), and post-processing (adjusting thresholds to equalize outcomes across groups).

While these techniques show theoretical promise, they face significant limitations in practice. First, “scrubbing” race is often futile because race is highly correlated with other data points like zip code, income, and education level, redundant encodings that allow the algorithm to reconstruct racial bias. Second, there is a fundamental “fairness-accuracy trade-off”; modifying an algorithm to be more “fair” often reduces its overall predictive accuracy, leading policymakers to reject the “fairer” models.

Moreover, the argument that algorithms are preferable to “biased human judges” creates a false dichotomy. While individual judges certainly harbor biases, those biases are individual, variable, and subject to appellate review. Algorithmic bias, by contrast, is systemic, scalable, and opaque. When a flawed algorithm is deployed, it does not discriminate against one defendant; it discriminates against thousands, instantly and invisibly. Technical tweaks cannot solve the fundamental problem: if the data reflects a racist world, the algorithm will produce racist outputs.

Discussion: The New Jim Code in Practice

The cases of facial recognition and predictive algorithms demonstrate how AI systems function as the “New Jim Code” (Benjamin, 2019). They do not need to be programmed with explicit racial bias to produce racist outcomes. By training on data that is itself the product of a racially inequitable society, these systems learn to equate blackness with criminality, effectively automating the racial hierarchy.

The Opacity of the Black Box

A central feature of this new regime is the “black box” problem. Modern AI, particularly deep learning models, operates through multi-layered neural networks that identify patterns too complex for human interpretation. The decision-making logic is not explicitly coded but “learned,” meaning even the developers often cannot explain why a specific decision was made. While proponents point to explainable AI (XAI) as a solution, tools designed to offer post-hoc explanations for algorithmic decisions, these explanations are often simplifications that lack the fidelity required for due process. In a courtroom, a defendant has the right to confront their accuser; when the accuser is a black-box algorithm protected by trade secrecy laws, that right is effectively nullified (Shah, 2018).

This shifts the locus of discrimination from the individual officer's discretion, which can be observed and challenged, to the opaque inner workings of an algorithm, which cannot. When a person is wrongly arrested because of a facial recognition match, the injustice is framed as a “glitch” in the machine rather than a systemic failure. This technological veneer insulates the justice system from accountability and makes reform more difficult.

Conclusion and Policy Implications

The uncritical adoption of AI in the United States criminal justice system poses a grave threat to civil liberties and racial justice. The promise of objective, unbiased policing has not materialized. Instead, these technologies have created a new, technologically sophisticated architecture for the perpetuation of systemic racism. They automate suspicion, amplify historical biases, and operate with a lack of transparency that undermines accountability.

Addressing this challenge requires a fundamental shift in how these technologies are governed. Simply attempting to “de-bias” the algorithms is insufficient, as the data itself is a reflection of a biased world. A more robust response is necessary, one centered on democratic control and the prioritization of civil rights. This should include several key policy actions.

First, there must be a federal moratorium on the use of facial recognition technology by law enforcement until its accuracy and racial fairness can be guaranteed. Several cities, including San Francisco, Oakland, and Boston, have previously taken this step, serving as models for responsible governance. Second, there must be mandatory transparency and independent auditing for any predictive algorithm used in the justice system. The public has a right to know how these systems work and to challenge their conclusions. Finally, the focus must shift from prediction and punishment to specific investment in the communities that are currently the targets of this new wave of surveillance. Resources currently spent on expensive surveillance licensing should be redirected toward community-led violence interruption programs and housing stability initiatives, addressing the root causes of crime rather than automating its punishment.

Without these safeguards, the United States risks building a future where systemic racism is not only preserved but made more efficient, more pervasive, and harder to fight than ever before. Justice cannot be automated. It must be actively pursued through fair practices, accountable institutions, and a commitment to equality that no algorithm can replace.