Abstract
People frequently compare the racial composition of stopped individuals with the racial composition of the local residential population to assess unequal policing. This type of evaluation rests on the assumption that the census-derived population accurately reflects the population at risk to be stopped. For vehicle stops, existing research indicates that this assumption is very problematic, resulting in highly unreliable assessments of black-white policing disparities. However, there is little research on the significance of this assumption for stopped urban pedestrians. Analyzing more than 100,000 investigatory stops in Chicago, the present study finds that similar to vehicle stops, most pedestrian investigations do not involve neighborhood residents, and estimates of racial disproportionality are inflated when this issue is ignored. Still, the degree to which estimates are inflated appears less than that previously reported for vehicle stops, and sizable racial disparities remain unexplained after the issue is taken into account. Implications for future research are discussed.
Police departments across the United States now collect a variety of data related to investigatory detentions of vehicle occupants and pedestrians. Often connected to litigation and consent decrees, these data have been used to inform significant policy changes in local policing, especially those aimed at curbing racial profiling. Focusing on vehicle stops, early court cases established that it was inappropriate to assess the degree of racial profiling by simply comparing the racial composition of stopped individuals to the racial composition of the local residential population (State v. Soto 1996). Among other issues, such as potential racial differences in driving behavior, this conclusion was based on two easily imagined and empirically established points: (1) not everyone living in an area drives in it, and (2) the existence of highways as well as the travel capabilities of automobiles make it extremely unlikely that drivers will be exclusively or even mostly stopped in their home areas.
Although analogous points can be made about mass transit and the ebb and flow of urban pedestrian populations, these arguments have not been subject to the same empirical verification efforts. Moreover, simply referencing established findings for vehicle stops would be unconvincing to those who view the context of pedestrian stops as very different. For example, although not everyone living in an urban neighborhood drives, it could be argued that everyone living in an urban neighborhood could reasonably be seen as a pedestrian. Furthermore, it could be contended that unlike the setting for highway stops, many urban neighborhoods are characterized by high “walk scores” because the things residents need or want are available nearby.
Thus, unsurprisingly, although the practice has been mostly abandoned for vehicle stop analyses, it is still common to find examples of pedestrian stop research using residence-based neighborhood population data as the benchmark (i.e., the assumed total population at risk to be stopped). Although scholars, journalists, and lawyers using this benchmark denominator undoubtedly recognize that the measure is imprecise, unlike for the vehicle stop research literature, there is a dearth of evidence regarding the degree of inaccuracy. Lacking empirical guidance, analysts are forced to rely on their reasoned impressions of urban living and, perhaps, wishful thinking. That is, given the widespread availability of local residential population data, it would simply be enormously helpful if researchers could rely on such data to give an approximate baseline standard of the expected composition of stopped individuals absent bias in police decision making.
Earlier scholarship on vehicle stop benchmarking has emphasized how improperly matching the numerator (characteristics of people stopped) to the denominator (people available to be stopped) can lead to erroneous conclusions (Fridell 2004). For example, Walker (2001) explained,
If nonresidents passing through the area violate traffic laws at a higher rate than residents, police officers are justified in making more traffic stops of nonresidents. And if minorities constitute a disproportionate share of the nonresident drivers, and in turn a disproportionate share of the nonresident traffic law violators, then the police are justified in stopping them at a higher rate than for white drivers. The full impact of nonresident drivers and police response to those drivers is not known. (pp. 79–80)
Examining data for 27 counties in Pennsylvania, Tillyer, Engel, and Wooldredge (2008) demonstrated that the method of comparing all stops with the residential census population produced vastly different estimates of racial disproportionality than a method that matched detainees who were county residents to the county resident denominator. Although Tillyer et al. also observed notable differences in estimates using other benchmarks (e.g., daytime traffic observations), the method of comparing all stops with the residential census population exhibited the greatest dissimilarity. For instance, the overall census benchmark suggested that on average, black motorists were 11.85 times more likely to be stopped than white motorists, whereas when the benchmark was constructed using only detainees who were actually residents, black motorists were only 1.47 times more likely. Tillyer et al. suggested that dramatic differences in estimates were most likely to occur in counties with busy thoroughfares and where the minority residential population was relatively small.
The potential for there to be greater amounts of measurement error in areas where minorities make up smaller portions of the population is particularly worrisome, as these are exactly the places on which analysts typically focus when investigating racial disproportionality in pedestrian stops (see, e.g., Rudovsky et al. 2016). From a theory perspective, this focus makes sense given criminological work on what police officers perceive to be “out of place,” as well as sociological scholarship emphasizing that racial threat effects will be most visible in defended white spaces (Anderson 2015; Carroll and Gonzalez 2014; Harris 2002; Tomaskovic-Devey and Warren 2009). This focus might also make sense in terms of plain math, depending on the point one is trying to emphasize: the smaller the relative size of the minority population, the greater the potential for higher disproportionality values. In the end, it is important to evaluate not only the average amount of benchmark imprecision but also how this inaccuracy may vary on the basis of the racial composition of residents in an area.
Along these lines, the present research offers empirical evidence regarding how estimates of racial disproportionality in pedestrian stops might differ across areas depending on both the prevalence of minority residents and whether the benchmark involves limiting the numerator to stopped local residents to match the denominator. More specifically, analyzing a uniquely detailed data set of investigatory detentions for the city of Chicago, this study addresses the following questions: (1) To what degree are detained urban pedestrians stopped in the areas where they live, and how does this vary from place to place? and (2) How are racial disproportionately indices and ratios affected by this variation, especially in contexts in which the black residential population is relatively small? It is important to note that estimating the extent to which racial differences in pedestrian behavior explain racial differences in police contact is beyond the scope of the present study. The data do offer some opportunities worth pursuing in this regard, and these are discussed for future research to consider.
Data and Methods
The present analyses use publicly available data from the Chicago Police Department (Chicago Police Department 2018). These data cover investigatory stops from February 29, 2016, to January 16, 2018 (total n = 174,292, with 118,196 of these not involving a vehicle). The data are based on officially approved investigatory stop report forms that explicitly exclude voluntary mere encounters and probable-cause stops (except in cases in which no other documentation adequately captures the reason for the detention). Only about 12 percent of these pedestrian investigations resulted in arrests. Although comparable in many respects with the publicly available and frequently analyzed Stop-Question-and-Frisk database for New York City (see, e.g., Gelman, Fagan, and Kiss 2007; White and Fradella 2016), the Chicago data set contains a number of unique variables. Especially relevant for the present research, the Chicago data set is the only publicly available source demarcating the police districts and beats where suspects reside in addition to the districts and beats where suspects were stopped.
Similar to New York City precincts, police district boundaries in Chicago average about 10 square miles of land area and typically incorporate residential populations greater than 100,000. Currently, Chicago delineates 22 districts. Much closer to the concept of a neighborhood, police beats are typically less than 1 square mile, with residential populations less than 10,000; there are currently 274 beats within Chicago city limits. Both police district and police beat boundaries were reformulated in 2012 to address contemporary community needs, including recent crime trends.
Census block group demographic data from the American Community Survey (2012–2016) were matched to policing area boundaries using coordinates for population centroids. Although this is standard procedure when using noncensus geographies, the matching process is imperfect. Although all 22 police districts included some residential areas, several police beats were nonresidential and could not be matched to the demographic data (particularly in the 1st District and 16th District). Additionally, because the American Community Survey data are derived using complex sampling techniques, there is uncertainty around the demographic counts and the margin of error can be especially significant for smaller geographic units. The American Community Survey for Chicago estimated a citywide residential population of approximately 2.7 million with 29 percent Latinx, 33 percent non-Latinx white, 30 percent non-Latinx black, and 6 percent non-Latinx Asian (on the basis of self-identification). The present study’s other data source, the Chicago Police Department’s Investigatory Stop Report database, indicated that the citywide racial/ethnic composition of pedestrian detainees was 16 percent Latinx, 9 percent non-Latinx white, 74 percent non-Latinx black, and 1 percent other (on the basis of officer perception).
To gauge the importance of residential status for the evaluation of racial disparities in investigatory stops, comparisons are made between disproportionality calculations when all detainees are included in the computation versus when only residents are in the estimate. Put differently, the present study assesses the degree to which disproportionality estimates are robust to violating one of the rules of benchmarking. As Fridell (2004) explained in her popular guidebook,
The recommendation that researchers compare against census data only data on residents stopped by police represents the rule of benchmarking called “matching the numerator and the denominator.” . . . The “numerator” refers to the data collected on stops made by the police. The “denominator” is the benchmark information. . . . in this case the benchmark is U.S. Census data, and it encompasses only jurisdiction residents. Therefore, the researcher must impose the same parameter on the numerator by including in the analyses only the police stops that were of residents. (p. 83)
Although previous research has used this type of focused benchmarking for vehicle stops, the potential confounding impact of nonresidents in the numerator has largely been ignored in pedestrian stop research on racial disparities.
Disproportionality indices describe the disparity between the “actual” and “expected” rates of stops by dividing the racial composition of detainees in an area by the benchmarked composition. These are constructed separately for (1) all stops compared with the residential population benchmark and (2) only stops of residents compared with the residential population benchmark (Tillyer et al. 2008). A disproportionality index value greater than 1.0 indicates that the rate of stops is greater than expected in comparison with the benchmark, but it does not provide a specific comparison between the rates of, for example, black pedestrians stopped relative to whites. A disproportionality ratio is needed for this purpose; this can be calculated by dividing the disproportionality index for black pedestrians by the disproportionality index for white pedestrians. In addition to summarizing general differences in disproportionality when using all versus only resident detainees, the present study assesses where discrepancies are systematically higher.
This is potentially important, as the strategy of collecting larger sample sizes might help reduce the influence of random error, but it will not mitigate systematic biases. Because of the smaller sample sizes of stops for creating disproportionality indices and ratios at the police beat level, the share of data noise to data signal will likely be greater. Robust regression techniques, particularly iteratively reweighted least squares, are thus used to help ensure that observed patterns in the data are meaningful.
Results
To What Degree Are Pedestrians Stopped in the Areas Where They Reside?
Figure 1 summarizes the percentage of investigatory stops taking place in the detainees’ home police districts or beats. As expected, this percentage varies depending on the type of investigatory stop and whether the home area is defined using a larger or smaller geographic scope. On average, vehicle stops were less likely to involve residents than pedestrian stops, although the difference between the two types of stops was perhaps not as large as some might intuit on the basis of common perceptions of urban neighborhood walkability and insularity. Moreover, the stop type difference was not as great as the difference associated with the geographic size of the area: about twice the percentage of detainees were residents at the district level (typically >10 square miles) than at the beat level (typically <1 square mile).
Percentage of Chicago investigatory stops taking place in the detainees’ home areas.
Still, even at the larger district level and focusing only on pedestrian stops, the slight majority of detainees were, on average, nonresidents. Therefore, in an absolute sense, the underlying assumption of census benchmarking is clearly not met. Examining spatial variation in the percentage of detainees that are local residents, the highest levels were about 58 percent in the South Chicago and Rogers Park districts (see Figure 2). The lowest levels were in the central business districts, with the Near North area having an especially low percentage (only 20 percent).
Percentage of pedestrian stops involving residents by Chicago police district.
Such variation among local areas complicates logical predictions of when census benchmarking for pedestrian stops is most likely to lead to erroneous conclusions. The higher the proportion of nonresidents among detainees, the greater the significance of any differences in racial composition between residents and nonresidents for standard disproportionality calculations. However, if the racial composition of nonresident detainees approximates that of stopped residents, then the overall impact could be minimal. Further complicating matters, social science scholarship has emphasized that certain policing disparities are theoretically more likely to emerge in the context of predominately white spaces. Yet social science research has also suggested that such contexts are more likely to have higher nonresident minority influx, and thus the potential for systematic calculation error is greater. In particular, as poignantly argued by Anderson (2015), “White people typically avoid black space, but black people are required to navigate the white space as a condition of their existence” (p. 10). Ultimately, a direct empirical examination of the importance of ignoring residency distinctions in racial disproportionality calculations is needed.
How Does Ignoring Detainee Residence Affect the Assessment of Racial Disproportionately in Pedestrian Stops?
Table 1 provides data on how police districts vary both in terms of their census racial composition and the degree to which black individuals are disproportionately represented in pedestrian stops, with the disproportionality index calculated separately with and without nonresidents. As can be seen, stop disproportionality values tend to be higher in districts with a lower percentage of black residents (of course, it is important to recognize that by mathematical design, districts that are 100 percent black cannot have disproportionality values >1). The district noted earlier as having the lowest percentage of resident detainees, the Near North area, also had one of the highest overall disproportionality indices: black individuals were about 11 times more likely to be stopped than the census benchmark would predict. Interestingly, however, this disproportionality value did not appear to be significantly affected by the frequency of nonresident stops; the result was comparable when the calculation was restricted to district residents only.
Police District by Percentage Black Residents and Stop Disproportionality Indices.
Note: A disproportionality index value of 1 indicates a perfect match between the census benchmark and the racial composition of detainees. Higher values indicate a greater than expected percentage of black detainees. For example, a value of 3.2 indicates that black individuals are 3.2 times more likely to be stopped than their representation in the census population.
Indeed, unlike results from previous vehicle-stop studies using rural and suburban counties, rankings of police districts by disproportionality values were generally comparable regardless of whether nonresidents were excluded from the calculation. Disproportionality ratios comparing black individuals more specifically with white individuals revealed a similar pattern but with expectedly higher values and a slight tendency for the resident-only index to actually be larger than the overall measure of disparity (see Supplemental Material). The Jefferson Park district was a notable exception: both the disproportionality index and the more standardized disproportionately ratio appeared significantly lower when nonresidents were excluded. The uniqueness of this case may be due to the presence of O’Hare Airport in this area, as well as other distinctive features of this geographically large district (e.g., the Norridge and Harwood Heights island).
In general, police districts are very large areas, with a few covering more than 20 square miles each. One advantage to using such large geographies is that within-unit sample sizes are greater, which allows the creation of more reliable measures. However, a clear disadvantage of using police districts is the possibility of aggregation bias. Fortunately, the data also include police beats, which are much smaller spatial units that can be similarly analyzed with appropriate adjustments for influential outliers.
Figure 3A illustrates the relationship between the census percentage white and the percentage of detainees who are nonresidents at the beat level. The data suggest an accelerating positive relationship whereby the percentage of those stopped who are nonresidents tends to be higher in beats where the census percentage of white residents is also higher. Therefore, like the district-level data, the beat-level data indicate the possibility of systematic calculation error, contingent on the existence of systematic differences in the racial compositions of resident and nonresident detainees.
Estimates of racial disproportionality in investigatory stops are more sensitive to the inclusion of nonresidents in predominately white areas. (A) Percentage of stops involving nonresidents. (B) Difference in racial composition of residents versus nonresidents detained.
Figure 3B shows that indeed, such differences exist at the beat level. More specifically, the data indicate a modest linear relationship whereby nonresident detainees tend to be composed of a greater proportion of black individuals than is the case for resident detainees in predominately white beats. Both of these observed relationships, displayed in Figures 3A and 3B, were statistically significant using heteroscedasticity-consistent standard errors (p < .05). In combination, these data patterns suggest that the use of the census benchmark with nonresident detainees in the numerator will inflate estimates of racial disproportionality in majority-white spaces.
Still, the degree of inflation matters, and in this case, high levels of racial disproportionality in predominately white areas were still present when the calculation was restricted to residents. This can be seen in Figure 4, which illustrates both the degree to which disproportionality ratios can be significantly biased by the inclusion of nonresidents in the numerator and how disproportionality remains pronounced in majority-white police beats even when this significant bias is removed. These general findings were robust to different estimation strategies (see Supplemental Material).
Regardless of whether nonresidents are excluded from the calculation, black stop disproportionality remains pronounced in predominately white police beats.
Discussion and Conclusion
In a strict sense, the implicit assumption of census benchmarking, that all pedestrians stopped in an area are residents, is not justified by the data. Even for spatial units as large as police districts, the majority of stopped pedestrians live elsewhere. For police beats, smaller areas that more closely approximate neighborhoods, the average percentage of detainees who are residents is particularly low: only 26 percent. Nevertheless, what researchers, journalists, lawyers, and others wishing to use census benchmarking really need to know is how this violation of assumption might influence the appropriate interpretation of results.
If the racial makeup of nonresidents stopped roughly matches that of residents stopped, then the conceptually problematic aggregation of the two in the disproportionality numerator may not matter (empirically). However, the data analyzed here suggest that places systematically vary not only in the percentage of detainees who are residents but also in the degree to which the racial composition of stopped residents matches that of nonresidents. In particular, the present analysis indicated that police beats with higher percentages of white residents tended to be places exhibiting (1) higher percentages of nonresident detainees and (2) larger proportions of black individuals among nonresident detainees than resident detainees. These data patterns imply that assessments of racial stop disproportionality using the overall census benchmark in majority-white neighborhoods will inflate disparities. For example, the present analyses revealed that excluding nonresidents from the numerator would reduce the median black/white disproportionality ratio by about 66 percent for predominately white police beats.
It is possible that the proximate cause of the inflated disproportionality estimate, the higher black prevalence for nonresidents stopped versus residents stopped, could itself be an indicator of a more nuanced type of racial profiling. But reasonable conclusions would require further data on the characteristics of nonresidents available to be stopped, and the issue would need to be analyzed from an intersectionality framework that is rarely applied in this line of research. Such a novel application might contend that the interaction between being classified as black and nonlocal has special relevance for heightened police suspicion in predominately white neighborhoods (potentially reflecting social class distinctions between minority residents and minority nonresidents). However, the more common explanation for high disproportionality indices is consistent with the well-known case of Harvard professor Henry Louis Gates Jr.; regardless of social class designations and resources, minority residents are deemed more threatening and “out of place” in majority-white areas and thus subject to additional police scrutiny (Ogletree 2010).
Although the present analysis found that disproportionality estimates can be significantly amplified when nonresidents are included in the numerator, the results also suggested that this methodological issue cannot fully explain the commonly reported finding of high black disproportionality for pedestrian stops in majority white areas. For example, properly matching the numerator to the denominator, black residents were still 7 times more likely to be stopped than white residents in police beats where white people predominate. In the end, aggregating stopped urban pedestrian residents and nonresidents together introduces a significant bias, at least for analyses using smaller geographic units, but sizable racial disparities persist even when this bias is removed.
It is important for future research to isolate other potential explanations for such persistently large racial disparities. One frequently discussed possibility is that black neighborhood residents behave differently than other neighborhood residents and are at greater risk for meeting the reasonable suspicion standard because of their behavior. For example, in the landmark case of Floyd v. City of New York (2013), the city’s expert, Dennis C. Smith, argued that the most appropriate benchmark was not the total estimated population (from census sources) but rather the total estimated criminal population (from additional police data on suspects and arrestees). Smith suggested that if 80 percent to 90 percent of known suspects for serious crimes are racial/ethnic minorities, then it is reasonable that racial/ethnic minorities would constitute a similar percentage of those stopped for investigation. However, District Court judge Shira Scheindlin (2013) rejected Smith’s argument, noting that such thinking could lead to “indirect racial profiling” and that
crime suspect data may serve as a reliable proxy for the pool of criminals exhibiting suspicious behavior. But there is no reason to believe that crime suspect data provides a reliable proxy for the pool of non-criminals exhibiting suspicious behavior. Because the overwhelming majority of people stopped fell into the latter category, there is no support for the City’s position that crime suspect data provides a reliable proxy for the pool of people exhibiting suspicious behavior. (p. 54)
Scheindlin’s position appeared anchored in the fact that regardless of detainee racial classification, the vast majority of investigatory stops in New York did not reveal weapons, uncover contraband, or lead to probable-cause arrest, which is also true for the Chicago data analyzed here.
Ideally, similar to the vehicle-stop research literature, observational data could be used to assess the public actions of resident pedestrians in those neighborhoods exhibiting significant disproportionality. Because such a research endeavor would require considerable resources, a more realistic and less costly approach might involve a variety of pedestrian stop outcome tests (Ayres 2002). Such tests, when carefully used, might help illuminate the degree to which behavioral differences can explain observed disparities (Harris 2002). Because outcome tests can be affected by issues of inframarginality, the recently proposed threshold test might be especially useful for future research to consider (Simoiu, Corbett-Davies, and Goel 2017).
Beyond the focus of the present study, the Chicago Police Department’s Investigatory Stop Report database includes a variety of unique variables for future research to analyze. For example, in addition to a measure of officer perception of the suspect’s Latinx ethnicity, the public data also include officer classification of the suspect’s skin tone. Although demographic data for police officers are not provided, another unique feature of the database is that it incorporates anonymized identification numbers for the officer(s) conducting the investigatory stop. Future research concerned with understanding racial disparities in stops and stop outcomes might consider implementing officer-level analyses (or multilevel models) to supplement more traditional stop-level inquiries.
Overall, the present analyses suggest that (1) as with vehicle stops, pedestrians are commonly detained outside their neighborhoods of residence; (2) evaluations of racial disproportionality based on residential census data, particularly in majority-white neighborhoods, are sensitive to the erroneous inclusion of nonresident pedestrians in the numerator; (3) the degree of sensitivity to this error for calculations of racial disproportionality in pedestrian stops appears less than that previously reported for vehicle stops; and (4) estimates of racial disproportionality in pedestrian stops are reduced, but disparities remain high when analyses match the numerator to the denominator by exclusively focusing on neighborhood residents. As more and more police departments make their investigatory stop data publicly available, the findings of the present study can hopefully help guide analytic approaches and interpretations of results in this important area.
Supplemental Material
SRD818746_Supplemental_Material – Supplemental material for Neighborhood Residence and Assessments of Racial Profiling Using Census Data
Supplemental material, SRD818746_Supplemental_Material for Neighborhood Residence and Assessments of Racial Profiling Using Census Data by Lance Hannon in Socius
Footnotes
Supplemental Material
Supplemental material for this article is available online.
Author Biography
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
