Investigating the Relation Between Police Behavior and Perceptions of Procedural Justice: A Response to Rick Trinkner

Introduction

In “The influence of police treatment and decision-making on perceptions of procedural justice: A field study” we investigated whether police-behavior that signals higher quality of treatment or decision-making leads to higher perceived procedural justice (Terpstra & van Wijck, 2021). We found no evidence that this is the case. Trinker (2022) argues that, based on our research, it would be premature to conclude that behavior signaling fairer treatment and decision-making is unassociated with the procedural justice judgment of those interacting with police officers. This clearly concerns the heart of our analysis. We very much appreciate that Trinkner carefully criticized our analysis and we are happy to respond to the critical remarks.

Our paper was intended as a contribution to the very scarce literature on the relation between how people are treated by the police and their perceptions of procedural justice. There is hardly any empirical research on the relation between police behavior in practice on the one hand and procedural justice judgements on the other. The study by Worden and McClean (2017) is the most important exception.

We agree with Trinkner that our general conclusion represents a challenging finding to procedural justice theory. The theoretical relevance is clear, since it concerns a crucial link in procedural justice theory. Also, the social relevance is clear. It would be important to know that if police-officers behave in a certain way, this can be expected to increase perceived procedural justice. This knowledge could be an important starting-point for policymaking.

We also agree with the Trinkner that it is important to reflect on the question of whether our conclusion is potentially based on misconceptions. Hopefully, the discussion triggers research into the relation between police behavior and perceptions of procedural justice. Afterall, without insight in this relation it would be premature to base policy conclusions on procedural justice theory.

This response is organized as follows. First, we reflect on the remark that our conclusion is premature and may be based on misconceptions. Second, we reflect on Trinkner's statement on our discussion of “objective procedural justice”. Third, we reflect on the potential misconceptions in our analysis. Finally, based on this reflection, we formulate a number of lessons for future research.

Premature Conclusion Based on Misconceptions?

Trinkner argues that our conclusion is premature and that our results could stem from misconceptions: “While it could be the case their results reflect that officers’ behavior has no influence on judgments of procedural justice, it could also be the case that their results stem from misconceptions about the relation between objective behavior and subjective judgments embedded within their methods.” This, of course, concerns the hart of our analysis, since our goal was to investigate whether police-behavior that signals higher quality of treatment or decision-making leads to higher perceived procedural justice.

Before responding to this to this critical remark, we first make a preliminary remark regarding our conclusion. In Trinkner's words, our conclusion is “that behavior signaling fairer treatment and decision-making is unassociated with the procedural justice judgments of those interacting with police officers”. We intended, however, to formulate a slightly more nuanced conclusion: “We find no evidence that police behavior that signals fairer treatment or decision-making leads to higher perceived procedural justice.” Or: “Our results do not support the idea that higher quality of police treatment and decision-making leads to higher levels of perceived procedural justice.” What we basically are saying, is that we cannot reject the hypothesis that there is no influence. We reflected on potential explanations of this finding. We suggested that it is possible that a single encounter may be less important than assumed in shaping the pathway from procedural justice perceptions and compliance.

That being said, our findings could still be based on misconceptions about the relation between objective behavior and subjective judgments. In our understanding, this remark especially concerns our operationalization of “police behavior that signals higher quality of treatment or decision-making”. Our operationalization of “judgements of procedural justice” is standard and appears to be uncontroversial.

We agree with the main point raised by Trinkner: the fact that we did not find a relation between police behavior and PPJ, may be caused by the fact that there is no relation or by misconceptions. Consequently, the reviewer urges us to reflect on the question of whether we adequately operationalized police behavior. We will do that in this response.

Objective Procedural Justice

Trinkner indicates that “the crucial, yet often overlooked, distinction between subjective and objective procedural justice” is at the heart of the critique. As mentioned by Trinkner, this distinction is made by Tyler (1987). He seems to suggest that we intended to develop “an objective measure of procedural justice”. This, however, was not our intention. And, in fact, we don’t see merit in developing such a measure.

It is, however, crucial to investigate the relation between, in Tyler's (1987) terminology, “objective issues” and perceptions of procedural justice. To explain what these “objective issues” could be, Tyler refers to Thibaut and Walker (1975). This seminal work concerns procedural justice in the legal process. Thibaut and Walker (1975) focused on three objective qualities: pretrial bias, order effects in evidence presentation, and the amount and accuracy of information reaching the decision maker. Tyler (1987) also refers to a set of six criteria against which the fairness of a procedure can be assessed, suggested by Leventhal (1980): consistency, bias-suppression, accuracy, correctability, representativeness, and ethicality. Leventhal, however, does not attempt to develop one measure of objective procedural justice. The weight attached to different criteria can be different in different contexts. Assuming this is correct, developing one measure of objective procedural justice is not particularly fruitful. It is, however, to be expected that better scores on the criteria positively influence an individual's evaluation of procedural justice.

Our research differs from the work by Thibaut and Walker (1975) because it is not about behavior of actors in legal procedures. It is about behavior of police officers engaged in traffic controls. It is to be expected that specific behavior of police officers will have an influence on perceptions of procedural justice. It is about the relation between “objective issues” and perceptions of procedural justice. We do not see merit in first translating the objective issues into a measure of objective procedural justice, especially because the weight attached to the different criteria may be context dependent.

Consequently, we did not investigate the relation between an “overall measure of objective procedural justice” and “perceived procedural justice”. The main purpose of our study was to extend the research on the relationship between police behavior and perceptions of procedural justice by answering the following research question: to what extent does police behavior that signals higher quality of treatment or decision-making lead to higher perceived procedural justice?

To reflect previous research by Worden and McLean (2017) and Jonathan-Zamir et al. (2015) we did, however, perform a robustness check in which we used the different behaviors to construct an overall observed procedural justice scale. This scale is useful to obtain a broader assessment of officer's behavior (Jonathan-Zamir et al., 2015). It is not meant to be a complete representation of objective procedural justice. This is also reflected in our conclusion, in which we stipulate that our results add to the insights on the relation between specific categories of behavior of the police and perceived procedural justice.

Still, the question remains whether our operationalization of police behavior is based on misconceptions and leaves room for improvement.

Reliability and Validity

Trinkner addresses a number of potential methodological misconceptions that can be of influence on how to interpret our results. Below we discuss these potential misconceptions.

The first potential methodological misconception concerns the content validity of the items used in our research. The number of behaviors that could be observed in our research could justifiably be indefinite. The problem that arises, is that when variables that ought to be included in a model are left out, the estimate of the effect of the variables we do include in the model can be affected (Mauro, 1990). When the goal is to establish a single measure of objective procedural justice, the effect of potential omissions is even larger. In our research we chose to use a previously validated method, based on observable choices made by police-officers. The behaviors included in our research are not all-inclusive. We utilized a previously used method as a starting point to add to the scarce literature on the relationship between observed police behavior and perceived procedural justice.

The second potential misconception concerns the construct validity of the scales used in our research. The author states that the setup of our research suggests that actors enter police interactions with a mental checklist of 18 specific behaviors and base their judgments on the number they can tick off. Our perspective, however, is slightly different. The 18 specific behaviors are viewed as tapping different facets of treatment and decision making and are not reflective of an underlying construct. Including all 18 separate behaviors in our regression analyses would have been ideal. Due to the restricted number of observations that follows from the intensity of our field work and to follow indices from previous research by Tyler (2004), Schulhofer et al. (2011), Mazerolle et al. (2014), we categorized these behaviors into four different constructs that were also used by Jonathan-Zamir et al. (2015) and Worden and Mclean (2017). Higher scores on these constructs indicate more behavior that signals fairer treatment and decision making by police officers. The Kaiser-Meyer Olkin Measure of sampling adequacy and the polychoric correlation coefficients gave no reason to deviate from these categories. With this approach, we made a choice to limit ourselves and stay close to previous research to have a validated measure and potentially be able to compare results. We acknowledge that the SSO-method used in our study is only validated by two other studies.

The third potential misconception addressed by Trinkner mainly concerns the reliability of the observations of police behavior. He states that these observations are interpretations of behavior and that social judgments of situations can vary dramatically across actors and observers (e.g., Mikula, 1986; Nisbett et al., 1973). The author states that the same behavior can be interpreted differently when including non-verbal communication cues. In this regard it is important to commemorate that Systematic Social Observation reduces potential subjectivity because the observers observe a specific event and set of behaviors (rather than accessing general impressions of the police), have no personal stake in the situation (compared to the participants), are using an explicit observation protocol (that provides clear definitions for the coded items) which they are trained to use, and their work product is reviewed by other researchers (Jonathan-Zamir et al., 2015).

However, we do recognize and acknowledge that Systematic Social Observation can be challenged by observer error (Mastrofski et al., 2010). This is why we did extensive tests on inter-observer reliability during both the pilot traffic controls and the main phase of our study. These tests showed negligible differences between observers. This is an indication that the method used is effective in reducing potential subjectivity.

Implications for Future Research

This brings us to our perspective on future research. We concur with Trinkner that the current state of the literature sheds little light on the relation between objective features of an interaction and actors’ judgment making process. Our field study is a step in trying to gain more insight into that process. In the specific situation of moped traffic controls, we found no relationship between behaviors that in previous research were believed to foster perceptions of procedural justice. These results emphasize the potential methodological gaps in the current literature on the relationship between police behavior and perceived procedural justice. Below, we discuss three points of attention for future research that are essential in closing these potential gaps.

First and foremost, we see no merit in establishing a validated measure of objective procedural justice in future research. As Trinkner justly points out, subjective procedural justice is composed of social judgments rooted in actors’ interpretations, not of behaviors. These interpretations are context dependent, making it very difficult to create a scale based on an all-encompassing list of behaviors. Trying to determine the ‘real level of procedural justice’ does not seem to be a particularly fruitful avenue for future research. It is, however, crucial to investigate the relation between, in Tyler's (1987) terminology, “objective issues” and perceptions of procedural justice.

A potential pitfall with the selection of objective issues to be included in future research is that there are many behaviors that potentially can be included. The 18 behavioral items used in our research potentially contain omissions. This raises the question on how to discover which items should additionally be included in research on the relationship between police behavior and perceived procedural justice. Since the dependent variable is perceived procedural justice, the actor's perspective on this is crucial. In addition, the actor's perspective can help discover the impact of preexisting attitudes toward police and toward the law in general (Gau, 2011). We do not however agree that starting with the actors’ subjective judgments and work from there in developing objective measures is the only, or even the most important route towards no insights. In our view, it is recommendable to take previous literature and research as the starting point for the selection of objective issues. Including specific behaviors that are most consistently related to subjective procedural justice across multiple studies and contexts can help uncover specific behaviors related to perceived procedural justice in different settings. And using previously validated measures based on concepts that are widely used in the field can also be potentially fruitful.

How the specific objective issues then should be operationalized is the final important point of attention. Trinkner raises the critique that observations are subjective in itself. In our current study we addressed this issue by using SSO and extensive testing to reduce this potential subjectivity. However, there still is room for improvement. Experience has shown that the more discrete and limited the decisions an observer must make, the easier it is to achieve reliable results (Mastrofski et al., 2010). In the current study we used a method based on previous research. For future research it seems promising to see how the coding criteria of behavior can be made even more explicit. The potential downside to this, is that a more stringent focus on specific behavior, by definition excludes other behaviors that could be meaningful in understanding the relationship between police behavior and perceptions of procedural justice. The use of modern techniques, such as cameras could be of assistance in that regard, since those kinds of techniques make it possible to gather more information from an interaction, as well as the possibility for multiple researchers to review the behavior under investigation. Camera's do however also come with limitations. With the current state of technology, it is practically impossible to capture all the elements of an interaction. And since all methods come with limitations, we think it is very important that future research should not be restricted to SSO research alone. Both field experiments and quasi experiments can help gain valuable insight into the relationship between police behavior and perceived procedural justice.

Final Thoughts

We thank Trinkner for critically analyzing our work and fostering the discussion on how to move forward in the research on the relation between how people are treated by the police and their perceptions of procedural justice. In this response we reflected on the potential methodological misconceptions brought forward by Trinkner. The potential misconceptions show that the field still contains gaps that need attending to. With our study, we wanted to contribute to a more thorough understanding of these gaps. We hope that this discussion leads to more future research on these gaps and the crucial relationship between police behavior and perceptions of procedural justice.