Sage Journals: Discover world-class research

Abstract

Following the EU Victim Directive, Dutch police officers are obliged to assess a victim’s vulnerability to repeat victimization. This study explored the utility of unstructured police information for the prediction of repeat victimization, as well as its incremental value over and above structured police information. Police records over a period of 6 years were retrieved for a sample of 116,680 victims. Unstructured information was transformed into numeric features using count-vector and TF/IDF methods. Classification models were built using decision tree and random forest models. AUC values indicate that a combination of structured and unstructured police information could be used to correctly classify a majority of repeat and non-repeat victims.

Keywords

repeat victimization risk assessment police predictive validity

In the Netherlands, almost 2 million individuals annually report being a victim of one or more types of crimes such as violent crimes (e.g., threat, assault, sexual crimes), property crimes (e.g., burglary, theft, robbery), or vandalism (Statistics Netherlands, 2020). Criminal victimization can have extensive impact on victims’ lives (Jackson & Gouseti, 2016; Lefebvre et al., 2021). From all individuals that do experience victimization, a small proportion of individuals experience repeat victimization, thereby accounting for a large percentage of the total number of victimization incidents reported (Gabor & Mata, 2004; SooHyun et al., 2017). Victims of violent crimes experience negative effects of victimization for longer periods of time compared to victims of other crimes (Jackson & Gouseti, 2016), this study, therefore, focused on repeat victimization of violent crimes.

In 2012, the EU Victim Directive was accepted by the European Parliament, thereby establishing minimum standards on the rights, support, and protection of victims of crime (European Parliament, 2012). Article 22 of this directive describes that Member States shall ensure a timely and individual assessment of victims to identify vulnerability, specific protection needs and to discuss special protection measures. Police officers play a crucial role in the support and protection of victims as they frequently encounter victims of crime. As a result of the Victim Directive, and more specifically article 22, Dutch police developed an individual assessment procedure for all victims that report a crime to the police. As part of this procedure, Dutch police officers are expected to review a victim’s police contact history and discuss a victim’s vulnerability to repeat victimization. Police officers, however, have limited time and reviewing a victim’s police contact history can be a time-consuming and laborious process. To support police officers in the individual assessment of victims, Raaijmakers et al. (2020) developed a risk assessment instrument that estimates the risk of repeat victimization for victims that report a crime to the police. This instrument, called ProVict, displays whether a victim is classified as high- or low-risk for repeat victimization and it summarizes a victim’s police contact history that contributed to the classification. The ProVict score may guide police officers in their assessment of a victim’s vulnerability to repeat victimization.

ProVict is based on structured information regarding victims’ police contact history as retrieved from the Dutch police database, as well as on victims’ age and gender. The risk score is based on the number of prior police contacts and the types of crimes the victim has previously been involved in. Although the validation study demonstrated that the predictive validity of ProVict was acceptable (Raaijmakers et al., 2020), findings suggest that there is still considerable room for improvement in the prediction of repeat victimization of violent crimes. This improvement could possibly be realized by not only using structured police information, but also using unstructured police information (e.g., verbatim reports that contain police officers’ transcription of interviews with victims, witnesses, and suspects, or their own observations) that is included in the Dutch police database. This unstructured police information may include valuable cues regarding individuals’ vulnerability for repeat victimization. The present study aimed to investigate (1) the predictive validity of unstructured police information in the prediction of victim’s risk for repeat victimization of violent crimes and (2) the incremental validity of unstructured police information over and above structured police information (ProVict).

Risk Assessment in the Police Context

Over the last decades, specialized instruments to assist in decisions about risk assessment are increasingly being used by professionals in criminal justice systems (Storey et al., 2011). Different types of risk assessments can be distinguished: unstructured professional judgments, actuarial tools, and structured professional judgments (SPJ). Whereas unstructured professional judgments rely on the professional’s clinical discretion, qualification, and experience regarding risk assessment, actuarial tools depend on characteristics that have been identified by statistical methods as predictors for a given outcome. SPJ approaches combine the actuarial approach with a clinical judgment approach (Kropp, 2004). For a long time, risk assessments have relied mainly on unstructured professional judgments. Several studies, however, demonstrated that risk assessments based on unstructured professional judgments had less predictive accuracy than actuarial tools and SPJ approaches (Petherick et al., 2021; Taxman et al., 2020). As a result, the focus in research and practice has shifted away from relying only on unstructured professional judgments toward actuarial tools and SPJ approaches. Examples of SPJ approaches that have been developed to classify high-risk victims by police officers are the Brief Spousal Assault Form for the Evaluation of Risk (B-SAFER; Kropp et al., 2005) and the Domestic Abuse, Stalking, and Honor Based Violence form (DASH; CAADA, 2012). Some studies suggest that SPJ approaches and actuarial tools achieve similar accuracy (Hanson et al., 2007), however van der Put et al. (2019) demonstrated that actuarial tools outperformed SPJ approaches when predicting domestic violence.

An example of an actuarial tool is the Ontario Domestic Assault Risk Assessment (ODARA; Hilton et al., 2004). It was developed for police officers to assess the risk of domestic violence recidivism. Like the ODARA, other actuarial instruments often focus on specific outcomes such as domestic violence (Messing & Thaller, 2013; Svalin & Levander, 2020; van der Put et al., 2019) or do not directly predict repeat victimization, as they focus on the offender. The European Victim Directive requires an individual assessment for all victims of crime, Raaijmakers et al. (2020), therefore, developed an actuarial risk assessment tool that directly focuses on victims and their risk for repeat victimization of violent crimes. This instrument, called ProVict, is a collection of algorithms that calculate the risk of repeat victimization through an automatic procedure. To calculate a risk score, ProVict uses structured information from the police database pertaining to a victim’s police contact history. Structured information is often classified as quantitative data: it fits a pre-defined data structure and is highly organized. In addition to structured information, the police database contains unstructured information in the form of police officers’ notes, transcripts of interviews with offenders or witnesses, and victims’ statements. Unstructured data, in general, consist of free-text forms and are often categorized as qualitative data, with no pre-defined structure or format.

Over 90% of all information available in official police records consists of unstructured data (Poelmans et al., 2010). Unstructured data in the form of written texts are considered to be the richest source of information for crime analysis. Extracting and collecting meaningful information from unstructured information is challenging as the process is complex and laborious when performed manually (Grimmer & Stewart, 2013). To solve this problem, text mining has been identified as a promising approach for crime analysis (Hassani et al., 2016). Text mining approaches automatically extract relevant information from unstructured information (Antons et al., 2020). To date, however, relatively few text mining approaches that use police information have been developed (Haleem et al., 2019). Examples of text mining approaches that have been applied to unstructured police information are attempts to identify domestic violence (Poelmans et al., 2010), potential jihadists (Elzinga et al., 2010), mentions of mental health disorders (Haleem et al., 2019; Karystianis et al., 2018), and abuse types conducted by offenders and injuries sustained by victims (Karystianis et al., 2019). These studies suggest that text mining techniques can be used to extract information from unstructured police information, which may subsequently be used for the prediction of victims’ risk of repeat victimization.

Repeat victimization has theoretically been explained by individual risk factors such as biological aspects (e.g., gender, physical vulnerability), risky lifestyles, (e.g., staying out late drinking), and occupation (e.g., working nightshifts) (Nagin & Paternoster, 2000). Furthermore, previous victimization experiences may lead to psychological distress, risky lifestyles, and substance use, which may increase the risk of repeat victimization (Gong, Kamboj, et al., 2019; López-Martínez et al., 2018; Rancher et al., 2019). Previous studies employing a text mining approach to police data identified individuals or incidents of interest by using custom-made dictionaries containing words related to these risk factors (Elzinga et al., 2010; Haleem et al., 2019; Karystianis et al., 2018, 2019, 2020; Poelmans et al., 2010). To our knowledge, no studies have used unstructured police information to assess a victim’s risk of repeat victimization of violent crimes in general. As the group of repeat victims is highly heterogenous (Obsuth et al., 2018), we employed an exploratory, data-driven, approach, meaning that we did not develop an a priori dictionary, but let the models reveal which words were related to repeat victimization.

Combining Structured and Unstructured Data for Risk Assessment

Research has demonstrated the incremental value of unstructured information over structured information in the prediction of a wide-range of topics. Combining unstructured and structured information improved the prediction of child abuse (Amrit et al., 2017), medication administration errors (Härkänen et al., 2020), suicide attempts (Adamou et al., 2018), use of health care services (Hatef et al., 2021), coronary artery disease (Jonnagaddala et al., 2015), and drug treatment outcomes (Laar et al., 2020). With regard to risk assessment of repeat victimization based on official police records, to the best of our knowledge, the incremental value of unstructured police information over structured police information is not yet investigated. It is argued that structured and unstructured information both contain unique information (Verma & Maiti, 2018). We, therefore, expected that unstructured police information provides incremental validity over and above structured police information in the prediction of repeat victimization.

Present Study

We proceeded in three steps. First, we investigated the validity of unstructured police information for predicting repeat victimization. Second, we assessed the incremental validity of risk scores based on unstructured police information over and above risk scores based solely on structured police information (ProVict). Third, we assessed the predictive accuracy (i.e., sensitivity and specificity) of risk scores for repeat victimization based on both unstructured and structured police information.

Method

Sample and Design

Data were retrieved from the registration system used by the Dutch police. A random sample of individuals (N = 150,000) was drawn from all individuals registered by the Dutch police as a victim between July 1st, 2016 and June 30th, 2017. The last victimization incident in this period will be referred to as the index incident. Information on all police contacts across the 4 years prior to the index incident, as well as information on violent victimization incidents across the 2 years after the index incident was collected. Information on the 4 years prior to the index incident was used to predict victimization in the 2 years after the index incident. Individuals that had died within 2 years after the index incident, whose date of birth or gender was unknown were removed from the data, as this information is needed for the ProVict predictors. Individuals who had no police contacts in the 4 years prior to the index incident were also removed from the data as they had no unstructured police information. The anonymized dataset for analysis eventually contained 116,680 individuals. Victims were aged between 0 and 100 years (M = 37.1, SD = 14.3), 46% was female and 54% was male. The total dataset was randomly split into a training set (n = 58,461) and a test set (n = 58,219). Models were developed on the training set and validated on the test set.

Measures

Repeat victimization

We distinguished two types of violent crimes: severe and less severe violent crimes. According to the definition used by the Dutch police, severe violent crimes include crimes such as sexual assault and human trafficking. Victims of these severe violent crimes are believed to have an elevated risk of repeat victimization, secondary victimization, intimidation, and retaliation. We also focused on less severe violent crimes. Examples of less severe violent crimes are threat and violent theft. Both outcome variables were operationalized as dichotomous variables indicating whether or not a victim had experienced victimization of a severe and less severe violent crime, respectively, within 2 years after the index incident.

Textual reports

Textual reports that were created across 4 years prior to the index incident were retrieved from the police system. These textual reports included, for example, investigator notes, transcripts of interviews with offenders or witnesses, and victims’ statements. When individuals who come into contact with the police do not speak Dutch, an interpreter is provided. Textual reports are, therefore, always in Dutch. Per victim, all textual reports were concatenated. In other words, for each victim all texts were added together, resulting in one overarching document per victim. To improve the efficiency of this process, stop words (commonly occurring terms that are uninformative) were removed from these documents.

Next, textual data in the overarching documents were transformed into numeric features to obtain predictor variables. For this purpose, two data processing strategies were applied: count-vector and TF/IDF (term frequency/inverse document frequency). The count-vector procedure replaced a text with a vector (list) of integer counts. Across victims, the vector elements represented the same words. Each element of the vector represented a word in the text and the number of times that word occurred in the text (term frequency). In the TF/IDF procedure, which is related to the count-vector procedure, the term frequencies were weighted by the inverse document frequency. This is a measure indicating how common a word was across documents, or across victims as each victim had one overarching document. Frequently occurring words across all victims thus received lower weights than less frequently words across all victims, as frequently occurring words are not necessarily the most relevant words. When a word appears many times in one document (e.g., one victim), while not appearing many times in other documents (victims), it is arguably more relevant for that specific victim than words that appear many times across all victims.

To restrict the length of the vector, in both procedures, only the 10,000 most commonly occurring words were vectorized. To improve efficiency of the process, the 2,000 most distinguishing words were selected as model building features. These words were selected using a chi-square selection algorithm.

Analyses

Model building

For each outcome (severe and less severe violent crimes), the objective was to use unstructured information to build a classification model which optimally allocates victims to the correct categories (high- and low-risk) based on the variables yielded from their overarching documents. Classification models can be built using several methods of analysis. Previous studies argued to replace traditional regression models with decision tree or random forest models (Berk, 2021; Berk & Sorenson, 2020; Lussier et al., 2019; Ngo et al., 2015, 2018). In this study, two analytical procedures were considered: decision tree and random forest. It is argued that decision tree and random forest models are suitable models for analyzing rare events, such as repeat victimization of severe and less severe violent crimes, since these models have no assumptions for distributions (Hegelich, 2016).

Decision tree

A decision tree model iteratively splits a dataset into a sequence of nested smaller groups based on the best distinguishing feature in relation to the target variable (i.e., repeat victimization). The first split into two groups starts with the strongest distinguishing feature, the next split selects the best distinguishing feature for each separate group (branch), and so on. This results in a reverse tree-like model representation. The main advantage of a decision tree model is its easy interpretation. Disadvantages include a lack of robustness, that is, small changes in the data may cause significant changes in the model and hence its predictions. Moreover, decision tree models may be prone to overfitting (Hegelich, 2016), implying that an excellent model fit to the training set will not generalize to the test set. In this study, a decision tree with a maximum depth of 10 nodes was developed.

Random forest

A random forest is an ensemble of decision trees. Sub-trees were constructed using a random subset of the records in the training set, as well as a random subset of the features. We used random forest with 25 trees with a maximum depth of 10 nodes. After training these sub-trees, a final risk score was calculated by averaging the predictions of the 25 individual decision trees.

Predictive validity unstructured data models

Both decision tree and random forest models resulted in a risk score (ranging from 0 to 1) that reflects the chance of repeat victimization for each victim. This risk score was based on the textual reports (unstructured information) that were transformed into numeric features using count-vector and TF/IDF procedures. In sum, four models for each outcome variable were estimated using two data processing strategies (count-vector and TF/IDF) combined with two classification procedures (decision tree and random forest). These models were evaluated with regard to the Area Under the Curve (AUC), obtained by Receiver Operator Characteristics (ROC) analysis. In the present context, AUC values reflect the probability that a randomly selected repeat victim has a higher risk score than a randomly selected non-repeat victim. The AUC value is one of the most well-known and widely used indicators to assess an model’s predictive validity (Messing & Thaller, 2013; Rice & Harris, 2005). According to rules of thumb proposed by Hosmer and Lemeshow (2000), AUC values between .60 and .70, between .70 and .80, and between .80 and .90, reflect moderate, acceptable, and excellent accuracy, respectively. According to Rice and Harris (2005), AUC values of .56, .64, and .71 correspond with small, medium, and large effect sizes, respectively. Based on the AUC, a final model based on unstructured information for severe violent crimes and a final model for less severe violent crimes was selected for subsequent analysis.

ProVict

To investigate the incremental validity of unstructured information over structured information (ProVict) in the prediction of repeat victimization, ProVict models for severe and less severe violent crimes were applied to the victims in this study. For this purpose, information regarding all predictors in ProVict was collected across the 4 years prior to the index incident. For a detailed description of ProVict, including predictors and associated weights, see Raaijmakers et al. (2020). By applying the ProVict algorithms for severe and less severe violent crimes, two risk scores that reflect the chance of repeat victimization of a severe and less severe violent crime, respectively, were calculated for the victims in this study.

Incremental validity

The incremental validity of unstructured information over structured police information in predicting repeat victimization was assessed with hierarchical logistic regression analyses. ProVict risk scores (based on structured information) and risk scores based on unstructured information were entered successively into the logistic regression model. Likelihood-ratio tests were performed to compare the successive models. Given our large dataset (test set: n = 58,219), we used a conservative approach and considered results significant when p < .001.

Model combination methods and evaluation

After examining the incremental validity of unstructured information over structured information, risk scores based on structured (ProVict), and unstructured information (final model) were combined to create a combination risk score. Following previous studies (Lehmann et al., 2013; Seto, 2005), several combination methods were used. These methods can be divided into two categories: intuitively appealing rules and statistical optimization methods. We applied three intuitively appealing rules: believe-the-negative, believe-the-positive, and averaging. The believe-the-negative approach implied that the lowest risk score was leading, whereas the believe-the-positive approach implied that the highest risk score was leading. The averaging approach implied that a new risk score was obtained by calculating the average of both risk scores. Two statistical optimization methods were used to combine the risk scores based on structured and unstructured information. First, both risk scores were entered in a logistic regression model to obtain the predicted probabilities based on both structured and unstructured information. Second, to account for the possibility that the effect of one risk score on repeat victimization depended on the other risk score, both risk scores and their interaction term were entered in a second logistic regression model in order to obtain these predicted probabilities.

The predictive validity of different combination methods was evaluated on the basis of their respective AUC values. Subsequently, the following predictive validity indicators were calculated for the best combination model: sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Discrimination indicators such as sensitivity and specificity reflect to what extent high- versus low-risk groups can be separated. Sensitivity refers to the proportion of repeat victims classified as high-risk, whereas specificity refers to the proportion of non-repeat victims classified as low-risk. In other words, sensitivity reflects the percentage of true-positives, whereas specificity reflects the percentage of true-negatives the calibration indicators such as the PPV and the NPV reflect to what extent the expected classification resemble the observed classification. The PPV indicates the proportion of high-risk victims who experience repeat victimization and the NPV indicates the proportion of low-risk victims who do not experience repeat victimization (Singh, 2013).

The predictive validity indicators mentioned above rely on the chosen cut-off threshold for distinguishing high-risk victims from low-risk victims. To investigate the consequences of using different thresholds, the test set was divided into 10 equal groups (deciles) based on individuals’ risk scores. For each decile, a risk score that represents the cut-off threshold was calculated. Individuals with risk scores lower than this cut-off threshold were considered to be low-risk and individuals with risk scores higher than or equal to the cut-off threshold were considered to be high-risk. Sensitivity, specificity, PPV, and NPV were calculated for these 10 cut-off thresholds.

The PPV and NPV rely heavily on the distribution of the outcome variable, in this study, repeat victimization. Helmus and Babchishin (2017), therefore, argued that indicators such as the PPV and NPV do not adequately reflect a model’s predictive accuracy, as they provide more information on the base rate of the outcome variable than model itself. These indicators do, however, provide relevant insights on the practical implications of different cut-off thresholds for distinguishing high- from low-risk victims.

Results

Descriptive Results

Repeat victimization of a severe violent crime within 2 years after the index incident was experienced by 0.9% of the victims. Repeat Victimization of a less severe violent crime within 2 year after the index incident was experienced by 2.0% of the victims.

Evaluation Unstructured Data Models

Table 1 shows the AUC values for all models based on unstructured information for both severe and less severe violent crimes. AUC values ranged from .74 to .80 for severe violent crimes and from .70 to .76. For less severe violent crimes, indicating acceptable to excellent discrimination and medium to large effect sizes for all models. The count-vector random forest (CV-RF) models produced the highest AUC values for both outcome variables, although the AUC values differed slightly. Hence, the count-vector random forest model resulted in the best model for risk assessment of repeat victimization based on unstructured data. Therefore, this model was selected for subsequent analyses. From the 2,000 most distinguishing words that were selected as model building features, the CV-RF model for severe violent crimes resulted in 1,111 significant features (i.e., words). The CV-RF model for less severe violent crimes resulted in 1,801 significant words. As we employed a data-driven approach, not all words were clearly interpretable in relation to an elevated risk of repeat victimization. However, some words did reflect aspects of known risk factors for repeat victimization described in the literature. For instance, for both severe violent and less severe violent crimes, several words were related to substance use (e.g., alcohol, beer, addicted, weed), psychological distress (anxious, to cry, stress, angry, aggressive), or family connections (daughter, relation, child, (ex)boyfriend, spouse).

Table 1.

AUC Values Models Based on Unstructured Information.

	AUC (95% CI)
	Severe violent crimes	Less severe violent crimes
Count-vector decision tree	.74 [0.71, 0.76]	.70 [0.69, 0.72]
Count-vector random forest	.80 [0.78, 0.82]	.76 [0.75, 0.77]
TF/IDF decision tree	.74 [0.71, 0.76]	.71 [0.70, 0.73]
TF/IDF random forest	.79 [0.77, 0.81]	.76 [0.74, 0.77]

Note. n = 58,219. AUC = area under the curve; CI = confidence interval.

Incremental Validity

Correlations between CV-RF risk scores and ProVict risk scores indicated a weak and moderate association, respectively, for severe violent crimes (r[58,217] = .30, p < .001) and less severe violent crimes (r[58,217] = .42, p < .001). Table 2 shows the results of models with only ProVict risk scores as predictor (Model 1) and models with both ProVict and CV-RF risk scores as predictors (Model 2). For both severe and less severe violent crimes, ProVict and CV-RF scores were significantly associated with the likelihood of repeat victimization. Likelihood ratio tests were significant, indicating that the goodness of fit of the models improved significantly when adding the CV-RF scores.

Table 2.

Hierarchical Logistic Regression Models for Severe and Less Severe Violent Crimes.

	Severe violent crimes				Less severe violent crimes
	Model 1		Model 2		Model 1		Model 2
	OR	SE	OR	SE	OR	SE	OR	SE
Constant	0.01***	0.05	0.01***	0.05	0.02***	0.03	0.01***	0.04
ProVict risk score	96.64***	0.34	19.71***	0.40	240.89***	0.23	29.81***	0.26
CV-RF risk score^a			1.15***	0.01			1.13***	0.01
Likelihood ratio	5,831.95		5,718.57		12,167.79		11,858.06
Δ Likelihood ratio	113.38***				309.73***

Note. n = 58,219.

Risk scores were multiplied by 100 to facilitate interpretation of the odds ratios.

***

p < .001.

Evaluation Combination Models

Table 3 presents the AUC values for the different methods that combined ProVict risk scores and CV-RF risk scores. These combination methods yielded accurate models (AUC > .70) for both outcomes. For both severe and less severe violent crimes, Combining two risk scores using statistical optimization methods resulted in higher AUC values than the intuitively appealing rules, although differences were small. Consequently, risk scores produced by a logistic regression analysis with both risk scores and the interaction term as independent variables showed the highest AUC values. The AUC value for severe violent crimes was higher than .80, indicating excellent discrimination and a large effect size. The AUC value for less severe violent crimes indicated acceptable discrimination and a large effect size, as the AUC value was higher than .70.

Table 3.

AUC Values Combinations Models Based on ProVict and CV-RF Risk Scores.

	AUC (95% CI)
	Severe violent crimes	Less severe violent crimes
Believe-the-negative	.78 [0.76, 0.80]	.76 [0.74, 0.77]
Believe-the-positive	.80 [0.78, 0.82]	.76 [0.75, 0.78]
Averaging	.81 [0.79, 0.83]	.77 [0.76, 0.78]
Logistic regression	.81 [0.79, 0.83]	.77 [0.76, 0.79]
Logistic regression + interaction term	.81 [0.80, 0.83]	.77 [0.76, 0.79]

Note. n = 58,219. AUC = area under the curve; CI = confidence interval.

Additional predictive validity indicators for the best combination method (logistic regression with interaction term) for severe violent crimes are presented in Table 4. These indicators illustrate how a risk assessment instrument based on structured and unstructured information can perform in practice, when applying different thresholds to distinguish between high- and low-risk victims. Table 4 indicates the typical trade-off between sensitivity and specificity; the higher the cut-off threshold, the fewer repeat victims are classified as high-risk (lower sensitivity), the more non-repeat victims are classified as low-risk (higher specificity). The PPV indicates that only a small proportion of high-risk victims are repeat victims, while the NPV indicates that a large proportion of low-risk victims do not experience repeat victimization. Similar indicators were found for repeat victimization of less severe violent crimes.

Table 4.

Predictive Validity Indicators Best Combination Method Severe Violent Crimes.

Cut-off thresholds	Sensitivity (%)	Specificity (%)	PPV (%)	NPV (%)
Top 100%	100.0	0.0	0.9	0.0
Top 90%	98.8	10.1	1.0	99.9
Top 80%	97.5	20.2	1.1	99.9
Top 70%	95.4	29.8	1.2	99.9
Top 60%	92.9	40.3	1.4	99.8
Top 50%	90.4	50.4	1.6	99.8
Top 40%	85.4	60.4	1.9	99.8
Top 30%	78.6	70.4	2.3	99.7
Top 20%	67.4	80.4	3.0	99.6
Top 10%	47.0	90.3	4.2	99.5

Note. n = 58,219. PPV = positive predictive value; NPV = negative predictive value.

Discussion

Following the Victim Directive, Dutch police officers are required to assess a victim’s vulnerability to repeat victimization. When victims are considered vulnerable (to repeat victimization), protection measures can be applied. Examples of protection measures are protection orders, shielding the victim’s name or address from the police report, audio-visually recording the victim’s interview, and referral to victim support services. As police officers have limited time and capacity, it is important that police officers correctly estimate the risk of repeat victimization and identify victims that are at-risk for repeat victimization. Moreover, it is important to correctly identify and protect repeat victims, as a small number of victims account for a large proportion of all victimization incidents (Gabor & Mata, 2004; SooHyun et al., 2017), and because the impact of victimization on an individual is often immense (Jackson & Gouseti, 2016). As a result, limited time and resources can be targeted at these high-risk victims. Identifying high-risk victims, however, is challenging as victims are a heterogenous group, with a great variety in frequency and severity of subsequent victimization incidents (Hope & Norris, 2013; Obsuth et al., 2018).

To guide police officers in the assessment of vulnerable victims, Raaijmakers et al. (2020) developed ProVict: an actuarial risk assessment tool that automatically calculates the risk of repeat victimization for victims that report a crime to the police. ProVict only used structured information from official police records. This study investigated the utility of unstructured police information for the prediction of repeat victimization, as well as its incremental value over and above structured police information. Finally, predictive accuracy of risk scores based on both unstructured and structured police information was assessed.

Firstly, unstructured police information showed predictive validity for the prediction of repeat victimization. For both severe and less severe violent crimes, AUC values indicated large effect sizes following Rice and Harris (2005) and acceptable and excellent discrimination following Hosmer and Lemeshow (2000). Secondly, in line with studies in other research areas (Adamou et al., 2018; Amrit et al., 2017; Härkänen et al., 2020; Hatef et al., 2021; Jonnagaddala et al., 2015; Laar et al., 2020), our findings suggest that unstructured information provides incremental validity over and above structured information in the prediction of repeat victimization. The goodness of fit improved when risk scores based on unstructured and risk scores based on structured information were included, compared to when only risk scores based on structured information were included. We conclude that structured and unstructured police information both contain unique information (Verma & Maiti, 2018) and incrementally contribute to the prediction of repeat victimization. However, the predictors of repeat victimization derived from unstructured and structured information, respectively, also showed some overlap. For example, significant ProVict predictors referred to missing person, assault, and threat cases, whereas significant words yielded from the CV-RF model were “missing person,” “assault,” “threat,” and synonyms of these words. Although we conclude that structured and unstructured police information contain unique information, some overlap is also present. Thirdly, AUC values of models based on a combination of both unstructured and structured information indicated large effect sizes (Rice & Harris, 2005) and excellent discrimination regarding severe violent crimes and acceptable discrimination regarding less severe violent crimes (Hosmer & Lemeshow, 2000). Discrimination indicators showed that models based on both unstructured and structured information were able to distinguish well between high- and low-risk victims. Across several cut-off thresholds (e.g., top 40%, top 30%, top 20%), a majority of the repeat victims and non-repeat victims was correctly classified as high- and low-risk, respectively.

A point of discussion concerns the method used for data processing of unstructured information. In the present study, two methods were used: count-vector and TF/IDF. Both methods do not capture semantic similarities between words (Qaiser & Ali, 2018). Future studies could, therefore, explore methods that do capture these semantic similarities, such as Word2Vec. Moreover, we used decision tree and random forest models as these approaches are suitable for analyzing rare events (Hegelich, 2016), such as repeat victimization. Berk and Sorenson (2020) recently applied three alternative algorithms for forecasting rare events. Future studies could investigate to what extent these alternative algorithms can successfully be applied to the prediction of repeat victimization. Future studies could also distinguish between different sources of unstructured police information (e.g., victims’ statements and police officers’ notes) and investigate whether these sources vary in their predictive validity regarding repeat victimization.

Finally, this study used official police records to facilitate the implementation of an instrument that predicts repeat victimization in Dutch police practice. The use of information that is already available by Dutch police is preferred over external information, as it simplifies the implementation of a risk assessment instrument. However, not all relevant information regarding risk factors for repeat victimization may be contained within official police records. In line with previous research, this study showed that words related to substance use, psychological distress, and family connections were associated with an increased risk of repeat victimization use (Gong, Kamboj, et al., 2019; Lawyer et al., 2006; López-Martínez et al., 2018; Rancher et al., 2019). To develop more knowledge on the risk factors of repeat victimization, future studies could focus on unstructured police information to identify other risk factors and explore whether information on these risk factors could, for example, be collected through an additional questionnaire. This study, however, focused on the development of an actuarial risk assessment instrument that can aid police officers in targeting their limited time and capacity to victims most at-risk of repeat victimization. This risk assessment instrument is based solely on structured and unstructured information from the police database, using no other data sources. This enabled an automated assessment of the risk of repeat victimization without any manual labor from police officers.

Based on our findings, we conclude that the combination of unstructured and structured police information may be a promising approach for risk assessment of repeat victimization within Dutch police. We conclude that a risk assessment instrument based on both structured and unstructured information provides more accurate predictions than a risk assessment instrument based solely on structured information (Raaijmakers et al., 2020). Such a risk assessment instrument may support police officers in their individual assessment of victims, by pointing out vulnerability to repeat victimization and indicating that for those victims further attention is needed. Such an instrument may also enable police officers to target their limited resources toward those victims that are most at-risk. Note that such an instrument is not intended to replace a police officer’s assessment, but is merely used to assist police officers.

For successful implementation, additional research is needed to explore practical possibilities or limitations of this risk assessment instrument in Dutch police practice. An investigation of the costs and benefits associated with the choice for different cut-off threshold should be part of such an investigation. As stated above, sensitivity and specificity values showed that across several cut-off threshold, a majority of repeat victims and non-repeat victims were correctly identified as high-risk and low-risk, respectively. At the same time, however, calibration indicators indicated that when applying these thresholds, a large number of high-risk victims do not experience repeat victimization (false-positives). These calibration indicators should be taken into account in cost/benefit analyses. It could be argued that correctly classifying a large number of repeat victims as high-risk outweighs the costs of classifying a certain number of non-repeat victims as high-risk. Note, that the relatively large numbers of false-positives are inevitable due to the low base rate of repeat victimization of (less) severe violent crimes. Note also, that not all false-positives in the present study may be non-repeat victims, as not all repeat victims may have reported their revictimization to the police. Previous studies showed that victims are less willing to report revictimization to the police, when previous police experiences were evaluated negatively (Colin Bolger & Walters, 2019; Koster, 2017).

To combine risk scores based on unstructured and structured information, this study removed individuals who had no police contact prior to the index incident from the data. When a risk assessment instrument for repeat victimization based on both structured and unstructured police information is considered for implementation, analyses that include persons who had no prior police contact should be executed. In practice, the instrument aimed to be used for all victims regardless of the presence of prior police contacts. Future research should address predictive validity when all victims are included.

This study demonstrated that unstructured police information may be a valuable addition to the risk assessment instrument ProVict (Raaijmakers et al., 2020). Our analyses revealed that a risk instrument using both types of data correctly classified a majority of repeat and non-repeat victims. In police practice, such an instrument may contribute to the identification of victims with an elevated risk of repeat victimization. Repeat victimization and its consequences may subsequently be prevented through applying adequate protection measures targeting these at-risk individuals.

Footnotes

Author Note

This study is part of a larger collaboration between Tilburg University, Radboud University, Praktikon, and the Dutch National Police.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was funded by the European Union.

ORCID iDs

Roos Geurts

Jacqueline Wientjes

Author Biographies

Roos Geurts is an external PhD-candidate at the Behavioural Science Institute (Radboud University Nijmegen). Her research focuses on risk assessment of repeat victimization in a policing context.

Niels Raaijmakers is an external PhD-candidate at the department of Criminal Law (Tilburg University). His research focusses on victim vulnerability, repeat victimization and the assessment of both in a policing context.

Marc J. M. H. Delsing is a senior researcher at Praktikon, Nijmegen, the Netherlands. His research interests include child and adolescent development, the evaluation of child and youth care interventions, risk assessment, and (longitudinal) research methodology.

Toine Spapens is full professor of Criminology at TIlburg University, the Netherlands, and specialises in multi-agency and cross-border law enforcement cooperation, particularly in the fields of organized crime and corporate environmental crime.

Jacqueline Wientjes is a behavioral scientist at the Dutch National police, unit Apeldoorn. She works on prediction models for the prevention of victimization and juvenile delinquency.

Dick Willems is a data scientist at the Dutch National Police, unit Amsterdam. He works on crime prediction and accessing information from unstructured data sources.

Ron H. J. Scholte is professor at the department of Developmental Psychopathology (Radboud University Nijmegen). His research focuses on the development of individuals and how psychosocial factors affect that development.

References

Adamou

Lagani

Antoniou

Charonyktakis

Greasidou

Tsamardinos

(2018). Mining free-text medical notes for suicide risk assessment [Conference session]. 10th Hellenic Conference on Artificial Intelligence, 1–8. July 2018. Patras, Greece. https://doi.org/10.1145/3200947.3201020

Amrit

Paauw

Aly

Lavric

(2017). Identifying child abuse through text mining and machine learning. Expert Systems with Applications, 88, 402–418. https://doi.org/10.1016/j.eswa.2017.06.035

Antons

Grünwald

Cichy

Salge

T. O.

(2020). The application of text mining methods in innovation research: Current state, evolution patterns, and development priorities. R&D Management, 50(3), 329–351. https://doi.org/10.1111/radm.12408

Berk

R. A.

(2021). Artificial intelligence, predictive policing, and risk assessment for law enforcement. Annual Review of Criminology, 4, 209–237. https://doi.org/10.1146/annurev-criminol-051520-012342

Berk

R. A.

Sorenson

S. B.

(2020). Algorithmic approach to forecasting rare violent events: An illustration based in intimate partner violence perpetration. Criminology & Public Policy, 19, 213–233. https://doi.org/10.1111/1745-9133.12476

CAADA. (2012). CAADA-DASH risk identification checklist (RIC) for MARAC agencies. Coordinated Action Against Domestic Abuse.

Colin Bolger

Walters

G. D.

(2019). The relationship between police procedural justice, police legitimacy, and people’s willingness to cooperate with law enforcement: A meta-analysis. Criminal Justice Journal, 60, 93–99. https://doi.org/10.1016/j.jcrimjus.2019.01.001

Elzinga

Poelmans

Viaene

Dedene

Morsing

(2010). Terrorist threat assessment with formal concept analysis [Conference session]. IEEE International Conference on Intelligence and Security Informatics: Public Safety and Security, 77–82. 23-26 May 2010. Vancouver, BC, Canada. https://doi.org/10.1109/ISI.2010.5484773

European Parliament. (2012). Directive 2012/29/EU of the European Parliament and of the Council of 25 October 2012 establishing minimum standards on the rights, support and protection of victims of crime, and replacing Council Framework Decision 2001/220/JHA. https://eur-lex.europa.eu/eli/dir/2012/29/oj

10.

Gabor

Mata

(2004). Victimization and repeat victimization over the life span: A predictive study and implications for policy. International Review of Victimology, 10(3), 193–221. https://doi.org/10.1177/026975800401000301

11.

Gong

A. T.

Kamboj

S. K.

Curran

H. V.

(2019). Post-traumatic stress disorder in victims of sexual assault with pre-assault substance consumption: A systematic review. Frontiers in Psychiatry, 10, 92. https://doi.org/10.3389/fpsyt.2019.00092

12.

Grimmer

Stewart

B. M.

(2013). Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political Analysis, 21(3), 267–297. https://doi.org/10.1093/pan/mps028

13.

Haleem

M. S.

Han

Harding

P. J.

Ellison

(2019). An automated text mining approach for classifying mental-ill health incidents from police incident logs for data-driven intelligence [Conference session]. IEEE International Conference on Systems, Man and Cybernetics, 2279–2284. October 2019. Bari, Italy. https://doi.org/101109/SMC.2019.8914240

14.

Hanson

R. K.

Helmus

Bourgon

(2007). The validity of risk assessments for intimate partner violence: A meta-analysis. Public Safety Canada, 7, 1–21.

15.

Härkänen

Vehviläinen-Julkunen

Murrells

Paananen

Franklin

B. D.

Rafferty

A. M.

(2020). The contribution of staffing to medication administration errors: A text mining analysis of incident report data. Journal of Nursing Scholarship, 52(1), 113–123. https://doi.org/10.1111/jnu.12531

16.

Hassani

Huang

Silva

E. S.

Ghodsi

(2016). A review of data mining applications in crime. Statistical Analysis and Data Mining: The ASA Data Science Journal, 9(3), 139–154. https://doi.org/10.1002/sam.11312

17.

Hatef

Rouhizadeh

Singh

Weiner

J. P.

Kharrazi

(2021). Assessing the impact of social needs and social determinants of health on health care utilization: Using patient- and community-level data. Population Health Management, 24, 222–230. https://doi.org/10.1089/pop.2020.0043

18.

Hegelich

(2016). Decision trees and random forests: Machine learning techniques to classify rare events. European Policy Analysis, 2(1), 98–120. https://doi.org/10.18278/epa.2.1.7

19.

Helmus

L. M.

Babchishin

K. M.

(2017). Primer on risk assessment and the statistics used to evaluate its accuracy. Criminal Justice and Behavior, 44(1), 8–25. https://doi.org/10.1177/0093854816678898

20.

Hilton

N. Z.

Harris

G. T.

Rice

M. E.

Lang

Cormier

C. A.

Lines

K. J.

(2004). A brief actuarial assessment for the prediction of wife assault recidivism: The Ontario domestic assault risk assessment. Psychological Assessment, 16(3), 267–275. https://doi.org/10.1037/1040-3590.16.3.267

21.

Hope

Norris

P. A.

(2013). Heterogeneity in the frequency distribution of crime victimization. Journal of Quantitative Criminology, 29(4), 543–578. https://doi.org/10.1007/s10940-012-9190-x

22.

Hosmer

D. W.

Lemeshow

(2000). Applied logistic regression (2nd ed.). John Wiley & Sons.

23.

Jackson

Gouseti

(2016). Threatened by violence: Affective and cognitive reactions to violent victimization. Journal of Interpersonal Violence, 31(18), 2987–3016. https://doi.org/10.1177/0886260515584336

24.

Jonnagaddala

Liaw

S. T.

Ray

Kumar

Chang

N. W.

Dai

H. J.

(2015). Coronary artery disease risk assessment from unstructured electronic health records using text mining. Journal of Biomedical Informatics, 58(Suppl), S203–S210. https://doi.org/10.1016/j.jbi.2015.08.003

25.

Karystianis

Adily

Schofield

Knight

Galdon

Greenberg

Jorm

Nenadic

Butler

(2018). Automatic extraction of mental health disorders from domestic violence police narratives: Text mining study. Journal of Medical Internet Research, 20(9), e11548. https://doi.org/10.2196/11548

26.

Karystianis

Adily

Schofield

P. W.

Greenberg

Jorm

Nenadic

Butler

(2019). Automated analysis of domestic violence police reports to explore abuse types and victim injuries: Text mining study. Journal of Medical Internet Research, 21(3), e13067. https://doi.org/10.2196/13067

27.

Karystianis

Simpson

Adily

Schofield

Greenberg

Wand

Nenadic

Butler

(2020). Prevalence of mental illnesses in domestic violence police records: Text mining study. Journal of Medical Internet Research, 22, e23725. https://doi.org/10.2196/23725

28.

Koster

N. S. N.

(2017). Victims’ perceptions of the police response as a predictor of victim cooperation in the Netherlands: A prospective analysis. Psychology Crime and Law, 23(3), 201–220. https://doi.org/10.1080/1068316x.2016.1239098

29.

Kropp

P. R.

(2004). Some questions regarding spousal assault risk assessment. Violence Against Women, 10(6), 676–697. https://doi.org/10.1177/1077801204265019

30.

Kropp

P. R.

Hart

S. D.

Belfrage

(2005). Brief spousal assault form for the evaluation of risk (B-SAFER). User manual. In PRI manuals. Vancouver: Proactive Resolutions.

31.

Laar

S. A.

Gombert-Handoko

K. B.

Guchelaar

Zwaveling

(2020). An electronic health record text mining tool to collect real-world drug treatment outcomes: A validation study in patients with metastatic renal cell carcinoma. Clinical Pharmacology and Therapeutics, 108(3), 644–652. https://doi.org/10.1002/cpt.1966

32.

Lawyer

S. R.

Ruggiero

K. J.

Resnick

H. S.

Kilpatrick

D. G.

Saunders

B. E.

(2006). Mental health correlates of the victim-perpetrator relationship among interpersonally victimized adolescents. Journal of Interpersonal Violence, 21(10), 1333–1353.

33.

Lefebvre

Fortin

Guay

(2021). Quality of life after violent crime: The impact of acute stress disorder, posttraumatic stress disorder, and other consequences. Journal of Traumatic Stress, 34, 526–537. https://doi.org/10.1002/jts.22623

34.

Lehmann

R. J.

Hanson

R. K.

Babchishin

K. M.

Gallasch-Nemitz

Biedermann

Dahle

K. P.

(2013). Interpreting multiple risk scales for sex offenders: Evidence for averaging. Psychological Assessment, 25(3), 1019–1024. https://doi.org/10.1037/a0033098

35.

López-Martínez

A. E.

Serrano-Ibáñez

E. R.

Ruiz-Párraga

G. T.

Gómez-Pérez

Ramírez-Maestre

Esteve

(2018). Physical health consequences of interpersonal trauma: A systematic review of the role of psychological variables. Trauma Violence & Abuse, 19(3), 305–322. https://doi.org/10.1177/1524838016659488

36.

Lussier

Deslauriers-Varin

Collin-Santerre

Bélanger

(2019). Using decision tree algorithms to screen individuals at risk of entry into sexual recidivism. Criminal Justice Journal, 63, 12–24. https://doi.org/10.1016/j.jcrimjus.2019.05.003

37.

Messing

J. T.

Thaller

(2013). The average predictive validity of intimate partner violence risk assessment instruments. Journal of Interpersonal Violence, 28(7), 1537–1558. https://doi.org/10.1177/0886260512468250

38.

Nagin

Paternoster

(2000). Population heterogeneity and state dependence: State of the evidence and directions for future research. Journal of Quantitative Criminology, 16(2), 117–144. https://doi.org/10.1023/a:1007502804941

39.

Ngo

F. T.

Govindu

Agarwal

(2015). Assessing the predictive utility of logistic regression, classification and regression tree, chi-squared automatic interaction detection, and neural network models in predicting inmate misconduct. American Journal of Criminal Justice, 40(1), 47–74. https://doi.org/10.1007/s12103-014-9246-6

40.

Ngo

F. T.

Govindu

Agarwal

(2018). Traditional regression methods versus the utility of machine learning techniques in forecasting inmate misconduct in the United States: An exploration of the prospects of the techniques. International Journal of Criminal Justice Sciences, 13(2), 420–437. https://doi.org/10.5281/zenodo.2657668

41.

Obsuth

Mueller Johnson

Murray

A. L.

Ribeaud

Eisner

(2018). Violent poly-victimization: The longitudinal patterns of physical and emotional victimization throughout adolescence (11-17 years). Journal of Research on Adolescence, 28(4), 786–806. https://doi.org/10.1111/jora.12365

42.

Petherick

Kannan

Brooks

(2021). Victim precipitation: An outdated construct or an important forensic consideration? Journal of Forensic Psychology Research and Practice, 21(3), 214–229. https://doi.org/10.1080/24732850.2020.1850087

43.

Poelmans

Elzinga

Viaene

Dedene

(2010). Curbing domestic violence: Instantiating C-K theory with formal concept analysis and emergent self-organizing maps. Intelligent Systems in Accounting Finance & Management, 17, 167–191. https://doi.org/10.1002/isaf.319

44.

Qaiser

Ali

(2018). Text mining: Use of TF-IDF to examine the relevance of words to documents. International Journal of Computer Applications, 181(1), 25–29. https://doi.org/10.5120/ijca2018917395

45.

Raaijmakers

Geurts

Delsing

Bosma

Wientjes

Spapens

Scholte

(2020). Protecting victims: Using prior police contacts to develop and validate a risk assessment instrument for repeat victimization in the Netherlands. Manuscript Submitted for Publication.

46.

Rancher

Jouriles

E. N.

Rosenfield

Temple

J. R.

McDonald

(2019). The mediating role of trauma symptoms in the association between past and future teen dating violence victimization. Journal of Abnormal Child Psychology, 47(3), 475–485. https://doi.org/10.1007/s10802-018-0461-3

47.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s d, and r. Law and Human Behavior, 29(5), 615–620. https://doi.org/10.1007/s10979-005-6832-7

48.

Seto

M. C.

(2005). Is more better? Combining actuarial risk scales to predict recidivism among adult sex offenders. Psychological Assessment, 17(2), 156–167. https://doi.org/10.1037/1040-3590.17.2.156

49.

Singh

J. P.

(2013). Predictive validity performance indicators in violence risk assessment: A methodological primer. Behavioral Sciences & the Law, 31, 8–22. https://doi.org/10.1002/bsl.2052

50.

SooHyun

Martinez

N. N.

Lee

Y. J.

Eck

J. E.

(2017). How concentrated is crime among victims? A systematic review from 1977 to 2014. Crime Science, 6(9), 1–16. https://doi.org/10.1186/s40163-017-0071-3

51.

Statistics Netherlands. (2020). Less traditional crime, more cybercrime. https://www.cbs.nl/en-gb/news/2020/10/less-traditional-crime-more-cybercrime

52.

Storey

J. E.

Gibas

A. L.

Reeves

K. A.

Hart

S. D.

(2011). Evaluation of a violence risk (threat) assessment training program for police and other criminal justice professionals. Criminal Justice and Behavior, 38(6), 554–564. https://doi.org/10.1177/0093854811403123

53.

Svalin

Levander

(2020). The predictive validity of intimate partner violence risk assessments conducted by practitioners in different settings—A review of the literature. Journal of Police and Criminal Psychology, 35(2), 115–130. https://doi.org/10.1007/s11896-019-09343-4

54.

Taxman

F. S.

Smith

Mason

(2020). Risk-Need-Responsivity (RNR) classification models: Still evolving. Aggression and Violent Behavior. Advance online publication. https://doi.org/10.1016/j.avb.2020.101459

55.

van der Put

C. E.

Gubbels

Assink

(2019). Predicting domestic violence: A meta-analysis on the predictive validity of risk assessment tools. Aggression and Violent Behavior, 47, 100–116. https://doi.org/10.1016/j.avb.2019.03.008

56.

Verma

Maiti

(2018). Text-document clustering-based cause and effect analysis methodology for steel plant incident data. International Journal of Injury Control and Safety Promotion, 25(4), 416–426. https://doi.org/10.1080/17457300.2018.1456468

Assessing the Risk of Repeat Victimization Using Structured and Unstructured Police Information

Abstract

Keywords

Risk Assessment in the Police Context

Combining Structured and Unstructured Data for Risk Assessment

Present Study

Method

Sample and Design

Measures

Repeat victimization

Textual reports

Analyses

Model building

Decision tree

Random forest

Predictive validity unstructured data models

ProVict

Incremental validity

Model combination methods and evaluation

Results

Descriptive Results

Evaluation Unstructured Data Models

Incremental Validity

Evaluation Combination Models

Discussion

Footnotes

Author Note

Declaration of Conflicting Interests

Funding

ORCID iDs

Author Biographies

References