Abstract
This paper documents and illustrates a practice-ready procedure for estimating changes in crash frequency for specific application circumstances for three intelligent transportation system treatments—Closed Circuit Television Cameras (CCTV), Dynamic Message Signs (DMS), and Road Weather Information Systems (RWIS). The procedure will allow an agency to directly evaluate the change in safety that may be associated with a contemplated treatment. In effect, the approach mimics the application of a Crash Modification Function (CMFunction) in that each potential application will, in principle, have its own Crash Modification Factor (CMF). The procedure uses an empirical Bayes framework with safety performance functions (SPFs) for treatment and non-treatment reference sites. The paper also presents those SPFs, which were developed from Pennsylvania freeway data. In principle, this cross-sectional approach can be applied, as it has been, for other safety treatments where safety effects vary with application circumstance and where that variability cannot be captured with conventional before–after studies.
Keywords
Intelligent transportation systems (ITS) are a significant component of the operations of state departments of transportation (DOTs) and are increasingly used to improve roadway operations. Understanding the safety implications of ITS can help secure funding for ITS. Many ITS applications are useful in managing traffic and improving response to incidents, and therefore could be expected to reduce the frequency and severity of crashes. However, there are currently very few studies that have developed high-quality crash modification factors (CMFs) for ITS applications. Knowing the safety implications of specific ITS treatments can help agencies implement these treatments for safety reasons in addition to operations purposes.
This paper addresses a need for estimating changes in crash frequency for specific application circumstances for three ITS treatments—closed circuit television cameras (CCTV), dynamic message signs (DMS), and road weather information systems (RWIS). This introduction establishes that need in first summarizing the pertinent details of these treatments and related existing knowledge on their safety effects. In so doing, it is recognized that these systems are not safety treatments, per se. In reality, there is range of ways in which they are used by agencies to make a variety of decisions that could indirectly affect safety performance.
CCTV are the cameras used to monitor traffic activity on major roadways. They are typically included as a method of collecting field data for traffic analysis and can be used as a component to monitor traffic in traffic monitoring stations. Depending on the agency, the cameras can be used for different purposes, including viewing roadway incidents/crashes or measuring speed and traffic composition. The main motivation for installing and using CCTV is for traffic management and to reduce congestion. CCTV are typically used for traffic observation, traffic incident or event verification, weather verification, traveler information, field device verification, and intelligent work zones ( 1 ). As a result, there are no studies or CMFs that have determined the safety impact of CCTV in the context of change in crashes.
DMS, also called variable message signs or changeable message signs, are electronic signs that are placed along or above major highways to deliver a wide range of information to drivers through changeable messages. The Manual on Uniform Traffic Control Devices ( 2 ) limits the use of DMS to display specific types of messages. Many studies investigated driver reactions to DMS but did not examine safety performance or produce CMFs (see, for example, Shealy et al. [ 3 ]). A literature review conducted for this research did include studies that examined the effect of DMS on speed, but the primary focus was crash-based studies that examined safety performance ( 4 – 8 ). Some of the studies were naïve before–after evaluations and did not adequately address many possible sources of bias, including regression-to-the mean, changes in traffic volume, and trends in the system. Overall, those studies provided mixed results, with some analyses showing a decrease in crashes when DMS are used, whereas other studies showed an increase in crashes.
RWIS efficiently process weather data and condense these data into an actionable format to support decision-making. Often managed at the State level, an RWIS comprises environmental sensor stations (ESS), a communications system, and a central data collection system. The ESS are placed in and near the roadway, though newer systems may also contain vehicle-based mobile RWIS sensors. There are over 2,400 RWIS ESS owned by State transportation agencies across the U.S., with at least one ESS in forty-nine States and Washington, DC ( 9 ). A 2019 survey of thirty-nine State DOTs found that 94% of agencies subscribe to RWIS data, and that the use of agency sensors such as RWIS has remained generally consistent over the past 10 years ( 10 ). Most safety research involving RWIS employs data collected through the system to study the effects of weather and road surface conditions on safety performance. Thus, the CMF Clearinghouse does not contain any CMFs for RWIS. However, a handful of studies have attempted to evaluate the safety effects of the RWIS itself. For example, El Esawey et al. performed a thorough evaluation of the safety benefits of using RWIS coupled with DMS on rural highways in British Columbia, Canada ( 11 ). The study analyzed data from six different locations where the DMS systems were in place and used an empirical Bayes (EB) study design ( 12 ) with three to four winter seasons of before data and three to six winter seasons of after data, depending on the location. The results showed a 32% reduction in winter fatal and injury crashes, which was statistically significant at a 95% confidence level.
Given the dearth of knowledge about the safety effects of the three ITS treatments and a need by agencies to estimate the changes in safety of specific applications, the research for this paper was aimed at developing, documenting, and illustrating a practice-ready procedure for addressing that need. Early in the research, it was revealed that conducting a robust before–after study would be an insurmountable challenge, given paucity of required data and the need for the results to reflect the reality that the safety effects of these treatments will vary with application circumstance. Therefore, it was decided to pursue a cross-sectional regression-type approach using data from treatment and non-treatment groups, recognizing the potential issues with this study design; these include inappropriate functional form, omitted variable bias, or correlation of variables that could lead to counterintuitive results in comparing the crash experience of two groups of sites or in making inferences about the safety effects of a variable ( 13 ).
Methodology
Overview
In pursuing a cross-sectional regression-type approach it was natural to first consider a more traditional approach of combining treatment and non-treatment groups and including an indicator variable for the treatment, and interaction terms between the treatment indicator and other site characteristics. However, the available data were not nearly extensive enough for such an approach. It was then decided to pursue an approach based on a procedure that was first developed and validated for estimating the expected changes in safety for a contemplated traffic signal installation ( 14 ). The procedure was later proposed and illustrated for roundabout installation ( 15 ).
The principle of the adopted procedure is as follows. Separate safety performance functions (SPFs) were estimated for sites with and without each of the three ITS treatments as a function of exposure and a set of other independent variables. Consistent with the Highway Safety Manual (HSM) ( 16 ) method for estimating the expected safety benefit of a planned countermeasure, the procedure is applied in concert with the EB methodology ( 12 ) if, as is likely the case, a crash history is available as a specific application circumstance. For untreated reference sites with higher-than-expected crash frequency, which are the sites that would typically get considered for a safety treatment, the EB method in essence will refine (increase) the SPF expected crash frequency to account for risk factors not included in the SPF. In so doing, it will estimate a safety benefit that is larger than that obtained by comparing predictions from the two sets of SPFs for treatment and non-treatment sites.
The EB methodology is used with the crash data for a recent period at a potential treatment site and the SPF prediction for the site to estimate the expected number of total and fatal plus injury (FI) crashes that would have occurred in the after period without treatment. The EB estimate for property damage only (PDO) crashes is then derived as the EB estimate for total crashes minus the EB estimate for FI crashes.
The appropriate treatment site SPF is then used to estimate the expected number of total and FI crashes that would occur if the treatment were implemented. The estimate for PDO crashes is then derived as the SPF estimate for total crashes minus the SPF estimate for FI crashes. The expected change in safety for a potential application of the treatment is the difference between the EB estimate and the treatment site SPF prediction.
Methodological Steps
An analyst may apply the procedure using the following steps, which are adapted from documentation in Kopitch and Saphores ( 8 ).
Step 1
Assemble appropriate SPFs (shown later) for equivalent non-treatment and treatment sites and the data required to apply the SPFs. For the past, say, 3–5 years, obtain the count of total and FI crashes, and for the same period obtain the annual average daily traffic (AADT) values. Estimate the AADTs that would prevail for the period after the treatment is installed,
- If the SPFs cannot be assumed to be representative of the jurisdiction, they must be calibrated using data from a sample of sites representative of that jurisdiction using the HSM calibration procedure.
Step 2
Use the EB methodology with the data from Step 1 and the SPF for non-treatment sites to estimate the expected number of total and FI crashes that would occur without treatment. The EB estimate for PDO crashes is then derived as the EB estimate for total crashes minus the EB estimate for FI crashes. The equations are as follows:
where
x = the count of crashes (total or FI) in n years
P = the SPF prediction for the given input data (total or FI crashes)
w1, w2 are weights estimated as follows:
where ф is the inverse overdispersion parameter estimated in the SPF calibration (these are presented later with the SPFs).
Step 3
Use the appropriate treatment site SPF and the AADTs from Step 1 to estimate the expected number of total and FI crashes that would occur if the treatment were implemented. The estimate for PDO crashes is then derived as the SPF estimate for total crashes minus the SPF estimate for FI crashes.
Step 4
Obtain for FI and PDO crashes, the difference between the estimates from Steps 2 and 3.
Step 5
Applying suitable severity weights and dollar values for FI and PDO crashes, obtain the estimated expected economic benefit of implementing the treatment.
Data
Datasets for estimating the SPFs were assembled for sites in Pennsylvania with the three treatments and for untreated reference sites for each. Unusable or unrealistic segments, such as segments with unrealistic AADT values or number of lanes, were filtered out. It was necessary to specify a buffer for the ITS treatments because they affect the surrounding area of a treatment location and not just the individual location. The research team identified a buffer around a specific treatment and assigned crashes in that buffer area to the ITS treatment for analysis. A 15 mi buffer was used for the DMS and RWIS treatments and a 1,500 ft buffer was used for CCTV treatments after preliminary analysis indicated that this afforded the best predictive ability for the SPFs developed. Summary statistics are provided in Tables 1 and 2. The research team was not able to access specific messages or general message categories displayed on the DMS at different times, so those data are not included in the summary statistics or the analysis. Note that the average AADTs for the treatment sites are higher than for the non-treatment sites, suggesting that the SPFs for treatment sites are likely more reliable for medium and higher AADTs than those for the non-treatment sites.
Number and Mileage of Sites and Proportion of Sites with Key Characteristics
Note: CCTV = closed-circuit television; DMS = dynamic message signs; RWIS = road weather information system; na = not applicable.
Data ranges for AADT and Segment Length
Note: AADT = annual average daily traffic; Avg. = average; CCTV = closed-circuit television; DMS = dynamic message signs; Max. = maximum; Min. = minimum; RWIS = road weather information system.
Safety Performance Functions (SPFS)
The following model form was used for estimating SPFs for treatment and non-treatment reference sites for each of the three treatments and for total and FI crashes:
where
L = segment length;
AADT = average annual daily traffic;
The Xs are segment characteristics.
α, τ, γ, and the βs are coefficients estimated from data in the regression analysis procedure.
Consistent with common practice, generalized linear modeling with a negative binomial (NB) error structure was used for model estimation, using the software R. An inverse overdispersion parameter of the NB distribution is also estimated in the model calibration process. As estimated with the R software, the larger the value of the inverse overdispersion parameter, the better the model is for the same estimation data.
Non-treatment reference sites were selected from similar locations where the given treatment had not been applied. The research team first used propensity score matching methods to better match the characteristics of treated and untreated sites, specifically in the case where there were fewer treatment sites than reference sites. This matching method first estimates propensity scores for each site, which is a probability between 0 or 1 that site will receive a treatment ( 17 ). For this study, propensity scores were calculated based on the following characteristics: AADT, segment length, total roadway width, divisor width, number of lanes, district, area type (e.g., urban or rural), speed limit, surface type, and if the site is on the National Highway System. After calculating propensity scores for each site, the treatment and non-treatment sites were matched based on a one-to-one nearest-neighbor approach, where each non-treatment site was match to a treatment site with the closest propensity score within 10% ( 17 ).
SPFs were then estimated for matched and unmatched data and the two sets of models compared. In the end, the SPFs with the unmatched data turned out to be superior based on goodness of fit measure and logic of the direction of effects for variables. This was likely because of the significantly reduced sample sizes for the matched data.
The estimated SPFs for the unmatched data summarized in Tables 1 and 2 are shown in Tables 3–5. It should be stressed that these are predictive models and are not intended for inferring causation or for quantifying the safety effects of individual variables for the reasons outlined in the introduction. It is also important, as emphasized in the HSM ( 16 ), and as noted earlier, that the SPFs be calibrated for application in jurisdictions other than Pennsylvania using procedures documented in the HSM.
Parameter Estimates (and Standard Errors) for Safety Performance Functions for Sites with and without CCTV
Note: AADT = annual average daily traffic; CCTV = closed-circuit television; FI = fatal plus injury.
Parameter Estimates (and Standard Errors) for Safety Performance Functions for Sites with and without DMS
Note: AADT = annual average daily traffic; DMS = dynamic message signs; FI = fatal plus injury; na = not applicable.
Parameter Estimates (and Standard Errors) for Safety Performance Functions for Sites with and without RWIS
Note: AADT = annual average daily traffic; RWIS = road weather information system; FI = fatal plus injury; na = not applicable.
CMFs Derived from Illustrative Applications of the Proposed Procedure
For these illustrative applications, which are provided to demonstrate that the results from applying the procedure are reasonable, a freeway with three lanes in one direction (Lanes >2) is being considered for one of the three treatments. The following parameters pertain:
Segment length = 1.2 mi
Lane width = 12.5 ft
Speed limit = 70 mph
Fixed barrier; Divisor width = 15 ft
Surface type is plain Portland cement concrete base
Rural area
It is also assumed for convenience that the SPFs presented earlier may be applied without calibration. Table 6 shows the CMF derivation for two levels of AADT (medium and high) and assumed crash frequencies for 5 years before treatment that are assumed to be roughly proportional to those AADTs. AADTs are assumed to increase by 5% between the before and after period. The implied CMFs are estimated by dividing the crashes/year expected with treatment by the crashes/year expected without treatment.
CMFs Derived from Illustrative Applications of the Proposed Procedure
Note: AADT = annual average daily traffic; CCTV = closed-circuit television; CMF = crash modification factor; DMS = dynamic message signs; FI = fatal plus injury; PDO = property damage only; RWIS = road weather information system.
The following is an example calculation for CMFs in Table 6. It pertains to a three-lane site with AADT of 40,000 being considered for CCTV with an estimated AADT of 42,000 after implementation. The crash counts (x) in the past n = 5 years were: Total = 13; FI = 6.
For convenience of presentation, some steps from the methodological overview are combined, so the steps are given letters instead of numbers to avoid confusion. It is also assumed for convenience that the appropriate SPFs in Table 3 may be applied without calibration.
Step (a)
First, the SPFs for non-treatment sites are used to predict the number of crashes by severity based on the parameter estimates in Table 3 for the constant term, speed limit, surface type, lane count, lane width, presence of a fixed barrier, divisor width, AADT, and segment length.
Inverse overdispersion parameter (ф) = 0.4884
Inverse overdispersion parameter (ф) = 0.4774
Next, the weights for the SPF prediction (P) and the crash counts (x) for the EB method, and the EB estimates are calculated. The SPF inverse overdispersion parameter (ф) is also used in these calculations.
Total crashes:
FI crashes:
Because AADT is expected to increase in the after period, albeit only slightly, an adjustment is made to the EB estimate (m) to account for this change. This factor is calculated from the AADTs and their calibrated exponents as follows:
The adjusted m is now equal to: 12.560 × 1.045 = 13.125 for total crashes.
For FI crashes:
The adjusted m is now equal to: 5.487 × 1.048 = 5.751 for FI crashes.
The expected number of crashes by severity at the site in future years if CCTV installation does not take place is estimated to be 13.125 total and 5.751 FI crashes, over a five-year period.
Step (b)
The treatment site SPF from Table 3 is used to predict the expected number of crashes should the CCTV be implemented. In this case, as noted, for convenience the SPF was deemed adequate and was not calibrated specifically for the jurisdiction. (Note: The future AADT is used in this calculation.)
Total crashes/year = e(-6.5088-0.3302-0.2528+0.2176+0.1927-0.1979-0.2367) × 42,0000.7296 × 1.20.5382 = 2.115. Over 5 years, this would be 2.115 × 5 = 10.574.
FI crashes/year = e(-8.6940-0.3915-0.2488+0.2655+0.2467-0.0880-0.2136)\×42,0000.8456×1.20.5514 = 0.979. Over 5 years, this would be 0.979×5 = 4.894.
The expected number of crashes by severity over 5 years at the site if CCTV were to be implemented is 10.574 total and 4.894 FI crashes.
Step (c)
The expected reduction in total crashes is equal to 13.125−10.574 = 2.551 crashes over 5 years.
The expected reduction in FI crashes is equal to 5.751−4.894 = 0.857 crashes over 5 years; the implied CMF is 4.894/5.751 = 0.85, shown in the first row of Table 6.
The expected reduction in PDO crashes is crashes 7.374−5.680 = 1.694 crashes over 5 years; the implied CMF is 5.680/7.374 = 0.77, shown in the first row of Table 6.
The CMFs in Table 6 illustrate the variability of CMFs with specific treatment application circumstances. One such CMF, for PDO crashes for RWIS for the lower AADT, indicates a potentially small increase in the minor crashes that is, in any case, compensated for by the larger decrease in the more severe FI crashes.
Conclusions
The research addressed a need identified in NCHRP Project 17-95 to develop CMFs for the various typically deployed ITS applications, DMS, CCTV, and RWIS. In so doing, the study provides an important contribution to the CMF knowledgebase by developing, documenting, and illustrating a procedure for estimating changes in crash frequency for specific application circumstances so that an agency can directly evaluate the change in safety that may be associated with a contemplated CCTV, DMS, or RWIS treatment. That procedure, which used an EB framework with SPFs for treatment and non-treatment reference sites, was developed and validated in previous research for evaluating the safety effects of traffic signal installation. Nevertheless, further validation for a wider range of site and treatment types would be useful and has been actually recommended as a research need by the TRB Safety Performance and Analysis Committee at the Transportation Research Board 2024 Annual Meeting.
Further research can also shore up the results from this project with the use of data from other jurisdictions to enhance the procedure developed for directly estimating the change in safety of a contemplated ITS treatment by testing its transferability and by increasing the robustness of the SPFs used in that procedure.
The procedure, as documented, is practice ready. However, potential users should be aware of caveats that result from some of its limitations. First, a fundamental assumption is that if a treatment that is evaluated with the procedure is implemented, then the site will have a safety experience with the treatment similar to equivalent ones used for the development of the treatment site SPFs. Second, the SPFs used in the procedure for treatment and non-treatment sites are intended for use in crash prediction with the EB methodology, so caution should be exercised in using them for comparison of the safety of the two site types. Finally, in accord with the HSM recommendations, the SPFs should desirably be calibrated for application in jurisdictions and time periods that are different from the ones for which there were developed.
Footnotes
Acknowledgements
Data for the research were provided by the Pennsylvania Department of Transportation. This research is part of the NCHRP Project 17-95, which is part of the National Cooperative Highway Research Program (NCHRP). NCHRP is administered by the Transportation Research Board (TRB) and funded by participating member states of the American Association of State Highway and Transportation Officials (AASHTO). NCHRP also receives critical technical support from the Federal Highway Administration (FHWA), United States Department of Transportation.
Author Contributions
The authors confirm contribution to the paper as follows: study conception and design: B. Persaud, R Srinivasan, V. Gayah; data collection: K. Kersavage, V. Gayah, R. Srinivasan; analysis and interpretation of results: V. Gayah, K. Kersavage, R. Srinivasan, B. Persaud, S. Hallmark, T. Saleem, C. Mohammadi; draft manuscript preparation: B. Persaud, R. Srinivasan, V. Gayah, K. Kersavage, S. Hallmark, T. Saleem, C. Mohammadi. All authors reviewed the results and approved the final version of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research funding was provided by the National Cooperative Highway Research Program (NCHRP) Project 17–95.
