Improving the Accuracy of the Spatial Transferability of Direct-demand Models for Bicycle Volume Estimation at Intersections

Abstract

Direct-demand (DD) models are used to estimate bicycle exposure (typically expressed as annual average daily bicycle volume [AADB]) when observed counts are unavailable at a site. The DD models typically estimate exposure as a function of site characteristics, such as the geometry, surrounding land use, and sociodemographic characteristics. Developing a DD model requires observed volume counts and associated site characteristics data for many sites in the target jurisdiction. In the absence of this data, it is desirable to apply a DD model from another jurisdiction. However, the naïve transferability of DD models results in AADB estimates with large errors. This paper investigates the use of calibration methods to enhance the spatial transferability of DD models to estimate bicycle volumes at intersections. This paper examined five DD models across four jurisdictions: (1) City of Milton (52 sites); (2) City of Toronto (28 sites); (3) Region of Waterloo (158 sites) in Canada; (4) Pima County (70 sites); and (6) Arizona, US, covering a total of 308 sites. Five local calibration techniques were evaluated for their effectiveness in mitigating errors in naïve estimates. The findings indicate that calibration, particularly regression-based methods, significantly improves the accuracy of the AADB predictions, with calibration Model 3 being the most effective for jurisdictions with less than 80 count sites ( $k < 80$ ). For $k = 5$ , Model 3 reduced error by 56%, and Models 3 and 5 achieved up to an 80% error reduction at $k = 30$ . As more sites became available, Model 5 emerged as the superior calibration method for $k \geq 80$ .

Keywords

annual average daily bicycle volume direct-demand models spatial transferability cyclist exposure local calibration Monte Carlo simulation

A measure of exposure is required as an input to most road safety analyses. Traditionally, only exposure of vehicles, frequently expressed as annual average daily traffic (AADT), has been used. However, with the increased focus on improving safety for vulnerable road users (VRUs, i.e., bicyclists and pedestrians), there is a need to have estimates of exposure for bicyclists and pedestrians as well as vehicles. When continuous counts (CCs) of VRUs are available, then the annual average traffic for VRUs can be directly calculated. When CCs are available only for a subset of sites and short-term counts (STCs) are available for the remaining sites, exposure can be estimated using a factor-based approach ( 1 , 2 ). For sites where neither CCs nor STCs are available, it has been proposed to apply direct-demand (DD) models ( 3 , 4 ), which estimate exposure as a function of site characteristics (e.g., geometry, surrounding land use, population demographics). In addition, statistical approaches such as DD models are common alternatives for estimating VRU exposure. The DD models have typically been regression models, but more recently, machine learning approaches have also been proposed ( 5 , 6 ). However, developing a DD model requires CCs from some sites in the jurisdiction, and studies suggest a minimum of at least 50–70 sites are required to develop a DD model ( 4 , 7 , 8 ). Furthermore, even when sufficient CCs are available, the local jurisdiction may not have the resources or expertise to develop the DD models. Using an existing DD model developed in another jurisdiction (i.e., spatially transferring) can be a practical alternative for estimating VRU exposure in the target jurisdiction ( 4 ).

The spatial transferability of existing DD models has been examined for pedestrians ( 4 ) and cyclists ( 9 ). The findings show that naïvely transferring DD models from other jurisdictions can result in large estimation errors (expressed as the average absolute error between the estimated and true counts for all sites in a jurisdiction divided by the average true count across all sites in the jurisdiction) from 0.52 to 50 for pedestrian exposure ( 4 ) and from 0.41 to 31.9 for bicyclist exposure ( 9 ). Furthermore, Soldouz and Hellinga ( 9 ) showed that naïvely transferring DD models developed in one jurisdiction to estimate bicyclist exposure in another jurisdiction produced estimation errors (quantified as the root mean squared exposure estimation error [RMSE] from 7–600 times larger than the RMSE associated with the original jurisdiction from which the DD model was developed. Significantly, Soldouz and Hellinga ( 9 ) also showed that there is only a weak correlation between measures of similarity between the development and target jurisdictions and the performance accuracy of the spatially transferred model. Therefore, selecting a DD model from a development jurisdiction that has similar levels of bicycling activity or similar site characteristics does not guarantee that the model will perform well in the target jurisdiction.

Sobreira and Hellinga ( 10 ) investigated and evaluated methods to improve the spatial transferability of DD models for estimating pedestrian exposure and found that these could improve (reduce) estimation errors by between 4.5% and 65% (compared with the naïve transferability method). Inspired by this previous work, the goal of this paper is to evaluate methods for improving the naïve transferability of existing DD models for bicycle exposure estimation at intersections. The next section summarizes the literature review. The Methodology section describes the estimation procedure for naïve transferability and the assessment of selected calibration methods. This is followed by the Results and Discussion section. Finally, the last section presents the conclusions and recommendations for future studies.

Literature Review

Spatial transferability in transportation engineering has been primarily explored in mode choice modeling ( 11 – 13 ). Transferability frameworks can be categorized in various ways, with the most common approach being naïve transferability. Although naïve transferability generally results in lower estimation accuracy, it can serve as a useful benchmark for evaluating other transferability calibration approaches ( 10 , 11 , 13 ).

Another major category is scaling approaches, with the most common being the adjustment of the original model’s parameters, estimates, and constants. Another scaling approach that has been used to transfer an already developed model involves retaining all parameters from the original model and adjusting the constant to calibrate it locally—also known as the alternative-specific constant ( 11 , 14 ). These approaches are practical because a locally calibrated model does not require collecting a complete data set from multiple sources to develop a model from scratch. Instead, it only requires data relevant to the already developed model, making the process more cost- and time-efficient.

This literature review section is divided into two sections. The first discusses previous work related to the spatial transferability of DD models for estimating VRU exposure. The second section presents DD models for estimating bicycle exposure.

Previous Work on Spatial Transferability of DD Models for Estimating VRU Exposure

Several studies have explored the spatial transferability of DD models used to estimate active transportation volumes, yielding mixed results ( 9 , 15 – 18 ). Wang et al. ( 15 ) found that DD models developed for Minneapolis, MN, and Columbus, Ohio, performed poorly when applied to the other city, with error rates 1.7 times higher than in their original contexts. More recent studies ( 16 , 18 ) explored the effect of combining static and crowdsourced data on model accuracy, finding that models incorporating both data types reduced error rates but still showed significant increases (up to 1.6 times) when applied across different locations or years within the same jurisdiction.

Existing research on the spatial transferability of DD models reveals mixed results, with transferred models showing errors 1.0–1.7 times higher than when applied within their original jurisdictions. These studies often rely on limited STCs to estimate the AADB, introducing inaccuracies, and use varied data sources that combine multi-use paths and roadway segments. Temporal and spatial transferability are frequently assessed together, complicating the isolation of spatial factors. Addressing these gaps, Soldouz and Hellinga ( 9 ) analyzed the transferability of five DD models across four jurisdictions using true AADB values, finding that applying models naively across regions leads to substantial estimation errors.

One study examines improving the naïve transferability of existing DD models for estimating annual average daily pedestrian traffic (AADPT) ( 10 ). This study proposed calibration methods that serve as a benchmark for this paper. That study proposed and evaluated five calibration methods to improve the naïve transferability of three DD models for estimating the AADPT. The three DD models used in this paper are in log-linear or negative binomial format as depicted in Equation 1.

\ln (AADPT) = ρ_{0} + ρ_{1} X_{1} + \dots + ρ_{n} X_{n}

(1)

where

$ρ$ = model coefficients, and

$X$ = explanatory variables.

The five local calibration models evaluated by Sobreira and Hellinga ( 10 ) are presented and described as follows.

Calibration Model 1

δ = \frac{1}{J} \sum_{i = 1}^{J} \frac{\ln (V_{i})}{\ln ({\hat{V}}_{i})} ∴ \ln ({\hat{V}}_{m = 1, i}) = δ \times \ln ({\hat{V}}_{m = 0, i})

(2)

Calibration Model 2

δ = \frac{\sum_{i = 1}^{J} \ln (V_{i})}{\sum_{i = 1}^{J} \ln ({\hat{V}}_{m = 0, i})} ∴ \ln ({\hat{V}}_{m = 2, i}) = δ \times \ln ({\hat{V}}_{m = 0, i})

(3)

Calibration Model 3

\ln ({\hat{V}}_{m = 3, i}) = α \times \ln ({\hat{V}}_{m = 0, i}) + β

(4)

Calibration Model 4

\ln ({\hat{V}}_{m = 4, i}) = α \times \ln {({\hat{V}}_{m = 0, i})}^{β}

(5)

Calibration Model 5

\ln ({\hat{V}}_{m = 5, i}) = α_{0} γ_{0} + α_{1} γ_{1} q_{1, i} + \dots + α_{n} γ_{n} q_{n, i}

(6)

where

$δ$ = calibration factors for model $m$ =1–2,

$α, β, and γ$ = calibration regression coefficients for model $m$ =3–5,

$J$ = total number of intersections in a jurisdiction,

$V_{i}$ = true AADB for intersection $i$ ,

${\hat{V}}_{m, i}$ = AADB estimate for intersection $i$ using local calibration models $m$ =1–5, and

$m = 0$ (base estimation without local calibration).

The accuracy of the model estimates of the AADPT was quantified for each jurisdiction for the mean absolute error (MEA). (Equation 7), and the error indicator $E$ (which is the mean absolute error [MAE]. divided by the average true AADB [Equation 8]).

{MAE}_{m} = \frac{\sum_{i = 1}^{J} | V_{i} - {\hat{V}}_{m, i} |}{J}

(7)

E_{m} = \frac{MA E_{m}}{\frac{1}{J} \sum_{i = 1}^{J} V_{i}}

(8)

There are two key findings from the study ( 10 ) that are relevant to this paper.

Locally calibrating DD models for estimating pedestrian exposure resulted in a substantial improvement in estimation accuracy (average of 35% reduction in estimation error).

Local calibration Models 2 and 3 were the best, with Model 2 performing better when CCs are available for less than 2.5% of all sites in the target jurisdiction and Model 3 otherwise.

DD Models for Estimating Bicycle Exposure

Building on the insights gained from pedestrian exposure modeling, Soldouz and Hellinga ( 9 ) reviewed existing DD models for estimating bicycle exposure. The goal of the review was to identify DD models that could be used to evaluate the spatial transferability of models. Therefore, only those models that were developed in jurisdictions within Canada or the US and contained explanatory variables for which values could be retrieved from open source data sets, such as census data, land use, and network data, and did not require STCs or crowdsourced information were considered. Five DD models were found that satisfied the previous criteria, and they are summarized in Table 1.

Table 1.

Summary of Selected Bicycle Volume Estimation Direct-Demand Models

Model	Dependentvariable	Independent variable	Definition	Coefficient
Griswold et al. ( 19 ) California, US 81 sites	ln( $V_{2 hr}$ ) ^a	Constant		–1.127
		Bike_mark	Presence of bicycle markings on any approach(1 = yes, 0 = no)	0.459
		Avg_slope	Average slope in degrees within an 800 m(0.5 mi) radius	–0.470
		CNRH	Connected node ratio within an 800 m (0.5 m) radius	4.634
		Count_2009	Binary variable indicating whether the count was conducted in 2009 (1 = yes, 0 = no)	1.036
Strauss et al. ( 20 )Montreal, Canada647 sites	$\ln (AADB)$	Constant		–0.227
		Emp	Number of employment (in thousands) in a400 m (0.25 mi radius buffer)	0.285
		Sch	Presence of school in 400 m buffer (1 = yes, 0 = no)	0.288
		Metro	Number of metro (subway) stations in an 800 m buffer	0.218
		Land	Land use mix index (between 0 and 1) in an 800 m buffer	0.173
		BL	Length of bicycle facilities (km) in an 800 m buffer	0.139
		Com	Commercial land use area (in thousands of m²) in a50 m buffer	0.015
		Three_leg	Intersection has only three legs (1 = yes, 0 = no)	-0.572
Strauss and Moreno ( 21 )Montreal, Canada758 sites	$\ln (AADB)$	Constant		2.592
		Emp	Number of employment (in thousands) in a 400 m buffer	0.378
		Sch	Presence of school in 400 m buffer (1 = yes, 0 = no)	0.251
		Metro	Number of metro (subway) stations in an 800m buffer	0.256
		Land	Land use mix index (between 0 and 1) in an 800 m buffer	1.244
		BL	Length of bicycle facilities (km) in an 800 m buffer	0.070
		Inc	Average annual income (in thousands of dollars) in a50 m buffer	0.005
		B_Ln	Presence of a bike lane on any approach(1 = yes, 0 = no)	0.451
		Cycle_Tr	Presence of a cycle track on any approach(1 = yes, 0 = no)	0.989
		S_Len	Average street length (km)	–0.016
		Park	Presence of parking entrance in 50 m buffer(1 = yes, 0 = no)	–0.301
		Bus	Number of transit bus stops in a 150 m buffer	0.087
Tabeshian and Kattan ( 22 )Calgary, Canada34 sites	$V_{2 hr}$ ^b			MLR	Poisson
		Constant		NA ^c	1.577
		Com	Hectares of commercial area in a 161 m (0.1 mi buffer)	2.03	0.156
		Res	Hectares of low-density residential area in a161 m (0.1 mi) buffer	2.25	0.1413
		Inst	Hectares of institutional area in an 800 m(0.5 mi) buffer	0.664	0.0341
		Lanes	Number of street lanes approaching the intersection	–0.504	–0.02
		Bus	Number of transit bus stops in a 400 m (0.25 mi) buffer	1.218	0.059

Note: AADB = average daily bicycle volume.

^a Dependent variable $V_{2 hr}$ = 2 h cyclist count (time during day of 2-h period not specified and varied by site).

^b Dependent variable $V_{2 hr}$ = 2 h PM peak (04:00–06:00 p.m.). Two direct-demand models: (1) multiple linear regression (MLR); and (2) Poisson, were developed.

^c Not applicable, the MLR model has no constant.

Four of the five selected models are log-linear models with a structure as depicted in Equation 9, and the remaining multiple linear regression (MLR) model by Tabeshian and Kattan ( 22 ) has a linear structure as depicted in Equation 10.

\ln (\hat{V}) = ρ_{0} + ρ_{1} X_{1} + ρ_{2} X_{2} + \dots + ρ_{n} X_{n}

(9)

\hat{V} = ρ_{0} + ρ_{1} X_{1} + ρ_{2} X_{2} + \dots + ρ_{n} X_{n}

(10)

where

$\hat{V}$ = dependent variables (bicycle exposure estimation),

$ρ_{0}$ = intercept,

$ρ_{1}, ρ_{2}, \dots, ρ_{n}$ = coefficients, and

$X_{1}, X_{2}, \dots, X_{n}$ = explanatory variables (e.g., socioeconomic, network, and land use characteristics near the site)

The next section describes the empirical data used to quantify the AADB estimation accuracy when naïvely transferring these five DD models to other jurisdictions. These results are the basis for assessing the accuracy improvements that can be provided by local calibration of spatially transferred DD models.

Data Description

A total of 308 signalized intersections were selected as target jurisdictions from the following locations: (1) 52 from the City of Milton, ON, Canada; (2)28 from the City of Toronto, ON, Canada; (3) 70 from Pima County, Arizona, US; and (4) 158 from the Region of Waterloo, ON, Canada. The bicycle volumes vary across these four jurisdictions. The CCs were obtained for a full year, between July 2023 and June 2024, by the same vendor for all four target jurisdictions, offering minute-by-minute bicycle counts for each turning movement and crossing at intersections using a camera-based data collection system. To validate bicycle count accuracy, manual counts and video recordings were collected during peak hours (04:30–6:30 p.m.) at a busy intersection in Toronto with an AADB = 4,000. Two analysts independently reviewed the video to produce manual counts, which averaged 672.5 and were the ground truth. The corresponding count from the vendor’s camera-based continuous count system was 675, demonstrating over 99% accuracy. This verification was limited to a single location and did not test different conditions, such as weather, intersection configuration, or sites with lower AADBs; the results suggest that the camera-based counts used in this paper are reliable for this analysis.

The data were aggregated to provide daily (24 h) counts for each day at each intersection, including all approaches. The counting system only reports data when a count is registered. Consequently, there is no distinction between instances where the system is not operating and when no users are traversing the intersection, as in both cases, no counts are reported. Therefore, in this paper, sites were included if: (1) the camera-based system was installed for the full 12-month period; and (2) there were no large gaps in the data collection (i.e., sites were excluded if no vehicle traffic was detected for more than four consecutive days). The true AADB count for each intersection was calculated using Equation 11.

{\bar{V}}_{i} = \frac{\sum_{n = 1}^{d_{i}} v_{i}}{d_{i}}

(11)

where

$v_{i}$ = total daily bicycle volume for intersection $i$ on day $n$ ,

$d_{i}$ = total number of valid days for site $i$ , and

${\bar{V}}_{i}$ = true AADB for site $i$ .

Table 2 provides descriptive statistics for the true AADB of the selected intersections across the four target jurisdictions. On average, the sites in Pima County exhibit the lowest level of bicycling activity (22 bicyclists/day), followed by the City of Milton (54/day), the Region of Waterloo (84/day), and the City of Toronto sites with a very high average of 2,511 bicyclists/day.

Table 2.

Descriptive Statistics of AADB for Target Jurisdictions

Jurisdiction	Number ofintersections	Mean	Minimum	Maximum	Standarddeviation
City of Milton	52	54	5	139	33
City of Toronto	28	2,511	94	6,732	1,999
Pima County	70	22	2	82	20
Region of Waterloo	158	84	2	453	86

Note: AADB = annual average daily bicycle volume.

For Canadian jurisdictions, the 2016 census data was the source for demographic information and variables, and the 2022 census for Pima County, Arizona, US. To extract the explanatory variables provided in Table 1 for the target jurisdictions, open source data sets were obtained, including OpenStreetMap ( 23 ) data for network and land use variables in addition to open sources from target jurisdictions ( 24 – 28 ). The following section outlines the methodology used for estimating the AADB using the selected DD models, as well as the application and assessment of the chosen calibration models.

Methodology

Figure 1 shows the methodology for this study. First, the dependent variable from the selected DD models in the literature is examined. If the dependent variable is the AADB, the next step is to calculate the error metrics to assess the accuracy of the DD models as base models (naïve transferability). For dependent variables in other formats, such as 2-h PM counts, they need to be estimated as a form of AADB. The average ratio of the true AADB to the average 2-h PM counts is calculated for each site across all four target jurisdictions, then multiplied by the 2-h estimate from the DD models. Then, the error metrics must be calculated. In the next step, the selected calibration models from the literature ( 10 ) are applied using a Monte Carlo simulation. Finally, the error metrics are recalculated and compared with the base naïve transferability estimation of the AADB.

Figure 1.

Methodology flowchart.

Step 1: Naïve Application of the DD Models

In the first step, all five DD models from Table 1 are applied to the four target jurisdictions to obtain estimates of bicycle volumes for all sites.

Two of the models (Strauss et al. [ 20 ] and Strauss and Moreno [21]) are structured to estimate bicycle volume as the AADB, so no further processing of these estimates is required. However, the remaining three DD models are structured to estimate exposure as bicycle volume in 2-h. For these three models, further processing, as part of Step 2, is required to estimate the AADB.

Step 2: Expansion of 2-h Volume Estimates to AADB

The two DD models developed by Tabeshian and Kattan ( 22 ) used data from Calgary, Canada, which included bicycle volume counts collected over 2 h from 04:00 to 06:00 p.m. across 34 sites. This paper reported a mean 2-h PM volume of 14.3 bicyclists but did not provide information on how this value can be scaled to estimate the AADB. To estimate the AADB, the average ratio of the AADB to the average 2-h bicycle PM volume for all sites across the four target jurisdictions was calculated, as shown by Equation 12. The average ratio was 7.26. This ratio was applied to each site’s 2-h volume estimate from the DD model to estimate the associated AADB.

R_{i} = \frac{1}{t} (\sum_{s = 1}^{t} \frac{v_{s}}{v_{s, 2 hr}})

(12)

where

$R_{i}$ = ratio of AADB to 2-h bicycle counts for jurisdiction $i$ ,

$v_{s}$ = AADB counts for site $s$ in jurisdiction $i$ ,

$v_{s, 2 hr}$ = annual average 2-h bicycle counts for site $s$ in jurisdiction $i$ , and

$t$ = total number of sites in a jurisdiction.

The Griswold DD model ( 19 ) was developed using 2-h bicycle counts from 81 sites in Alameda County, California, US. The reported mean 2-h count was 35.8; however, these counts were recorded at various times of the day. The ratio of the 24-h volume to the 2-h volume for all 2-h periods in the day was computed using data from all sites in each jurisdiction. It was assumed that the counts were taken during the daytime, and the variation in the ratios for each 2-h period was examined. The ratio did not vary substantially between 08:00 a.m. and 06:00 p.m., and therefore the average ratio computed across these time periods (average ratio = 7.78) was used. This ratio was used to scale all the Griswold model estimates from 2-h volumes to the AADB.

Step 3: Compute the AADB Estimation Accuracy for Naïve Transferability of DD Models

In Step 3, the accuracies of the five DD models (listed in Table 1) applied to the four jurisdictions are quantified using the two error metrics adopted by ( 10 ) ( ${MAE}_{m}$ from Equation 7 and $E_{m}$ from Equation 8), the Geoffery E. Havers (GEH) metric., adopted by ( 9 ) (Equation 13) and the root mean squared error (RMSE). adopted by ( 19 ) (Equation 14).

{GEH}_{i} = \sqrt{\frac{2 {({\hat{V}}_{i} - V_{i})}^{2}}{{\hat{V}}_{i} + V_{i}}}

(13)

RMSE = \sqrt{\frac{\sum_{i = 1}^{J} {({\hat{V}}_{i} - V_{i})}^{2}}{J}}

(14)

where

$V_{i}$ = true AADB volume for intersection $i$ ,

${\hat{V}}_{i}$ = estimated AADB for intersection $i$ , and

$J$ = total number of intersections in a jurisdiction.

The GEH. performance metric, named after its creator, Geoffry E. Havers, considers the absolute difference and the percentage difference between the observed and estimated counts. It has been frequently used in applications, such as evaluating the adequacy of transportation simulation and urban transportation demand model estimates of link volumes. In these contexts, the desired threshold is that 85% of links have a GEH value ≤ 5.

Step 4: Compute the AADB Estimation Accuracy for Locally Calibrated DD Models

The five selected local calibration models ( 10 ) are described in the literature review section (Equations 2–6) and adapted for application to the five AADB DD models in Table 1. In the literature ( 10 ), the DD models were all log-linear models (i.e., the dependent variable is ln(AADPT), as in Equation 1). However, only four of the five AADB DD models in Table 1 are log-linear, and the remaining model (MLR model from Tabeshian and Kattan [22]) directly estimates bicycle exposure (the dependent variable is not ln-transformed). Therefore, the local calibration models were modified from Equations 2–6 to the non-log-transformed form of the AADB for application in the MLR model.

The local calibration models rely on the AADB values calculated from a set of sites in the local jurisdiction for which CCs are available. Consequently, the results are a function of the number of sites k in this set. Furthermore, the results will be impacted by which sites are in the local calibration set. Consequently, the evaluation method utilizes a Monte Carlo simulation process. For each replication, k random sites in the local jurisdiction are selected as local calibration sites for which the true AADB is known. The local calibration models (Equations 2–6) in combination with the five DD models are applied using this set of local calibration data to produce estimates of the AADB for each site in each jurisdiction. For each value of k, 300 replications of the Monte Carlo simulation were performed. The value of $k$ varies from 5–10 in increments of one and then in increments of five up to the maximum number of sites within each target jurisdiction.

In practical applications, despite the methods used for estimating AADBs, such as expansion factors with short-term and CCs or DD models, unrealistic estimates (i.e., significant over- or underestimations) may occur. A practical approach for identifying and controlling these extreme errors is to establish and apply upper and lower thresholds to the estimation result, and this approach is adopted in this step. The lower threshold is 0.001 across all jurisdictions, as the AADB must be non-negative, and values of zero result in divide-by-zero errors from some local calibration models. In practice, the selection of an upper threshold would be informed by local knowledge of the level of bicycle activity in the target jurisdiction. Consequently, the following upper thresholds on the AADB estimates were applied: (1) City of Toronto = 10,000; (2) City of Milton = 1,000; (3) City of Waterloo = 1,000; and (4) Pima County = 500. If a model estimate of the AADB exceeded the upper threshold, it was replaced with the upper threshold; if it was less than the lower threshold, it was replaced with the lower threshold. The maximum threshold values were selected to exceed the maximum observed AADB for each jurisdiction (Table 2) to not artificially improve prediction accuracies by selecting threshold values equal to the maximum observed AADB values.

The mean and median of each of the four error metrics from the 300 replications were recorded for each combination of the DD model, local calibration model, jurisdiction, and k.

Step 5: Establish the Best Local Calibration Method and Quantify the Associated Performance Improvement

The last step of the assessment methodology is to compare the AADB estimation accuracy of the DD models without local calibration (i.e., naïve transferability) and with local calibration. This is achieved by computing the percentage improvement in the error metrics, as shown in Equation 15. Note that for the error metrics $E$ and RMSE., improvement is obtained by reducing the error metric and for the GEH. (percentage of sites with GEH .≤ 5), improvement is obtained by increasing the error metric, and this is reflected in the two different calculations for Δ in Equation 15. Positive values of $Δ$ indicate that the local calibration model has improved AADB estimation accuracy by reducing $E$ and RMSE., while increasing the percentage of sites with GEH ≤ 5. $GEH$

Δ = {\begin{matrix} \frac{e_{naive} - e_{m}}{e_{naive}} \times 100 % for RMSE and E \\ \frac{e_{m} - e_{naive}}{e_{na \cdot\cdot ve}} \times 100 % for GEH \end{matrix}

(15)

where

$Δ$ = percent improvement in the error metric of AADB model estimation accuracy,

$e_{m}$ = error metric when using local calibration model $m$ for AADB estimates, and

$e_{naive}$ = error metric when using naïve application of DD model for AADB estimates.

The next section presents and discusses the results.

Results and Discussion

This section of the paper is organized into three subsections, providing results from methodology Steps 2, 4, and 5, respectively.

Naïve Transferability of DD Models (Step 2 Results)

Table 3 provides the performance metrics for each DD model applied to each target jurisdiction. These results indicate that the spatial transferability performance of the five DD models varies widely across the four target jurisdictions and that the spatial transferability performance is quite poor. The average MAE. (Equation 7) is 623.9, with a minimum of 128.2 for Pima County and a maximum of 2,069.2 for the City of Toronto. The average error (Equation 8) $E$ = 2.8 across all four target jurisdictions and development DD models, from 0.8 to 5.8 across four target jurisdictions, meaning that the mean absolute AADB estimation error was from 80% to 580% of the mean true AADB in the jurisdiction. None of the DD models met the condition of having 85% of model estimates with ${GEH}_{i} \leq 5$ across four target jurisdictions. The RMSE. (Equation 14) averaged 777.9 and varied from 137.5 to 2,630.7 across the four target jurisdictions.

Table 3.

Performance Summary of Naïve Transferability of DD Models Applied to Target Jurisdictions

		Direct-Demand Models
Error indicator	Target jurisdiction	Griswoldet al. ( 19 )	Strauss andMoreno ( 21 )	Strausset al. ( 20 )	Tabeshian and Kattan ( 22 ) MLR	Tabeshian and Kattan ( 22 ) Poisson	Average
$MAE$	Milton	63	30	590	39	27	149.8
	Toronto	2,436	1,883	1,055	2,486	2,486	2,069.2
	Pima County	54	48	475	26	38	128.2
	Waterloo	66	50	522	55	51	148.8
	Average	654.7	502.7	660.5	651.5	650.5	623.9
$E$	Milton	1.16	0.56	10.92	0.72	0.5	2.8
	Toronto	0.97	0.75	0.42	0.99	0.99	0.8
	Pima County	2.47	2.19	21.59	1.2	1.75	5.8
	Waterloo	0.79	0.6	6.22	0.66	0.61	1.8
	Average	1.34	1.02	9.78	0.89	0.96	2.8
Percent of sites with $GEH \leq 5$	Milton	44.2	69.2	0	44.2	67.3	45.
	Toronto	7.14	14.3	14.3	3.57	0.	7.8
	Pima County	32.9	28.6	0	51.4	47.1	32
	Waterloo	41.13	53.2	0	41.7	58.8	38.9
	Average	31.3	41.3	3.6	35.2	43.3	30.9
RMSE	Milton	78.6	39.7	627.2	50.9	35.3	166.3
	Toronto	3,066.5	2,509.4	1,366.5	3,109.7	3,101.7	2,630.7
	Pima County	66.9	57	470.7	44.1	48.8	137.5
	Waterloo	88.2	75.1	549.6	85.8	86.4	177.1
	Average	825.1	670.3	753.5	822.6	818.1	777.9

Note: RMSE = root mean squared error; GEH = Geoffrey E. Havers measure; E = error indicator; MAE = mean absolute error; MLR = multiple linear regression. Bold text to highlight the average differences observed across the various performance metrics.

Compute the AADB Estimation Accuracy for Locally Calibrated DD Models (Step 4 Results)

The substantial estimation errors observed in the naïve transferability results highlight the need for better approaches to adapt DD models in a local jurisdiction. Figure 2 shows the effectiveness of the local calibration models for improving the accuracy of the DD models. For each graph, the y-axis represents the mean error indicator $E$ (average across all 300 replications) and the x-axis represents the number of sites . from which the AADB values are available to conduct the local calibration. Each graph corresponds to a unique combination of a DD model and a target jurisdiction. The value of $E$ for the naïve application of the DD is shown as a solid black horizontal line and is also noted at the top of each graph. For some combinations of DD, target jurisdiction, local calibration model, and value of $k$ , the value of $E$ is very large. To aid in the interpretation of the results as a function of $k$ and to permit comparisons between graphs, the y-axis scale remains constant for all graphs, and the maximum value is 2.5. Cases for which the mean $E$ is larger than 2.5, do not appear on these graphs. The number of sites for each jurisdiction is given in parentheses after the jurisdiction name.

Figure 2.

Performance of each direct-demand (DD)model (mean E over 300 replications) as a function of target jurisdiction, local calibration method, and number of local calibration sites (y-axis limited to 2.5).

Several observations can be made from the results shown in Figure 2.

In almost all cases, locally calibrating a DD model substantially improves the accuracy of the estimated AADB values.

The amount of improvement varies by local calibration model and by target jurisdiction.

There are some cases (especially for lower values of $k$ ) for which the local calibration results are worse compared with the naïve method.

Local calibration Models 2, 3, and 5 appear to perform much better than local calibration Models 1 and 4.

The performance of some of the local calibration models (particularly Model 5) is highly sensitive to the number of sites $k$ for which local calibration data are available. When $k$ is small, the local calibration model performs poorly. This suggests that the best local calibration model to use may be a function of the number of sites for which local calibration data are available.

Establish the Best Local Calibration Method and Quantify the Associated Performance Improvement (Step 5 Results)

Because the results from Step 4 demonstrated notable accuracy improvements, the next step was to identify which calibration model performs best and to quantify the magnitude of improvements. Figure 3 shows the overall performance of the local calibration models across all DD models, based on the mean and median values of the error indicator $E$ (Equation 8). The y-axis represents the error indicator $E$ , and the x-axis denotes the number of sites .. The charts on the left side compare the mean values across all DD models for each jurisdiction, and the charts on the right side show the results based on the median values. The black lines represent the original naïve transferability values of $E$ .

Figure 3.

Performance of all direct-demand (DD) models (mean and median of E over 300 replications) as a function of target jurisdiction, local calibration method, and number of local calibration sites (y-axis limited to 2.5).

Calibration Model 3 provides lower mean values of the error indicator $E$ than the other local calibration models for $k$ values of approximately 5–50, depending on the jurisdiction, and Calibration Model 5 outperforms the others for higher $k$ values. However, the improvement in performance provided by Calibration Model 5 versus Calibration Model 3 is not large and may not provide sufficient benefit, given the difficulty in determining, for a given DD model and target jurisdiction, the threshold value of $k$ after which, Model 5 is the better choice.

All performance comparison results for calibration methods are based on 300 replications. In practice, a jurisdiction will only apply the local calibration model once using the data from available sites. Therefore, the mean or median of a larger number of replications may not be meaningful. Instead, another analysis was conducted to determine the average probability of achieving a lower error indicator $E$ (Equation 8) using a local calibration model compared with the naïve transferability, calculated across simulation repetitions for all jurisdictions and calibration models. Figure 4 shows these results, with the y-axis representing the average probability of each calibration model yielding a lower error indicator $E$ compared with naïve transferability, and the x-axis shows the $k$ values. As the maximum $k$ varies across jurisdictions, the vertical dashed line indicates when a jurisdiction is excluded from the average calculation. The findings support the previous observation that applying Calibration Model 3 improves error indicator $E$ for lower $k$ values, and calibration Model 5 performs best for higher $k$ values (greater than 70). However, the superior performance of calibration Model 5 is not significantly better than Model 3 for the average probability of achieving a lower $E$ compared with naïve transferability. Therefore, similar to the average $E$ comparison, Model 3 remains the best option for improving naïve transferability in the AADB estimation. Furthermore, as shown in Figure 4, when $k = 10$ , there is approximately an 80% likelihood that the use of local calibration Model 3 provides better results than the naïve application of the DD model, and for $k = 30$ , this probability increases to approximately 94%.

Figure 4.

Average probability over 300 replications that local calibration provides a lower error $E$ than the naïve application of the direct-demand (DD) model (averaged across all DD models and all target jurisdictions).

In Step 5 of the methodology, the percentage improvement in error metrics (Equation 15) was calculated. Figure 5 shows the percent improvement in the error indicator $E$ averaged across all jurisdictions for different values of $k$ . Local calibration Models 3 and 5 show a general trend of increasing improvement as $k$ increases. The trend exhibits discontinuities because the number of jurisdictions over which the average performance improvement is computed does not remain constant for values of $k$ .

Figure 5.

Percentage improvement ( $Δ$ ) in $E$ versus naïve application of the direct-demand (DD) model.

Even for a small value for $k$ ( $k = 5$ ) Calibration Model 3 provides an average performance improvement (reduction in error $E$ ) of 56% compared with the naïve transferability. In contrast, the use of local calibration Model 5 results in a performance that is worse than the naïve application of the DD models. However, Model 5 also shows the greatest improvement in effectiveness as $k$ increases and provides a performance that is similar to Model 3 when $k$ is greater than about 30.

Figure 5 shows the performance improvement obtained from the use of local calibration models when only considering the error metric $E$ . Based on the error metric $E$ , the results suggest that local calibration Model 3 is best for values of $k$ up to at least 30 and maybe as high as 70, with average performance improvements from approximately 55% to 80%. However, do the other error metrics (i.e., $MAE$ , $RMSE$ , and $GEH$ ) provide similar results?

Figure 6 shows the optimal local calibration model (i.e., the model that provides the largest average performance improvement Δ across the four jurisdictions versus the naïve application of the DD models) for each value of $k$ for each error metric. Two observations can be made from the results.

Despite the error metric considered, local calibration Models 3 and 5 perform better than Models 1, 2, and 4 because these three models are never the best models for any error metric or value of $k$ .

The results show that Model 5 is best according to all four performance metrics for $k \geq 80$ . For all values of $k < 80$ Model 3 is best according to the RMSE, and for most values of $k < 80$ , Model 3 is best according to MAE and $E$ . It is only the metric GEH that indicates Model 5 is best for $k > 8 .$ However, an examination of the average percent of sites with $GEH$ ≤ 5 (averaged across the four jurisdictions) as a function of $k$ (Figure 7) shows that Models 3 and 5 provide very similar results, with Model 5 performing marginally better when $k > 15$ . However, neither model provides results that meet the desired threshold of 85%. These results suggest that the $GEH$ metric may not be as important as the other three metrics for assessing the local calibration models.

Figure 6.

Best local calibration model for each error metric as a function of $k$ .

Figure 7.

Average percentage of sites with GEH ≤ 5 as a function of $k$ .

Conclusions

This paper examines whether calibration methods can enhance the naïve transferability of existing DD models for AADB estimation. Drawing on calibration methods from a similar study on pedestrians, these techniques were applied across four target jurisdictions, covering 308 sites: (1) City of Toronto (28); (2) City of Milton (52); (3) Region of Waterloo (158); and (4) Pima County (70). Five existing DD models were the base for naïve transferability and subsequent calibration tests. Following an initial examination of naïve transferability, two threshold filters were introduced to address extreme overestimations and underestimations in the AADB estimation. The lower threshold was set at 0.001 across all four target jurisdictions, and the upper threshold varied depending on the jurisdictions. The findings demonstrate that these thresholds, particularly for lower site counts (i.e., $k = 5 - 20$ when applying calibration methods), are crucial in mitigating the impact of unrealistic AADB estimation results.

The results of this paper suggest that calibration, particularly regression-based methods, provides a substantial improvement in the accuracy of AADB estimates derived from spatially transferred DD models. Across all four target jurisdictions, Calibration Model 3 consistently outperformed others for lower site counts k, providing a 56% reduction in the error indicator $E$ when only five bicycle count sites were available ( $k = 5$ ). The improvement increased as $k$ increased, although the trend exhibited discontinuities as the number of jurisdictions considered in the calculations changed. For $k = 30$ , Models 3 and 5 provide approximately an 80% improvement (reduction) in the error indicator $E$ relative to the naïve application of the DD models. The results also showed that as more count sites are available for local calibration, Model 5 became the superior local calibration method. However, the value of $k$ at which Model 5 became better than Model 3 was different across the four jurisdictions and varied with the error metric considered. Considering the average performance across the four jurisdictions and all four error metrics, Model 5 was the superior choice for $k \geq 80$ . For values of $k < 80$ , Model 3 was generally the superior choice.

The main finding of this paper is that locally calibrating DD models substantially improves the accuracy of AADB estimates from spatially transferred DD models. This finding is supported by investigating different error indicators over a variety of sites that might be available for location calibration within a jurisdiction. Considering the ease of use and practicality of the calibration models, all five have a similar level of technical complexity. However, they may differ in ease of application. Models 1 and 2 can be viewed as scaled-based models that are slightly easier to apply and may not require highly technical skills. However, as the results show, they may not be the most effective for improving naïve transferability. Models 3 and 4 are slightly more technical but still relatively straightforward to use because they are based on linear and power regression, respectively. Like Models 1 and 2, they only require the naïve estimates and observed AADB as inputs. Among all five, Model 5 may be the most advanced for input requirements, as each coefficient is calibrated separately. However, for ease of use, it remains quite practical and comparable with the other models, as supported by the results. It is recommended that local calibration Model 3 is used when the number of sites in the local jurisdiction with available AADB counts is less than 80. When counts are available for 80 or more sites, then local calibration Model 5 should be used. When AADB counts are available for many sites in the target jurisdiction, it may be more appropriate to develop a DD model for the target jurisdiction rather than locally calibrate a DD model originally developed in another jurisdiction. The number of sites $k$ at which the local development of a DD model is expected to perform better than a locally calibrated DD model originally developed in another jurisdiction has not been explored in this paper. Therefore, this should be explored for future research, and guidelines should be established for practitioners to assist them in determining the optimal strategy (local calibration versus local development) as a function of the number of sites for which they have AADB counts.

In this paper, the effectiveness of regression-based local calibration models was explored. Future research should explore other forms of local calibration, such as machine learning techniques, Bayesian calibration, or both.

Practical Implications

Developing DD models can be challenging for jurisdictions that lack sufficient CC sites, resources, comprehensive data collection from multiple sources, and technical expertise. Therefore, this paper demonstrates that using and adapting existing DD models can effectively address data and resource limitations faced by jurisdictions. This paper also shows that implementing practical calibration models that are technically straightforward and easy to use can be a cost-effective approach for jurisdictions to estimate bicycle volumes at intersections.

Supplemental Material

sj-docx-1-trr-10.1177_03611981251401589 – Supplemental material for Improving the Accuracy of the Spatial Transferability of Direct-demand Models for Bicycle Volume Estimation at Intersections

Supplemental material, sj-docx-1-trr-10.1177_03611981251401589 for Improving the Accuracy of the Spatial Transferability of Direct-demand Models for Bicycle Volume Estimation at Intersections by Sina Azizi Soldouz and Bruce Hellinga in Transportation Research Record

Footnotes

Acknowledgements

The authors gratefully acknowledge the Region of Waterloo, City of Toronto, City of Milton, and Pima County for providing permission to use the bicycle count data and for providing rich open data portals that were essential sources of information for this research, and Miovision for providing access to the bicycle data. The work in this paper reflects the views of the authors and there is no explicit or implicit endorsement by any of the aforementioned jurisdictions or companies.

Author Contributions

The authors confirm contribution to the paper as follows: study conception and design: Hellinga, Azizi Soldouz; data collection: Azizi Soldouz; analysis and interpretation of results: Azizi Soldouz, Hellinga; manuscript preparation: Azizi Soldouz, Hellinga. All authors reviewed the results and approved the final version of the manuscript.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors gratefully acknowledge financial support from the Natural Sciences and Engineering Research Council of Canada and Transport Canada. The research was carried out by the authors, and no endorsement of the methods or findings by funding agencies is claimed or implied.

ORCID iDs

Sina Azizi Soldouz

Bruce Hellinga

References

Figliozzi

Johnson

Monsere

Nordback

Methodology to Characterize Ideal Short-Term Counting Conditions and Improve AADT Estimation Accuracy Using a Regression-Based Correcting Function. Journal of Transportation Engineering, Vol. 140, No. 5, 2014. https://doi.org/10.1061/(ASCE)TE.1943-5436.0000663.

Nordback

Marshall

W. E.

Janson

B. N.

Stolz

Estimating Annual Average Daily Bicyclists Error and Accuracy. Transportation Research Record: Journal of the Transportation Research Board, 2013. 2339: 90–97. https://doi.org/10.3141/2339-10.

Miah

M. M.

Hyun

K. K.

Mattingly

S. P.

A Review of Bike Volume Prediction Studies. Transportation Letters, Vol. 16, No. 10, 2024, pp. 1406–1433. https://doi.org/10.1080/19427867.2024.2310831.

Sobreira

L. T. P.

Hellinga

Comparing Direct Demand Models for Estimating Pedestrian Volumes at Intersections and Their Spatial Transferability to Other Jurisdictions. Transportation Research Record: Journal of the Transportation Research Board, 2023. 2677: 260–271. https://doi.org/10.1177/03611981231161061.

Munira

Sener

I. N.

Zhang

Estimating Bicycle Demand in the Austin, Texas Area: Role of a Bikeability Index. Journal of Urban Planning and Development, Vol. 147, No. 3, 2021, p. 04021036. https://ascelibrary.org/doi/abs/10.1061/%28ASCE%29UP.1943-5444.0000725.

Tao

Lindsey

Stern

Levin

The Use of Crowdsourced Mobile Data in Estimating Pedestrian and Bicycle Traffic: A Systematic Review. Journal of Transport and Land Use, Vol. 17, No. 1, 2024, pp. 41–65. https://doi.org/10.5198/JTLU.2024.2315.

Hankey

Mondschein

Buehler

Spatial Models of Active Travel in Small Communities: Merging the Goals of Traffic Monitoring and Direct-Demand Modeling. Journal of Transport & Health, Vol. 7, 2017, pp. 149–159. https://doi.org/10.1016/J.JTH.2017.08.009.

Munira

Sener

I. N.

Dai

A Bayesian Spatial Poisson-Lognormal Model to Examine Pedestrian Crash Severity at Signalized Intersections. Accident Analysis & Prevention, Vol. 144, 2020, p. 105679. https://doi.org/10.1016/J.AAP.2020.105679.

Azizi Soldouz

Hellinga

Examining Spatial Transferability of Direct-Demand Models for Estimating Cyclist Counts at Intersections. Transportation Research Record: Journal of the Transportation Research Board, 2025. 2679: 442–457. https://doi.org/10.1177/0361198124 1311563.

10.

Sobreira

L. T. P.

Hellinga

Enhancing the Spatial Transferability of Direct Demand Models for Estimating Pedestrian Volumes at Intersections. Transportation Research Record: Journal of the Transportation Research Board, 2023. 2678: 245–260. https://doi.org/10.1177/03611981231205881.

11.

Kavta

Bösehans

Bell

M. C.

Liao

de A. Correia

G. H.

Dissanayake

Assessing the Spatial Transferability of Mode Choice Models: A Case of Shared Electric Mobility Hubs (EHUBS) in Amsterdam and Manchester. Transport Policy, Vol. 156, 2024, pp. 101–111. https://doi.org/10.1016/J.TRANPOL.2024.07.017.

12.

Santoso

D. S.

Tsunokawa

Spatial Transferability and Updating Analysis of Mode Choice Models in Developing Countries. Transportation Planning and Technology, Vol. 28, No. 5, 2005, pp. 341–358. https://doi.org/10.1080/03081060500319694.

13.

Koushik

A. N. P.

Manoj

Nezamuddin

Prathosh

A. P.

Testing and Enhancing Spatial Transferability of Artificial Neural Networks Based Travel Behavior Models. Transportation Letters, Vol. 15, No. 9, 2023, pp. 1083–1094. https://doi.org/10.1080/19427867.2022.2130150;SUBPAGE:STRING:ACCESS.

14.

Santoso

D. S.

Tsunokawa

Comparison of Updating Techniques in Transferability Analysis of Work Trip Mode Choice Models in Developing Countries. Journal of Advanced Transportation, Vol. 44, No. 2, 2010, pp. 89–102. https://doi.org/10.1002/ATR.112.

15.

Wang

Hankey

Lindsey

Wang

Lindsey

Humphrey

H. H.

Monitoring and Modeling of Urban Trail Traffic Validation of Direct Demand Models in Minneapolis, Minnesota, and Columbus, Ohio. Transportation Research Record: Journal of the Transportation Research Board, 2016. 2593: 47–59. https://doi.org/10.3141/2593-06.

16.

Kothuri

Broach

Mcneil

Hyun

Mattingly

Miah

Md. M.

Nordback

Proulx

Exploring Data Fusion Techniques to Estimate Network-Wide Bicycle Volumes. NITC-RR-1269. Transportation Research and Education Center (TREC), Portland, OR, 2022. https://doi.org/10.15760/trec.273.

17.

Miah

Md. M.

Hyun

K. K.

Mattingly

S. P.

Broach

McNeil

Kothuri

Challenges and Opportunities of Emerging Data Sources to Estimate Network-Wide Bike Counts. Journal of Transportation Engineering, Part A: Systems, Vol. 148, No. 3, 2021. https://doi.org/10.1061/JTEPBS.0000634..

18.

Broach

Kothuri

Miah

Mcneil

Hyun

Mattingly

Nordback

Proulx

Evaluating the Potential of Crowdsourced Data to Estimate Network-Wide Bicycle Volumes. Research Article Transportation Research Record, Vol. 2024, No. 3, pp. 573–589. https://doi.org/10.1177/03611981231182388.

19.

Griswold

Medury

Schneider

Pilot Models for Estimating Bicycle Intersection Volumes. Transportation Research Record: Journal of the Transportation Research Board, 2011. 2247: 1–7. https://doi.org/10.3141/2247-01.

20.

Strauss

Miranda-Moreno

L. F.

Morency

Cyclist Activity and Injury Risk Analysis at Signalized Intersections: A Bayesian Modelling Approach. Accident Analysis & Prevention, Vol. 59, 2013, pp. 9–17. https://doi.org/10.1016/J.AAP.2013.04.037.

21.

Strauss

Miranda-Moreno

L. F.

Spatial Modeling of Bicycle Activity at Signalized Intersections. Journal of Transport and Land Use, Vol. 6, No. 2, 2013, pp. 47–58. https://doi.org/10.5198/jtlu.v6i2.296.

22.

Tabeshian

Kattan

Modeling Nonmotorized Travel Demand at Intersections in Calgary, Canada. Transportation Research Record: Journal of the Transportation Research Board, 2014. 2430: 38–46. https://doi.org/10.3141/2430-05.

23.

OpenStreetMap. https://www.openstreetmap.org/#map=3/71.34/-96.82. Accessed May 29, 2024.

24.

Bicycle Routes | Pima County Open Data. https://gisopendata.pima.gov/datasets/e590f125f2744ad0918c9025363004e b_2/about. Accessed May 29, 2024.

25.

Bikeways - City of Toronto Open Data Portal. https://open.toronto.ca/dataset/bikeways/. Accessed May 29, 2024.

26.

Cycling | Cycling | City of Waterloo Open Data. https://data.waterloo.ca/datasets/RMW::cycling/explore?location =43.437309%2C-80.459000%2C10.59. Accessed May 29, 2024.

27.

Zoning - Pima County | Pima County Open Data. https://gisopendata.pima.gov/datasets/PimaMaps::zoning-pima-county/about. Accessed May 29, 2024.

28.

Zoning By-Law - City of Toronto Open Data Portal. https:// open.toronto.ca/dataset/zoning-by-law/. Accessed May 29, 2024.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB