Detecting Anomalies in National Bridge Inventory Databases Using Machine Learning Methods

Abstract

National Bridge Inventory (NBI) data is regularly collected for 617,000+ national bridges in the U.S. These data, which consist of 100+ fields related to bridges and culverts, have been shown to contain errors. These errors could reduce the effectiveness of the decisions made based on this data, and cause safety issues. For this reason, an anomaly detection platform is developed to identify data anomalies in NBI datasets more effectively than existing rule-based error-check tools can. First, the user provides groups of correlated NBI fields as input to the platform. Then, for each group, it utilizes two tools to detect anomalous data and determine errors. The first tool uses three machine learning algorithms to identify anomalous data points and categorizes them based on their degree of anomaly. The second tool visualizes the distributions of the NBI fields in the group with histograms, scatter plots, and so forth. These plots are used to analyze the data points that are identified from the first tool as anomalies. The results of these two tools, together with expert knowledge about the data fields, are then used to distinguish data errors from outliers. The proposed platform is applied to a state’s NBI dataset that was submitted to the Federal Highway Administration (FHWA) in 2020. For this dataset, two groups of correlated fields are considered. The results showed the platform could effectively pinpoint anomalous values of NBI fields that individually, or in conjunction with other fields, do not follow the patterns that characterize most of the data, prompting the identification of potential inconsistencies and errors.

Keywords

data and data science data analytics machine learning (artificial intelligence)unsupervised learning national and state transportation data and information systems federal reports planning and analysis transportation planning policy and processes statewide planning

Get full access to this article

View all access options for this article.

References

American Society of Civil Engineers. 2021 Report Card for America’s Infrastructure, Bridges. 2021. https://infrastructurereportcard.org/wp-content/uploads/2020/12/Bridges-2021.pdf.

Din

Z. U.

Tang

Automatic Logical Inconsistency Detection in the National Bridge Inventory. Procedia Engineering, Vol. 145, 2016, pp. 729–737.

FHWA. Recording and Coding Guide for the Structure Inventory and Appraisal of the Nation’s Bridges. Report No. FHWA-PD-96-001. US Department of Transportation, Federal Highway Administration, Office of Engineering Bridge Division, Washington, D.C., 1995.

VDOT. State of Good Repair (SGR) Program Bridge Prioritization Formula. Virginia Department of Transportation, Structure and Bridge Division, 2018.

Cook

Bridge Failure Rates, Consequences, and Predictive Trends. PhD dissertation. Utah State University, 2014.

Dekelbab

Al-Wazeer

Harris

History Lessons From the National Bridge Inventory. Public Roads, Vol. 71, No. 6, 2008.

NBI Data Checks . Federal Highway Administration Office of Bridges and Structures. 2020. Accessed July 19, 2021. https://www.fhwa.dot.gov/bridge/nbi/checks/nbi_data_checks_2020_final.pdf.

VDOT. Inventory And Appraisal Coding Guide for Virginia’s Structures. Virginia Department of Transportation, Structure and Bridge Division, 2021.

FDOT. Bridge Management System Coding Guide. Florida Department of Transportation, Office of Maintenance. November5, 2019.

10.

National Bridge Inventory (NBI) . US Department of Transportation, Federal Highway Administration, Bridge & Structures. Accessed July 19, 2021. https://www.fhwa.dot.gov/bridge/nbi.cfm.

11.

Traffic Level of Service Calculation Methods, Appendix B . City/County Association of Governments of San Mateo County. Accessed July 19, 2021. https://ccag.ca.gov/wp-content/uploads/2014/07/cmp_2005_Appendix_B.pdf.

12.

Fiorillo

Nassif

Application of Machine Learning Techniques for the Analysis of National Bridge Inventory and Bridge Element Data. Transportation Research Record: Journal of the Transportation Research Board, 2019. 2673: 99–110.

13.

Heng

Nehme

J. A.

Ping

Bridge Deck Condition Rating Forecast—Machine Learning Models. Federal Highway Administration Office of Research, Development, and Technology. 2020. Accessed October 2021. https://highways.dot.gov/research/long-term-infrastructure-performance/ltbp/data-analysis/bridge-deck-condition-rating-machine.

14.

Your Ultimate Data Manipulation and Cleaning Cheat Sheet. Towards Data Science. 2020. Accessed July 23, 2021. https://towardsdatascience.com/your-ultimate-data-manipulation-cleaning-cheat-sheet-731f3b14a0be

15.

Verma

Isolation Forest Algorithm for Anomaly Detection. Heartbeat, Fritz AI. 2020. Accessed July 23, 2021. https://heartbeat.fritz.ai/isolation-forest-algorithm-for-anomaly-detection-2a4abd347a5

16.

Thudumu

Branch

Jin

Singh

J. J.

A Comprehensive Survey of Anomaly Detection Techniques for High Dimensional Big Data. Journal of Big Data, Vol. 7, No. 1, 2020, pp. 1–30.

17.

Zimek

Filzmoser

There and Back Again: Outlier Detection Between Statistical Reasoning and Data Mining Algorithms. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, Vol. 8, No. 6, 2018, p. e1280.

18.

Breunig

M. M. H.-P.

Kriegel

R. T. Ng

Sander

LOF: Identifying Density-Based Local Outliers. Proc., 2000 ACM SIGMOD International Conference on Management of Data, Association for Computing Machinery, New York, NY, 2000, pp. 93–104.

19.

Liu

F. T.

Ting

M. K.

Zhou

Z.-H.

Isolation Forest. Proc., 8th IEEE International Conference on Data Mining, IEEE, New York, 2008, pp. 413–422.

20.

Breiman

Random Forests. Machine Learning, Vol. 45, No. 1, 2001, pp. 5–32.

21.

Schälkopf

Platt

J. C.

Shawe-Taylor

Smola

A. J.

Williamson

R. C.

Estimating the Support of a High-Dimensional Distribution. Neural Computation, Vol. 13, 2001, pp. 1443–1471.

22.

Boser

B. E.

Guyon

I. M.

Vapnik

V. N.

A Training Algorithm for Optimal Margin Classifiers. Proc., Fifth Annual Workshop on Computational Learning Theory, Pittsburgh, PA, Association for Computing Machinery, New York, NY, 1992, pp. 144–152.

23.

Federal Highway Administration. Geometric Design. Mitigation Strategies for Design Exceptions. Lane Width. US Department of Transportation, Federal Highway Administration, Archived. Accessed July 28, 2021. https://safety.fhwa.dot.gov/geometric/pubs/mitigationstrategies/chapter3/3_lanewidth.cfm.