Three decades of statistical pattern recognition paradigm for SHM of bridges

Abstract

Bridges play a crucial role in modern societies, regardless of their culture, geographical location, or economic development. The safest, economical, and most resilient bridges are those that are well managed and maintained. In the last three decades, structural health monitoring (SHM) has been a promising tool in management activities of bridges as potentially it permits one to perform condition assessment to reduce uncertainty in the planning and designing of maintenance activities as well as to increase the service performance and safety of operation. The general idea has been the transformation of massive data obtained from monitoring systems and numerical models into meaningful information. To deal with large amounts of data and perform the damage identification automatically, SHM has been cast in the context of the statistical pattern recognition (SPR) paradigm, where machine learning plays an important role. Meanwhile, recent technologies have unveiled alternative sensing opportunities and new perspectives to manage and observe the response of bridges, but it is widely recognized that bridge SHM is not yet fully capable of producing reliable global information on the presence of damage. While there have been multiple review studies published on SHM and vibration-based structural damage detection for wider scopes, there have not been so many reviews on SHM of bridges in the context of the SPR paradigm. Besides, some of those reviews become obsolete quite fast, and they are usually biased towards applications falling outside of bridge engineering. Therefore, the main goal of this article is to summarize the concept of SHM and point out key developments in research and applications of the SPR paradigm observed in bridges in the last three decades, including developments in sensing technology and data analysis, and to identify current and future trends to promote more coordinated and interdisciplinary research in the SHM of bridges.

Introduction

Our society relies on civil infrastructure for economic and social life, where bridges play an important role in the mobility of people and goods. Bridges age naturally, but their structural deterioration can be accelerated by external phenomena both natural and human-derived, which put pressure on bridge owners and authorities to develop strategies for their permanent structural condition assessment.

Heuristic forms of condition assessment of bridges may date back to the origins of structural engineering. However, the implementation of a strategy based on monitoring systems to measure in real time the structural responses and on data analysis to identify anomalies and/or damage at early stages, has been defined as structural health monitoring (SHM). Damage is herein defined as changes to the material and/or geometric properties of the bridge components, which adversely affect the bridge’s current or future service performance and safety. ¹

Etymologically, SHM is a combination of words that has emerged around the late 1980s. ² It is associated with the classical medical and health sector, where the structural condition of a bridge (human body) is assessed through monitoring instruments (e.g., stethoscope), in order to support the assessment process by specialized engineers (doctors). In practice, SHM is a combination of expertise and knowledge from a variety of disciplines including structural engineering, material science, electrical engineering, mechanical engineering, computer science, and data management.

Since early 1990s, SHM has been proposed to support the bridge owners and authorities to find optimal life-cycle management solutions, and ultimately to avoid structural failures, based on a damage identification strategy, which includes the detection, diagnosis, and prognosis of damage. ³

SHM has been performed arguably on information from two independent approaches: physics- and data-based. The physics-based approach uses the inverse problem technique. In the data-based approach, machine learning algorithms are essential to learn (or to model) the structural behavior from experience (or past data), following the same principle as the human brain. Nonetheless, the authors believe that all approaches to SHM, as well as all traditional nondestructive evaluation (NDE) techniques, can be cast in the context of a pattern recognition problem, where damage is identified by comparing structural responses with a baseline or reference condition. In 1994, in the context of using modal properties for damage detection and structural monitoring in bridges, Farrar et al. ⁴ defined pattern recognition as an approach to accentuate changes in the frequency response functions measured on cracked and uncracked bridges. Therefore, on the assumption that the assessment of damage requires a comparison between two system states, herein the SHM is posed in the context of a statistical pattern recognition (SPR) paradigm, rooted in the artificial intelligence field. The main goal of the SPR paradigm is to distinguish patterns (or structural changes) related to the undamaged structural condition under operational and environmental variability from those associated with damaged conditions, by starting with sensor data from the monitored structure and finishing with damage identification to assess the actual structural condition. Extending the concept laid out in 2001, ¹ the SPR paradigm can be decomposed into four stages: (1) operational evaluation; (2) data acquisition; (3) feature extraction and generation; and (4) statistical modeling for feature classification.

The general approaches taken to SHM are either direct damage identification (visual inspections and/or NDE) or indirect damage identification. This article is fundamentally concerned with the latter. In indirect damage identification, detecting changes in structural properties or system behavior through development of intelligent and optimal measurement systems that allow remote condition monitoring of bridges is a milestone in the implementation of an efficient SHM strategy. Herein, the sensing technology is a critical part of an SHM system as well as the definition of return-on-investment to deploy it. Sensors cannot measure damage, but pattern recognition cannot detect hidden information if structural responses do not have signatures related to the real damage! Thus, there is no way to extract damage-sensitive features and develop statistical modeling for damage identification if the measured signals do not contain any information from damage. Therefore, the full-field implementation of SHM is dependent on improvements in all stages of the SPR paradigm.

There has been a large volume of research on condition assessment of bridges, but until today it has yielded relatively few routine bridge applications. ⁵ Nonetheless, the first decade of the millennium saw a boom in field deployments around the world, especially pushed by the bridge construction sector in China. ⁶ According to Cawley, ⁷ aside from the lack of attention to the business case for monitoring, the reasons for the slow transfer from research to practical application of SHM include insufficient attention to how the large data flows are handled and the lack of performance validation on real structures in real environments. The lack of validation is a particular issue in the case of one-of-a-kind structures such as bridges that have slow damage processes.

Nonetheless, the combination of low-probability high-impact risks of bridge failures has made the SHM technology appealing. At the European level, groundwork on vibration-based SHM has been funded through FP6, FP7, and H2020 in projects addressing wider scopes (e.g., risk reduction, resilience, life-cycle assessment, energy harvesting, and internet of things) and specific hazards (e.g., earthquakes, landslides, and flooding). The IRIS, BRIDGEMON and BridgeScan Projects are some examples. Even though the full-field implementation of SHM technology has been slower than expected in early 2000s, according to the 2019 JRC Policy Report, ⁸ the European Union countries must invest and address maintenance issues to ensure serviceability and safety of our bridges. This includes investing in SHM systems.

There has been growing interest in SHM and pattern recognition for bridges, as evidenced (Figure 1) by the number of publications in the field, according to Web of Science. However, few review articles have been dedicated to bridges. For instance, since 2005, in the International Journal of Structural Health Monitoring, only one out of 10 review papers were published on SHM of bridges (i.e., it has bridge in the title). In a more practice-based civil focused Journal of Civil Structural Health Monitoring, it is clear there is increasing interest in SHM applications for bridges (Figure 2). Among books written on SHM few have been posed exclusively in the context of bridges. ^2,9–11

Figure 1.

Number of publications based on keywords as “structural health monitoring bridges” and “pattern recognition bridges.”

Figure 2.

Number of publications based on keywords as “structural health monitoring bridges” in the journal of civil structural health monitoring.

Therefore, the main goal of this article is to review key developments in research and applications of the SPR paradigm observed in SHM of bridges in the last three decades, including developments in sensing technology and data analysis. A second goal is to identify current and future trends in order to promote more coordinated and interdisciplinary research in the SHM field of bridges. The reader should keep in mind that even though this article intends to perform a thorough independent review, its outcomes will always reflect the experience of the authors.

The organization of this article is as follows. Besides the Introduction, Structural Health Monitoring of Bridges gives some basic definitions and a contextualization of the SHM process and the SPR paradigm in bridge engineering for condition assessment. Operational Evaluation, Data Acquisition, Feature Extraction and Generation, and Statistical Modeling for Feature Classification are dedicated to the most recent updates on each stage of the SPR paradigm. The Future Trends section exposes some current and future trends. Finally, Summary and Conclusions section gives a summary, draws some conclusions, and points out some limitations and challenges for the widescale and routine implementation of SHM in bridges.

Structural health monitoring of bridges

From bridge collapses to bridge management systems, structural health monitoring and asset management

Primitive civilizations made bridges of stone and wood but it is believed that is the ancient Romans who were the greatest bridge builders that really started the story with the Roman arch. The Alcántara Bridge, in Spain, over the Tagus River and the Trajano Bridge, in Portugal, over the Tâmega River are two examples of that era (Figure 3). With the Middle Ages came the inhabited bridges, such as the Vecchio Bridge over the Arno River in Florence, Italy, with shops on stone arches.

Figure 3.

Roman bridges era: Trajano Bridge, Portugal.

A new era of bridge construction came with iron. The first bridge in the world to use iron as the main structural material was the Iron Bridge, in Shropshire, England, constructed in 1779 by Abraham Darby III. It revolutionized the way structures were built, enabling engineers to build ever larger structures and buildings. The iron bridge boom around the world was pushed by expansion of railway networks in the 19^th century with the steel bridge era (late 19^th century) probably initiated with the construction of the Eads Bridge (Illinois, 1874), as one of the firsts steel bridges to be built. ¹² With the development of steel, the size of structures increased even more. The steel-wire suspension Brooklyn Bridge (1883) stands as the first long span bridge and the Golden Gate Bridge (1933, Figure 4) as an icon of that era. Even though the Homersfield Bridge, a road bridge constructed in 1871, in England, by Sir Shafto Adair is one of the oldest surviving concrete bridges (it has a single 15 m span consisting of a wrought iron frame encased in concrete), the advent of the concrete bridge began after the Second World War, especially pushed by the road construction programs in the United States and in Europe. It represented the advent of massive highways and prestressing reinforced concrete bridges.

Figure 4.

Steel bridge era: Golden Gate Bridge, United States (1933).

Why do we need bridge management? While Roman bridges have lasted two millennia, new bridges are designed for a limited lifetime and have progressive needs for “doctors.” Even though the design lifetime of bridges should be over 100 years, some components require renewal during the lifetime of the bridges and very often the structural materials show accelerated degradation. It was the lack of scheduled bridge inspection programs and the priority on the construction of new bridges that laid the foundations for the first tragic bridge failures in the second half of the 20^th century.

The US Highway 35 Silver Bridge failure in the United States has been reported as the event that triggered the development of bridge management systems (BMSs). A BMS has been defined as a visual inspection-based decision-support tool to analyze engineering and economic factors and to assist the authorities in determining how and when to make decisions regarding maintenance, repair, and rehabilitation of structures. In the early 1990s, several software packages were developed to assist the authorities in managing bridges, such as PONTIS and BRIDGIT in the United States and DANBRO in Denmark. Figueiredo et al. ⁹ have already provided a thorough overview of the history of BMS and bridge failures.

In Portugal, until 1990s, bridge management was carried out in a simplified manner. Few accidents were reported and the lack of maintenance was not a serious issue yet, which can be explained by the reduced number of bridges. The early management systems had potential information flaws, derived by the manual filing systems. Moreover, they were not prepared to interact with financial programming as well as the needs of the whole transportation network. In mid 1990s, the Portuguese largest private concessionary, namely Brisa – Autoestradas de Portugal, made the first step towards the creation of a BMS, with the development of STONE. Nevertheless, the collapse of the Hintze Ribeiro Bridge in 2001 stands as the “tipping point” in terms of bridge maintenance. The centenary bridge disaster boosted the Portuguese authorities’ requirement for regular bridge inspections. As the collapse was later found to be related to streambed scouring caused by illegal sand extraction, which compromised the integrity of the foundation of one of the pillars, the authorities also realized the need for periodic underwater bridge inspections. In the early 2000, the GOA system was adopted by the main bridge owners. Today, most Portuguese bridges are under regular and scheduled inspection programs. ⁹

In spite of huge developments observed in the BMSs, many bridge failures have been recorded around the world. Even recently, the Polcevera Viaduct in Genoa, Italy, constructed in 1967 and designed by Riccardo Morandi, partially collapsed on August 14^th, 2018, during a rainstorm, causing the death of 43 people. The bridge was managed by a private company—Autostrade per l’Italia. Aftermath reports indicated the bridge design lacked redundancy and the bridge itself had shown signs of long-time degradation. ¹³ In accordance with the Ministry of Infrastructure, Autostrade was aware of the degradation but delayed the refurbishment and repair of the stays. Experts recommended the installation of a monitoring system to evaluate the bridge structural condition, but no action was made. ¹⁴ Could an SHM system had saved the lives of those 43 people?

Based on the authors’ experience, a few conclusions can be made about the past and present of the condition assessment of bridges:

• Most bridge failures take place due to a combination of several factors and after an extreme natural event.

• Around the world, the increasing number of existing bridges and the continuous need to maintain them, along with the information technology revolution, brought about the generalization of BMSs to deal with data and information collected from regular bridge inspections. Are BMSs enough for bridge management?

• The incidents that occurred around the world, after the implementation of BMSs, have clearly shown the insufficiency of the current BMSs to avoid bridge collapses, putting pressure on the bridge owners and authorities to improve them. Can the risk of bridge failures be further minimized?

• Nevertheless, to date, the periodic structural condition assessment of bridges essentially relies on weighted indices based on (subjective) visual inspections and/or preliminary NDE technologies. Can regular bridge inspections guarantee reliable condition assessment?

• A current challenge has been the development of efficient, optimized, targeted, cost effective, and re-liable condition assessment methodologies, that can generate better prediction models, to support the decisions of the authorities regarding bridge safety assessment and maintenance. Is SHM technology ready to support bridge management?

The financial implications and potential impact through optimal bridge management in developed societies are vast. For instance, in the United States, according to the American Society of Civil Engineers, the cost of eliminating all existing bridge deficiencies as they arise over the next 50 years is estimated at $850 billion in 2006 dollars, equating to an average annual investment of $17 billion. ¹⁵

Infrastructure asset management has become very popular in most developed countries in the 21^st century, as their infrastructure networks, mostly built in the 20^th century, have to be managed, operated, and maintained in a cost-effective way. Bridge owners and authorities increasingly rely on asset management procedures to ensure a coordinated approach to the optimization of costs, risks, service/performance, and sustainability of the bridges and the network as a whole. It is believed that permanent evaluation of the condition of bridges may be fundamental to accomplish optimal infrastructure asset management.

In conclusion, improved and more continuous condition assessment of bridges has been demanded by our society to reduce the risks of bridge collapses; the recent example in Genoa has once again reminded us of the need for new and more reliable tools to prevent such catastrophic events. The current limitations of visual inspections, which have been identified as a shortcoming in BMS, have driven the research to developments in long-term monitoring, namely to the advent of SHM, whose results may be integrated into BMSs in a systematic way. ¹⁶ Therefore, some of the answers to those questions and the opportunities of SHM for long-term condition assessment of bridges will be given throughout this article and others by the scientific community in the years to come.

Statistical pattern recognition paradigm

The SPR paradigm for bridge SHM can be described as a four-stage process as illustrated in Figure 5:

1. Operational evaluation,

2. Data acquisition,

3. Feature extraction and generation, and

4. Statistical modeling for feature classification.

Figure 5.

The SHM process for bridges based on a four-stage SPR paradigm.

The main objective of the SPR paradigm is to distinguish patterns (or structural changes) related to the undamaged structural condition under operational and environmental variability from those associated with damaged conditions, by starting from sensor data of the monitored structure and finishing with damage identification to assess the actual structural condition. Insights and new updates in each of the four stages will be the focus of Operational Evaluation, Data Acquisition, Feature Extraction, and Statistical Modeling for Feature Classification sections.

Even though the four-stage scheme makes easier the understanding of the SPR paradigm, it is sometimes difficult to define fixed boundaries between those stages. Therefore, one should note that inherent in the data acquisition, feature extraction, and statistical modeling portions of the SPR paradigm are data normalization, cleansing, fusion, and compression. ¹⁷ In SHM, data normalization is the process of separating changes in sensor reading caused by damage from those caused by varying operational and environmental conditions. Data cleansing is the process of selectively choosing data to pass on to, or reject from, the feature selection process. Data fusion is the process of combining information from multiple sensors in an effort to enhance the fidelity of the damage detection process. Data compression is the process of reducing the dimensionality of the data, or the features extracted from the data, in an effort to facilitate efficient storage of information, to enhance the statistical quantification of these parameters, and to permit data visualization as human brain provides very powerful capabilities if used to advantage. These four activities can be implemented in either hardware or software and usually a combination of the two is used. ¹⁸

Also inherent in data acquisition, feature extraction and generation, and statistical modeling for feature classification are the numerical models and the machine learning algorithms.

Data, information, knowledge and damage identification

In SHM, knowledge is all relevant information about the structural condition contained in the data, which can be used for condition assessment and to support the decision-making process regarding maintenance and repair of bridges. ¹⁹ The data can be measured directly from the structure or generated either numerically or experimentally using a downscaled laboratory model.

The structural condition assessment must be as detailed as possible to describe the damage impact on the structural system. Following the Rytter proposal, ²⁰ the scientific community has adopted a damage identification hierarchy as a sequence of different levels of knowledge about damage and how it affects the structure. In a broad sense, developments on damage identification can be divided into three areas: damage detection, damage diagnosis, and damage prognosis. In particular, damage diagnosis can be subdivided to better characterize the damage in terms of location, type, and severity. Figure 6 shows the hierarchical structure of damage identification decomposed in five levels which answer the following questions ³ :

• Is the damage present in the system (detection)?

• Where is the damage (localization)?

• What kind of damage is present (type)?

• What is the extent of damage (severity)?

• How much useful lifetime remains (prognosis)?

Figure 6.

Hierarchical structure of damage identification.

The answers to the questions above can only be made in a sequential manner. For example, the question on severity of damage can only be answered with a priori knowledge of the type of damage. At level five, the damage prognosis cannot be accomplished without an understanding of the damage accumulation process. For further discussion on the concept of damage prognosis, one should read the reference. ²¹

Definition of damage

Damage is traditionally defined as changes to the material and/or geometric properties of the bridge components, which adversely affect the bridge’s current or future service performance and safety. ¹ According to the definition, damage does not indicate necessarily a complete failure of a system, but an accumulative deterioration of the system functionality causing a suboptimal performance until reaching the failure state. ²² Nonetheless, systems may fail in a gradual or sudden manner depending on the type of the damage. For instance, failure due to corrosion or fatigue usually occur over long time periods, while earthquakes and flooding can lead to a rapid failure (e.g., scour) by exploiting weak local points or damages in the structure.

Actually, scour of bridge piers and abutments has been shown to be one of the most common causes of bridge failure. ²³ Scour is the process whereby increased flow rates around a pier erode the surrounding soil. This can be viewed as a change to the boundary conditions of the bridge that can compromise its structural integrity. However, this form of damage may not alter the local mass or stiffness properties of the structure itself, ¹⁰ which challenges the global vibration-based damage identification and imposes the deployment of alternative sensing and damage identification strategies. Meanwhile, the Polcevera Viaduct in Genoa underwent probably the first large bridge collapse, due to a strong decay in its capacity caused by corrosion. ²⁴

A thorough list and valuable pictures of current bridge damage scenarios is provided by Figueiredo et al. ⁹ A state-of-the-art review on the causes and mechanisms of bridge failures can be found in Refs. 24, 25.

Operational and environmental variability

Current procedures in bridge SHM have a considerable number of limitations and prevailing uncertainties due to non-stationary variability sources associated with operational and environmental conditions. Varying operational conditions include live loads such as traffic loads, speed of operation, and changing excitation sources. Varying environmental conditions are manifestations of weather in the form of temperature, humidity, wind, rainfall, and snow. Note that other sources of variability exist at the data acquisition stage such as instrumentation, random noise, estimation methods, and vibration source.

Daily temperature variations can impose changes in the natural frequencies with magnitude exceeding 10%. For instance, in 2001, Peeters et al. ²⁶ performed a study addressing the influence of the temperature effect on the modal parameters of the Z24 Bridge. The authors speculated that differences in the natural frequencies ranging from 14 to 18% must be explained by environmental changes. In another study carried out in 2008, in one simply supported span at the end of the Alamosa Canyon Bridge, in the United States, and asymmetrical variation in the first mode shape was observed throughout the day, as shown in Figure 7 (the colors are used to highlight the differences in the modal coordinates). This asymmetry along the longitudinal axis was correlated with the time of the day and associated solar heating. Note that these thermal effects were more pronounced because of the north-south orientation of the bridge. If not properly accounted for, such changes in the dynamic response characteristics can potentially result in false indications of damage. If the mode in Figure 7(a) was considered to be the baseline condition, a classification algorithm would identify the mode in Figure 7(b) as some form of an outlier. This outlier could inappropriately be labeled as damage if the environmental variability associated with this feature was not taken into account in the outlier detection process. ²⁷ A review on the effects of operational and environmental variability on SHM can be found in Ref. 28; a bridge-guided review on those effects can be found in Ref. 3.

Figure 7.

First mode shape of one simply supported span of the Alamosa Canyon Bridge during two distinct times of the day: (a) in the morning (7.75 Hz); and (b) in the afternoon (7.42 Hz).

Several authors have tackled these effects by proposing data-driven methods to normalize or remove those effects from the structural responses. ^27,29,30

Machine learning algorithms and learning strategies

In SHM, machine learning is the science of getting computers and algorithms to model bridges without knowing the physical laws of structures. Machine learning algorithms are essential to learn (or to model) the structural behavior from experience (or past data), following the same principle as the human brain, and to perform pattern recognition for damage identification. ²⁷

The machine learning algorithms used in statistical model development usually fall into three general categories: (i) group classification, (ii) regression analysis, and (iii) outlier detection. The appropriate algorithm to use depends on the ability and the data available to perform supervised or unsupervised learning. Supervised learning refers to the case where data from undamaged and damaged conditions are available to train the algorithms. Unsupervised learning refers to the case where training data are only available from the undamaged condition. When applied in an unsupervised mode, machine learning algorithms are typically used to answer questions regarding the detection and localization of damage. ²⁷ When applied in a supervised mode, the statistical algorithms can be used to better determine the type of damage, severity, and the remaining useful lifetime (Figure 6).

The unsupervised learning algorithms have been the focus of most studies, because bridges are high-capital-expenditure structures and so only data from the undamaged condition are generally available. The algorithms have been used especially for data normalization and data fusion. The main concern has been the damage detection, assuming the structures are undamaged and functioning under normal operational state conditions during a certain period of time (at least one year, in order to include seasonal variations). This strategy may be suitable to address the first level of the damage identification hierarchy, but its performance is highly dependent on the amount and variability of the data used during the training process. ³¹

In order to step forward in the damage identification hierarchy, supervised learning strategies must be pursued, which require labeled data for the training. The scientific community is trying to come up with innovative solutions for the supervised machine learning in the absence of measured data from damaged bridges. ²² One strategy has already been tackled by some researchers using real-world experimental data from the undamaged condition together with simulated data for the healthy and damage scenarios. The simulated data can be obtained either numerically using a calibrated finite element (FE) model ³² or experimentally using a downscaled laboratory model of the monitored bridge. Potentially, this strategy would eliminate the need for having experimental data from damaged bridges.

Unexploited in SHM is the semi-supervised learning, as an approach to machine learning that combines a small amount of labeled data with a large amount of unlabeled data during training. ^33,34 Semi-supervised learning falls between unsupervised learning (with no labeled training data) and supervised learning (with only labeled training data). This approach alleviates the burden of obtaining an entire set of damaged data sets, which is costly or impractical in bridge engineering.

From physics-based and data-based to hybrid-based approaches

SHM has been performed arguably on information from two independent approaches: physics- and data-based. The physics-based approach uses the inverse problem technique. The data-based approach has been rooted in the machine learning field.

The physics-based approach attempts to identify damage by relating the measured data from the structures to the data retrieved from their numerical models (e.g., calibrated FE models) ³² or, in special cases, experimentally using downscaled laboratory models of the monitored bridges. Nonetheless, the physics-based approach has been mainly rooted in FE model updating techniques; it aims at identifying structural damage by comparing the measured structural responses with the ones derived from baseline FE models, tailored for that specific structure and validated against its undamaged behavior. The structural responses recorded by the monitoring system are used iteratively to update some calibration parameters (e.g., stiffness properties and boundary conditions), defined at the finite element level, such as to minimize a set of objective functions that reflect the difference between the computed and the measured structural responses. ^35,36 The values of the calibration parameters that minimize the objective functions indicate the presence and, possibly, the localization and extent of the damage.

Unfortunately, considerable uncertainty and numerous sources of errors are inherent in FE models. Essentially, all material parameters fed into the FE models are uncertain, and most of them also change over time. The geometrical dimensions of the structural elements are less uncertain, but the dimensions of non-structural elements may be considerably different from those present in the design project. Sources of errors include the mathematical model of the structure, which may assume, for instance, small displacements, isotropy, linear-elastic behavior, and the solution method, which is heavily influenced by the quality of the mesh and the size of the finite element basis. A significant issue is whether or not the FE modeling can even represent the structure, through missing out key components or oversimplifying, leaving the objective function with few chances to be zero.

In the data-based approach, machine learning algorithms have been used to detect damage through variations in some damage-sensitive features extracted from the structural response data that are outliers in respect to the undamaged data. Several machine learning algorithms, with different working principles, have been proposed in the last two decades to separate changes in the features caused by structural damage from those caused by varying operational and environmental conditions. ^{27,31,37–39}

The issue of environmental variations (e.g., temperature, humidity, and wind speed) that often cause unwanted effects in the structural responses and affect the damage-sensitive features, possibly hiding the changes caused by damage, ²⁸ is common to both physics- and data-based approaches. In data-based approaches, observations corresponding to more extreme environmental variations are much rarer than observations under regular conditions. In spite of various techniques of data cleansing, fusion, and normalization, the rarity of such data typically hampers the capability of the machine learning algorithms to correctly classify observations under such conditions as compared to the regular conditions. ⁴⁰ In physics-based approaches, the uncertainty regarding the environmental conditions adds to that regarding the material parameters and structural integrity, typically resulting in having more calibration (unknown structural) parameters than objective functions (e.g., natural frequencies) to be used for model updating. However, additional constraints need to be enforced on the calibration parameters, such as physically plausible bounds and expected correlations between the parameters, which actually lead to an over-constrained problem. To solve it, a procedure is encoded as an iterative constrained minimization. In each calibration step, the calibration parameters are tuned to minimize the error function, while ensuring that the values of the calibration parameters remain within the limits of what is physically plausible. ⁴¹

Therefore, in the last decade, a hybrid approach aimed at combining the best capabilities of the physics- and data-based approaches has emerged. This framework leverages the benefits of both approaches while mitigating their shortcomings. The basic principles of the hybrid approach were stated in Ref. 42. The hybrid approaches hold the potential to enable the transition from unsupervised to supervised learning in bridge SHM and to advance on the damage identification hierarchy. FE models have been used as proxies (i) to simulate extreme environmental and operational conditions and (ii) to induce damage and to enable the supervised learning of the machine learning algorithms. When damage is detected, model updating techniques can be used to progress the damage identification hierarchy by suggesting plausible localizations and types of damage and, possibly, the likelihood of their occurrence.

Operational evaluation

The first stage for developing an SHM capability is to perform an operational evaluation for the candidate bridge. At this stage of the SPR paradigm, alternatives sources of response data can be designed and implemented in order to step forward in the damage identification hierarchy. Depending on the type of damage to be characterized, and the implementation period required for the SHM system, the structural response data can typically be obtained through three sources: (i) experimentally using a prototype structure (traditional approach), (ii) experimentally using a downscaled laboratory model, and (iii) computationally via numerical models like the finite element method. The first two sources depend directly on the data acquisition systems. The third makes usage of the experimental data to calibrate numerical models for data generation.

The operational evaluation stage defines, and to the greatest extent possible quantifies, the damage that is to be identified. It also establishes the benefits to be gained from the deployment of the SHM system. This process also begins to impose limitations on what will be monitored and how to perform the monitoring, as well as adapting the monitoring to the unique aspects of the structural system and unique features of the damage that is to be identified. Thus, the design of an SHM system must take into account the specifics of the bridge as well as the goals defined by the relevant bridge owners and/or authority.

Extending an initial proposal summarized in Ref. 3, one must try to answer eight essential questions regarding the implementation of an SHM system in a bridge:

1. What are the life safety and/or economic justifications for monitoring the structure?

2. What are the risks associated with the structure?

3. How is damage defined for the structural system being monitored?

4. What are the operational and environmental conditions under which the structural system of interest operates?

5. What are the limitations on acquiring data in the operational environment?

6. Which level of knowledge do we need about the structural condition?

7. How long do we need to monitor?

8. What is the rate of return on investment in SHM?

The answers for those questions may be presented as follows:

• Most high-capital-expenditure civil engineering structures, such as bridges, are one-of-a-kind structural systems, influenced by the physical environment where they are built. Therefore, it is more difficult to incorporate lessons learned from other nominally similar structural systems to define anticipated damage.

• Structural designs are often driven by low-probability, but extreme-impact events, such as earthquakes, hurricanes, terrorist actions, or floods.

• A ranked list of risks must be addressed by the SHM system. Designing an SHM system starts by identifying the risks associated with the specific structure. The risk analysis will lead to a list of possible damages and impose limitations on the usage of FE models. The severity and probability of each risk will be classified using the usual risk analysis procedure to produce a ranking of risks. In this context, risks that are more likely to occur simultaneously or cascading will deserve special attention. Some risks will be retained, others may be addressed by regular inspection, and the remaining will be dropped because of low impact and/or probability. ⁴³

• Identifying the existence and location of damage can be done in an unsupervised learning strategy, but identifying the type of damage present and the damage severity can generally only be done in a supervised learning strategy. ⁴⁴

• Generally, structural systems like bridges degrade slowly under normal use: corrosion and fatigue cracking, freeze–thaw/thermal damage, loss of pre-stressing forces, vibration-induced connectivity degradation, malfunction of expansion joints and bearings, etc. Therefore, the monitoring time period must take that into account.

• There is no widely accepted procedure yet to demonstrate the rate of return on investment with an SHM system. It is very hard to show the return on investment unless we can provide the owner with an accurate definition of the probability of damage detection. It is hard to provide this unless we can define the damage to be detected and the probability of future loading that may produce that damage. In one study, based on lessons learned from six different SHM systems on highway bridges, the best return on investment was realized from SHM instrumentation that was focused and limited in scope. ⁴⁵ In order to address this issue, several authors have proposed an approach based on the “value of information” (VoI). In SHM, VoI has its roots in decision theory. The value of SHM information may thus relate to increasing benefits or decreasing costs. ⁴⁶ Zonta et al. ⁴⁷ proposed a methodology for economic evaluation of the impact of monitoring on bridge management using the VoI. The methodology developed uses Bayesian logic and decision theory and is based on the hypothesis that the bridge owners will decide to act when the loss expected for non-action is greater than the cost of action. Following the same principals, Straub et al. ⁴⁸ proposed a roadmap to quantify the benefit of SHM, along with an influence diagram to support the modeling process.

For a more comprehensive operational evaluation, it is authors’ belief that we must pursue more SHM field technology demonstrations. In Brownjohn et al., ⁵ three examples of practical UK bridge structural monitoring in the 21^st century, initiated by operator requirements, are described, aiming at identifying the practical benefit to operators. The case studies concern “quasi-static” monitoring primarily due to thermal effects and not involving dynamic response. For instance, the understanding of temperature variations and bearing movements obtained from roughly two-year monitoring was subsequently employed to choose appropriate replacement bearings for the Cleddau Bridge.

Even though dynamic performance may not usually be the issue, its measurement has been a preferred tool in academic SHM community. Indeed, due to the high-capital-expenditure and practical limitations, there are only a few real-world bridge structures where the actual expected damage could be measured before complete demolition: Z24 Bridge in Switzerland ⁴⁹ and S101 Bridge in Austria. ⁵⁰

The Z24 Bridge was a standard post-tensioned concrete box girder bridge composed of a main span of 30 m and two side-spans of 14 m, as shown in Figure 8. The bridge, before complete demolition, was extensively instrumented and tested with the aim of providing a “feasibility tool” for vibration-based SHM in civil engineering, under the SIMCES Project. A long-term monitoring program was carried out, from 11 November 1997 until 10 September 1998, to quantify the operational and environmental variability present on the bridge and to detect the existence of damage artificially introduced. Every hour, environmental quantities, such as temperature at several locations, were measured from an array of sensors. Additionally, every hour, eight accelerometers captured the vibrations of the bridge for 11 min. For the purpose of extracting modal properties from the acceleration measurements, the bridge was assumed to be excited with white noise derived from operational conditions. Progressive damage (settlement, concrete spalling, landslide at abutment, concrete hinge failure, anchor head failure, and rupture of tendons) was inflicted shortly before the demolition of the bridge (from 4^th of August to 10^th of September 1998), in order to prove that realistic damage has a measurable influence on the bridge dynamics.

Figure 8.

Z24 Bridge in Switzerland.

The SIMCES project provided a unique database of high-quality, long-term measurements and damage test data, becoming the most successful benchmark data set in the history of bridge SHM. A number of interesting conclusions with regard to environmental influences, system identification, and damage identification (levels 1, 2, and 3) were drawn in Ref. 51. Other studies on this data set can be found in Refs. 31, 32, 40, 49, 52–65.

In 2008, the S101 Bridge (Figure 9), a post-tensioned three-span concrete bridge from the early 1960s was also progressively damaged in a large-scale test before its complete replacement. The static and dynamic behavior of the structure was measured continuously during the three-day damaging process to provide a benchmark for bridge SHM. The damage scenarios comprised pier settlements and tendon cuts. It was shown that the proposed methodology was able to clearly indicate the presence of structural damage and whether the damage led to a change of the structural system. In the course of the progressive damage test, it could be seen that even small damages (e.g., the pier settlement of only 1 cm) did cause distinct changes in the dynamic response of a structure. ⁵⁰ Dohler et al. ⁶⁶ demonstrated a clear link between the change in the modal parameters and the damage scenarios. With a statistical damage detection test, which considered the first 12 hours as reference state condition to output a periodic damage indicator, the authors found a link between the damage indicator at each test stage and the structural changes due to pier settlement. Nonetheless, for the cutting of single prestressing tendons, an early-stage change of the dynamic response behavior could not be indicated. However, no relevant temperature or other environmental changes were present, so the robustness of the methods to such changes could not be shown.

Figure 9.

S101 Bridge in Austria.

Data acquisition

The data acquisition portion of the paradigm involves selecting:

• The sensor types, numbers, and locations,

• The data acquisition/storage/processing/transmittal hardware and, if necessary,

• The excitation methods.

The actual implementation of this portion of the SHM process is application-specific. ³ There must be a trade-off between technology efficiency and cost. It is observed that the most important factors determining the quality of sensors and data acquisition (DAQ) hardware and their reliability are the long-term sensing ability, sensitivity, sampling rate, size, power consumption, and means of communicating data. Where multiple sensor systems are used, as for example with the monitoring system of the Humber Bridge, ⁶⁷ fusion of data streams can be complex, requiring baseline shifts and interpolation. The monitoring system was installed with Global Positioning System (GPS) plus data acquisition hardware for analog and digital signals for acceleration, displacement, and weather, all with different sample rates, time stamps, forms, availability, quality, etc. and requiring up-front design of a data fusion strategy.

Traditional sensor networks can only provide measurements at discrete point-wise spatial resolution, as usually only a limited number of sensors can be placed over the structure surface. Full-field monitoring is (so far) hardly achieved, which limits the effectiveness of the damage identification process, as the spatial resolution of sensor measurements are critical for numerous mode shape-based damage identification methods. ^68,69

A fundamental axiom regarding data acquisition is that sensors do not measure damage. ⁴⁴ Rather, they measure either the response of a system to its operational and environmental loading or the response to inputs from actuators embedded with the sensing system. Depending on the sensing technology deployed and the type of damage to be identified, the sensor readings may be more or less directly correlated to the presence and location of damage. Data treatment procedures, which are carried out in the feature extraction and statistical modeling for feature classification portions of SHM are the necessary components of an SHM system that convert the raw sensor data into information (or knowledge) about the structural condition.

Thirty years ago, the data acquisition for long-term monitoring was essentially based on electrical wired sensors and DAQ hardware to interrogate the signals. Nowadays, the desire of full-field monitoring and the technological revolution have pushed developments in non-contact sensing and enabled the integration of traditional sensors in mobile devices and/or attached with autonomous data acquisition and intelligent data treatment packages. It is worth noting that an intelligent and optimal measurement system that allows remote condition monitoring of bridges is a major goal for the development of an efficient SHM strategy.

This section provides a comprehensive survey of various sensing technologies and their evolution, organized by contact (wired or wireless systems) and non-contact sensing.

Wired systems

Traditional sensors [e.g., accelerometers, strain gages, linear variable differential transformers (LVDTs), and inclinometers] are normally posed in the context of wired monitoring systems. To guarantee that measurement data are reliably collected, most long-term SHM applications are still based on wired monitoring systems for communication between sensors, DAQ hardware, and the repository (local or remote computer).

The term “wired” is used to differentiate between wireless connections and those that involve cables. Inside the category of wired systems, we may differentiate between electrical and optical systems. Mechanical sensors, which detect some form of mechanical deformation and translate it into an electrical signal, are barely used nowadays in automatic applications.

The electrical systems are still the norm in long-term SHM, even though they can only measure at discrete points. Usually uniaxial/triaxial accelerometers are used to sense vibrations and are directly fixed to the bridge. The high sensitivity and high signal-to-noise ratio make them the perfect sensors to derive the modal properties from bridges under ambient vibration. Accelerometers do not provide a direct measurement of displacements; however, assuming proper initial conditions and high-pass filtering, double integration of accelerations can derive useful results over limited timescales. Along with using a low-noise accelerometer, the filtering is important to avoid a low frequency drift of the derived displacements as low frequency noise on the acceleration data tends to blow up the integration.

Generally, two types of strain gauge are applied in bridges: foil strain gauges (especially for steel bridges) and vibrating wire strain gauges (more robust for concrete applications). Dynamic strain measurements can deliver strains which may be correlated with stresses and modal properties. However, the minimum strain reading is around 1με (microstrain), jeopardizing their use for operational modal analysis, particularly since strain is the first space derivative of displacement compared to the second time derivative which enhances accelerations at dynamic frequencies. Hence, the low dynamic output level of a strain gage makes strain measurements susceptible to interference from other sources of electrical energy, which is the downfall of this type of sensor. LVDTs are electromechanical sensors used for measuring linear displacements. Due to the need of a reference point, in long-term monitoring the LVDTs are usually used to measure displacements at the expansion joints, generally caused by temperature variations. Potentiometers and pull-wire sensors are also be used to measure displacements, usually on a local scale.

Inclinometers are also used in bridges, especially for long piers of multi-span bridges. For instance, gravity-referenced inclinometers were installed to measure rotations at the top and bottom of both main piers of the São João Bridge, in Porto, ⁷⁰ while Hester et al. ⁷¹ used rotation measurements for bridge deck condition assessment.

The use of fiber-optic sensors in concrete was first suggested in late 1980s. ⁷² Subsequently, their application was extended to reinforced concrete structures. In the last two decades, a significant number of innovative sensing systems based on optical fiber sensors have been exploited in the civil engineering community due to their inherent distinctive advantages such as small size, light weight, immunity to electromagnetic interference and corrosion, embedding capability, the scale flexibility for small-gauge and long-gauge measurements, and relatively rapid installation which brings down the overall costs. The optical fiber sensing technology is capable of measuring strains, temperatures, accelerations, displacements, cracks, and corrosion. ^73,74

Fiber-optic sensors can be split into two big categories: point and distributed sensors. The former type, such as the fiber Bragg grating (FBG), measures the physical parameters only at a particular location but with relative high resolution and sensitivity. The detectable range of the latter one, such as optical time-domain reflectometry, is relatively large (usually the fiber length itself) and continuous but with only moderate resolution and limited sensitivity. However, numerous investigations have been carried out to design a novel sensor with the advantages of both types. ⁷⁵

The FBG-based sensors have been the most used in bridge monitoring. Although fiber-optic sensors are ideally suited for high-resolution monitoring, such as strains and temperatures, some logistical challenges to their routine installation during the construction process are being addressed; appropriate measures must be taken to avoid damage to the fibers from the harsh environment that exists on most construction sites and historically loggers were expensive and challenging to deploy on site. Additionally, efforts still need to be made to develop more sensitive accelerometers for civil applications in ambient vibration testing.

Despite the high popularity, distributed fiber-optic sensors have not been used extensively in bridges. The main applications of these sensors include the health monitoring for pipelines, levees, and landslides along railroads. A practical guide for distributed fiber-optic strain sensing for monitoring civil infrastructure can be found in Ref. 76. In Ref. 43, distributed strain monitoring systems have been selected because of its capability to monitor long lengths through a single cable, thus simplifying installation. The use of distributed fiber-optic sensing will help in overcoming the issue of optimal sensor location allowing full structure coverage over several kilometers.

However, there is still a lot of work to be done if fiber-optic sensor is to become a comprehensive, definite, and high-level feasible solution in SHM applications. ⁷⁷ Note that for campaign tests the pros and cons are different as one-time investments in running “wires” do not pay off so much.

Wireless systems

Two decades ago, wireless sensor networks (WSNs) emerged in SHM as a powerful low-cost platform for connecting large networks of sensors, in terms of both installation and maintenance. An early demonstration of wireless communications as a means for eradicating cables within a structural monitoring system was provided by Straser and Kiremidjian ⁷⁸ and a comprehensive review of WSN for SHM was presented in Ref. 79. A summary of the key differences between wired sensor networks and WSNs for SHM is presented in Ref. 80.

Wireless sensors are not sensors per se, but rather are autonomous data acquisition nodes to which wireless communication and traditional sensors can be attached. Perhaps the greatest attribute of the wireless sensor is its collocation of computational resources with the sensor.

One of the first WSNs for SHM was installed on the Golden Gate Bridge in 2007 by a research team at the University of California in Berkeley. ⁸¹ In this network, ambient vibration data was routed from the origin sensor node to a centralized base station, which processes the data and makes decisions regarding the structural health condition. With a total of 64 sensor nodes deployed on the bridge, this system is still one of the largest WSN-based SHM system to date. ⁸⁰ Another WSN-based SHM system has been deployed on the ZhengDian Bridge in China. ⁸² According to the authors, the limited battery lifetime is the most important problem to be solved in the process of engineering applications. The Jindo Bridge SHM system constitutes the largest deployment of wireless smart sensors for civil infrastructure monitoring to date. ⁸³ In total, 70 sensor nodes and two base stations have been deployed to monitor the bridge using an autonomous SHM application with excessive wind and vibration triggering the system to initiate monitoring.

The WSNs have the advantage to easily scale to a large number of nodes to allow dense sensor coverage of bridges. Generally, the frequency sampling is limited up to 100 Hz. However, after two decades, the power consumption is still the major limitation of wireless sensors operating on batteries, which has limited the WSN to temporary SHM applications. Besides, wireless systems may suffer unexpected constrains like lack of reliable data transmission (e.g., in a bridge box).

A final note concerns accelerometers based on microelectromechanical systems (MEMS), which have proven to offer a suitable solution for SHM in bridge applications. ⁸⁴ Recent developments in the fields of MEMS (both wireless and wire options) have introduced new opportunities to reduce the installation costs of structural monitoring systems. MEMS technology has led to the development of sensors that are low cost, low power, compact, and easy to install and can offer superior low-noise performance.

Non-contact sensing

The last decade has witnessed unprecedented growth in bridge health assessment using non-contact sensors such as cameras (photogrammetry and videogrammetry) and lasers. Other non-contact technologies such as radar and acoustics have shown to be capable of remote monitoring over very large distances. When employed with the aid of mobile platforms, such non-contact technologies, especially cameras, have shown to enable rapid assessment of the surface condition of bridge components, even in otherwise hard-to-access areas of the structure. Nonetheless, many issues regarding their applicability specific to the domain of bridge monitoring still need to be fully addressed, such as noise robustness, resolution, scalability, and accuracy, before such noncontact technologies can be proven as viable alternatives to conventional contact sensors. ⁸⁵ Platform stability—that depends on compensation technique—and weather/lighting conditions are also two limitations. Even though GNSS-based sensing still requires installation of receivers on the structure, it is herein considered a non-contact technology.

Video-based approaches

Video-based approaches are rooted in the photogrammetry whose principles were first developed as a means to create maps from aerial photographs. Photogrammetry has been defined as the process of deriving metric information about an object through measurements made on photographs of the object. ⁸⁶ Photogrammetry is based on remote sensing and digital image processing. By tracking discrete points in photographs (or frames) taken by cameras with photogrammetic principles, shape, strain, and displacement can be measured.

Even though it has been around for decades, ⁸⁷ thanks to the rapid advances in computer vision, the camera-based non-contact vision sensor has emerged as a promising practical alternative to conventional contact sensors for structural dynamic response measurement and health monitoring. ^88,89 Significant advantages of the vision sensor include its low cost, ease of setup and operation, and flexibility to extract displacements of any point in the structure from a single video measurement, by converting pixel displacements to physical displacements.

Vision-based displacement sensing systems typically comprise a video camera (or cameras), a zoom lens (or lenses), and a computer. ⁹⁰ The vision sensor can be termed as a non-contact distributed sensing technique as it can, simultaneously, tracking multiple points from a long distance.

Combined with advanced image processing algorithms such as optical flow ⁹¹ and 2D or 3D digital image correlation (DIC), ^92,93 successful studies have been demonstrated by structural dynamics (full-field structural dynamics response measurements and subsequently experimental modal analysis ⁹⁴ ) and health monitoring (damage detection) communities. ⁹⁵

A review of machine-vision-based methodologies for displacement measurement in bridges can be found in Ref. 96. Where feasible, the capability of a vision-based systems can be powerfully enhanced by leveraging the high resolution capabilities of even a single fixed accelerometer, as proposed in this study for accurate displacement measurement. ⁹⁷

For large size structures such as long-span bridges, multiple synchronized cameras targeting different sections of the structure can be applied to monitor the entire structure. Most of the current field studies have focused on measurement of relatively large amplitude displacements such as bridges subjected to moving train loads. The various noise sources in complex outdoor conditions such as heat haze pose challenges for accurate measurement of small-amplitude displacements, such as response of short- or medium-span concrete bridges under light-weight vehicles or ambient excitations.

Even though the technology has recently attracted a lot of attention in academic bridge SHM, vision-based sensor technology for SHM is still in its infancy in real-world applications. The majority of studies have still been focused on measurements of small-scale laboratory structures or field measurements of large structures at a limited number of points for a short period of time. ⁹⁰ In the near future, the technology is expected to be deployed in long-term bridge applications to fully validate its performance for damage identification in outdoor field environments. As with increasing capabilities in consumer-grade systems, professional systems feeding demand for motion capture in other domains such as sports may propagate to civil structure applications. ⁹⁸

Although these techniques offer clear advantages, they typically (but not necessarily) rely on speckle patterns or high-contrast markers positioned on the structural surface. The spatial position of these high contrast areas can be monitored over frames in order to compute local deformations using image intensity correlation or feature-point tracking. ⁹⁰ Moreover, that approach relies on the spatial context of spatio-temporal data (frames), similar to those seen in the traditional optical-flow methods. ⁹⁹ For wider acceptance of vision-based modal analysis methods, it is most desirable to develop methods that utilize the video measurements without additional structural surface preparation. Although very recent and impressive formulations have been proposed for operational modal analysis, ⁸⁹ they are still dependent on several user input parameters and supervision that are not suited for efficient and automated implementation on site applications.

A recently proposed method, ⁶⁸ termed full-field video dynamics algorithm, has been found to be efficient and can be implemented automatically. The main advantage of this technique is to process the video data into the temporal dimension only (time-series related to the pixel intensities) without considering the spatial context (frames). The end result is to indirectly perform spatial processing by looking at the temporal correlations of pixel time-series, which has been proven to effectively decompose individual vibration responses. Video of vibrating structures is inherently sparse; typically, video of vibrating structures has a dimensionality consisting of thousands or even millions of pixels and hundreds to thousands of frames yet the motion of the vibrating structure can be described using only a few mode shapes and their associated modal coordinates. This means that the video potentially contains large amounts of redundant information that increases the computational demand for video dynamics processing, but also allows one to effectively estimate vibration parameters. The proposed method exploits the physical relationship between unsupervised machine learning systems and structural dynamic models to process high-dimensional pixel measurements and motion information. Through a family of unsupervised machine learning algorithms, principal component analysis for dimension reduction and blind source separation for modal decomposition, it is able to perform automated, unsupervised, and efficient identification of the output-only structure’s modal frequencies, damping ratios and full-field (full pixel resolution) mode shapes. These modal properties can be used for system identification or as damage-sensitive features. ^100,101

Laser-based approaches

Non-contact displacement measurements made with laser scanning provide high spatial resolution sensing capacity without the need of installed sensors on the structures or inducing mass-loading effects. However, these measurement devices are relatively expensive and perform measurements sequentially, which could be time and labor tedious when the desired sensing areas are large.

Laser Doppler vibrometer (LDV) is the most common laser-sensing system for vibration-based health monitoring. The LDV is a precision optical transducer used for determining vibration velocity and displacement at a fixed point, based on the Doppler-effect: sensing the frequency shift of back scattered light from a moving surface. By attaching a scanning unit of mirror in front of the laser sensor head, scanning measurement for multiple points can be realized.

Light detection and ranging (LiDAR) is another laser-sensing system. A LiDAR estimates a distance from the device to a target point by measuring the time of flight of the incident laser pulse reflected off from the target point or by measuring the phase shift of the reflected laser beam with respect to the incident continuous wave laser beam.

Even though LiDAR has been used extensively for airborne laser scanning and bridge 3D representation, it still lacks the required accuracy for damage detection at bridge level. ¹⁰² Nevertheless, long-term displacement measurement of bridges using a LiDAR system was tested in Ref. 103. Strategically deployed reflectors provided a reference point, which is independent of the LiDAR position. Thus, LiDAR can be temporarily installed in the field only when measurements are necessary. Expensive LiDAR systems can be employed to measure the long‐term displacement of multiple bridges cost‐effectively because permanent installation is not required. However, this is a promising technology as nowadays certain smartphones are already equipped with this technology.

Additional laser technique with potential application to SHM includes the laser ultrasonic techniques, which is able to detect small-scale damages. Laser ultrasonic techniques identify a defect by measuring interaction between ultrasonic waves and the defect. Conventionally, ultrasonic waves are generated and measured using contact-type transducers such as piezoelectric ones, but nowadays non-contact laser solutions are also available. ¹⁰²

Global navigation satellite system

Over the last two decades, global navigation satellite system (GNSS) technology has been developed rapidly and applied to civil structures for appropriate monitoring of structural performance. Pioneering research on the dynamic monitoring of bridges using GNSS begun in the late 1990s. ¹⁰⁴ Initially, only the America’s GPS was available and even though most of the studies are still performed on GPS data, nowadays the GNSS consist of America’s GPS, Russia’s GLObalnaya NAvigatsionnaya Sputnikovaya Sistema (GLONASS), EU’s Galileo Navigation Satellite System (Galileo), and China’s BeiDou navigation satellite system (BDS).

Currently, this technology can only be applied usefully to monitor flexible structures with low modal frequency ranges, like suspended bridges, and it still has remaining issues when it comes to obtaining accurate measurements. ¹⁰⁵ However, advances in GNSS devices, multi-sensors techniques, and algorithms can mitigate erroneous sources of GNSS data. Many studies have been performed to investigate the use of the GNNS measuring the global deformation of large structures. ^106,107,108

The combination of real-time kinematic (RTK) with GNSS allows for more precise dynamic measures. RTK-GNSS consists of at least one base station to establish a known point, and rover stations established at measurement points in the structure. The real‐time coordinates of measurement points can be computed using at least four satellite signals and correction data broadcasted by base station. As demonstrated in the Humen Bridge, China, ¹⁰⁹ for wind-induced vibration data, the combination of RTK with GPS allows for dynamic position measurements with stated accuracy of ±5 mm for the horizontal position and ±10 mm for the vertical position. Three signal analysis techniques were applied to obtain modal parameters (the first lateral, vertical, and torsion modes) from the same wind-induced response, such as the time–frequency Wigner distribution technique, the frequency-domain fast Fourier transform technique, and the time-domain auto-regressive moving average vector technique.

Using precise controlled artificial oscillations, Nickitopoulou et al. ¹¹⁰ characterized GPS accuracy as 15 mm and 35 mm in horizontal and vertical coordinates, respectively, for 98.5% confidence interval. However, recent studies on GPS monitoring with advanced devices and algorithms demonstrated that the GPS technique can resolve low amplitudes of displacements within millimeter levels of accuracy. ¹⁰⁵ There are numerous spurious error sources for RTK-GPS that do not characterize as Gaussian noise, for example, due to environment, multi-path, and satellite positioning whose effects could be erroneously interpreted as structural movements so caution is required in data interpretation. Disturbances due to structural elements in the direct environment of the measurement point ^111,107 can also impact accuracy.

Even so, there are numerous demonstrations of the efficiency of RTK-GPS in full-scale measurements. For example, at the Nottingham Wilford suspension footbridge, United Kingdom, modal properties were extracted from low-frequency (<5 Hz) vibration response to in-service loads by processing the signal with the wavelets transform. ¹⁰⁶ In 2012, the authors ¹¹² estimated the first natural frequency (0.29 Hz) of a pillar from a bridge in northern Portugal, using the RTK-GPS combination, using a frequency sampling of 20 Hz. In this study, environmental interference as mentioned above was observed.

The main advantage of GNSS-based displacement measurement is its autonomous applicability, which do not require a fixed point (base station) in the direct environment of the structure. One possible disadvantage of GNSS and RTK-GPS is the limited sampling rate (up to 20 Hz or in certain cases up to 100 Hz), but there may be little advantage in measuring high-frequency signals. For example, if the accuracy of GPS is accepted as 1 mm, this translates to an unfeasible acceleration of 16 m/s² for a sinusoidal vibration, conversely a feasible 20 Hz bridge vibration of 1 m/s² translates to displacements not resolvable by GPS. The higher sample rates are suited to vehicle tracking or measurement averaging to improve static GPS measurements.

Recently, multi‐constellation GNSS is regarded as a promising development for SHM; that is, the combination of GPS and other GNSS constellations can enhance the overall performance of the monitoring system. Additionally, with continuous enhancements in the algorithm and hardware of GNSS, it is expected that the application of GNSS technology can be expanded to bridge cable‐force measurements and bridge weigh‐in‐motion as well as structural deformation monitoring. ¹⁰⁷

Unmanned aerial vehicles

Unmanned aerial vehicles (UAVs) equipped with cameras have emerged as valuable tools for remotely performing measurements in difficult access places.

Their potential time-and-cost-related benefits have already rendered them attractive for structural surveying. A recent report ¹¹³ provides a comprehensive state-of-the-art review of research work and applications focusing on the employment of UAVs for inspection, monitoring, and maintenance oriented towards facilitating an efficient structural health management.

In SHM, typically these UAVs are combined with DIC to perform non-contact, optically based measurements to monitor the health of bridges. Reagan et al. ¹¹⁴ developed a system to provide accurate crack measurements over several areas of concrete bridges using 3D DIC; while improving accessibility, this system was able to detect crack changes in the order of 10⁻⁵ m. UAVs normally require skilled pilot/operators and GNSS for autonomous flight (which might be critical for inspections beneath a bridge, for instance). UAVs can only operate in restricted airspace and require formal operator certification.

Kang and Cha ¹¹⁵ proposed and tested in a laboratory environment an autonomous UAV using ultrasonic beacons to replace the role of GPS, a deep convolutional neural network (CNN) for damage detection, and a geo‐tagging method for the localization of damage. Concrete cracks, as an example of structural damage, were successfully detected with 97.7% specificity and 91.9% sensitivity, by processing video data collected from an autonomous UAV.

An innovative approach for quantitative strain measurement was proposed by Ong et al. ¹¹⁶ The authors used a UAV for 3D scanning of a membrane structure and generating an FE model. After applying a known deformation to the membrane, they updated the 3D model (and consequently the FE one) with imagery from the deformed membrane. The FE model could, then, be analyzed to estimate the strains and, subsequently, the stresses in the deformed membrane.

In conclusion, UAV-based SHM techniques were found to favor the challenging tasks of: (i) the development of the digital replica (twin) of either a structure or an infrastructure asset and (ii) the detection of damages and defects based on image analysis. However, UAV-facilitated SHM is in its infancy and hence, several early stage symptoms need to be treated meticulously. ¹¹³ Future research activities are essential on advancements in image analysis and higher levels of autonomous operation (e.g., longer battery life).

Other non-contact technologies

Other non-contact technologies such as infrared thermography, radio detection and ranging (radar), acoustic emissions, and infrasound have shown to be capable of remote monitoring over very large distances.

Infrared thermography, also known as thermal imaging, is the detection of electromagnetic waves in the infrared spectrum. It utilizes the infrared spectrum to show differences in heat dissipation from a structure using a thermal imaging camera. ¹¹⁷ It permits one to visualize and quantify damage by detecting changes in heat transfer characteristics near the defect. Infrared thermography has been used scarcely for long-term SHM of bridges. Nevertheless, in the context of NDE (e.g., using drones), infrared thermography is commonly used for the detection of concrete deck delamination ¹¹⁸ by looking at changes in surface temperature.

An interferometric radar for non-contact measurement of deflections on bridges is described in Ref. 119. An innovative radar sensor was developed for remote (non-contact) measurement of deflections on bridges, in both static and dynamic conditions, with displacements sensitivity ranges between 1/100 and 1/10 mm.

Ground-penetrating radar (GPR) is one of the most used electromagnetic wave-based techniques in civil engineering owing to the noninvasive, low costs, and rapid monitoring nature of the technique. The GPR technique is also investigated to inspect the asphalt pavements in order to detect voids. ¹²⁰ GPR is effectively an NDE technique and the potential for SHM is still to be shown.

The acoustic emission technique has been proposed to detect elastic waves in structural components generated by damages, such as the initiation and propagation of cracks in concrete and the failure of steel wires. ¹²¹ In particular, it has shown potential to be a powerful technology for cable strand breaks in tensioning tendons and suspension bridge cables. ¹²²

Even though infrasound has long been used by geophysicists and the nuclear monitoring community, the use of infrasound in relation to infrastructure monitoring is relatively new. Infrasound, sometimes referred to as low-frequency sound, describes sound waves with a frequency below the lower limit of audibility (generally 20 Hz). In northern California, one study focused on the technique of infrasound for infrastructure monitoring with a detailed case study, involving a two-girder steel bridge, showed that infrasound was capable to detect natural modes of the structure from a distance of 2.6 km. ¹²³

One of the common challenges within those techniques is the interpretation, analysis, and automation of obtained data, which requires highly skilled and specialized experts.

Indirect monitoring

Indirect bridge monitoring methods, using the responses measured from vehicles passing over bridges, are under development for about a decade. A major advantage of these methods is that they use sensors mounted on vehicles, so no sensors or data acquisition system need to be installed on the bridges. Most of the proposed methods are based on the identification of dynamic characteristics of the bridges from responses measured on vehicles, such as natural frequencies, mode shapes, and damping ratios. In addition, some of the methods seek to directly detect bridge damage based on the interaction between the vehicle and bridge. In Ref. 124, it is reported that the main challenges for indirect bridge monitoring methods are: (i) the road profile, (ii) the limited vehicle-bridge interaction time, and (iii) environmental effect.

A crowdsourcing-based framework using smartphones for bridge health monitoring was also recently proposed by Mei and Gul ¹²⁵ and tested in laboratory environment. The damage detection is performed on vibration data using smartphones in a large number of moving vehicles as mobile sensors. It must be acknowledged that smartphones have currently several sensing limitations compared to traditional sensors, such as lower sampling frequency and lower resolution, but they are widely used and have access to the Internet.

Feature extraction and generation

The feature extraction and generation stage of the SPR paradigm has been well described in Ref. 19. Feature extraction generally involves data transformation and dimension reduction through a multi-step fusion process, in order to remove noise and unveil structural-related signatures from the raw data. The “generation” is herein added in order to include features derived from numerical models. Feature extraction is often described as a key step to ensure the reliability of SHM systems, as it directly affects the effectiveness and accuracy of the damage identification.

During the condition monitoring of a structure, there are often multiple sensor types and groups for different physical parameters distributed around the structure, whose data streams may be acquired at different sample rates and fused into the collected raw data. From the collected raw data, features sensitive to structural damage are extracted to support the structural condition evaluation. A traditional damage-sensitive feature is some quantity extracted from the structural response data that is correlated with the presence of damage in a structure (e.g., modal parameters, maximum displacements, regression model parameters, and residual errors). An adequate damage-sensitive feature varies consistently in accordance with the level of damage in the structure. However, in most real-world applications, the features directly extracted from the raw data are often correlated with multiple sources of operational and environmental variability (e.g., traffic, temperature, moisture, and wind speed). This increases uncertainty in the damage detection as it can mask damage-related changes in the features as well as alter the correlation between their magnitude and the damage level. ²⁷ Besides, there is no universal kind of feature or physical variable appropriate for all possible damage scenarios. Thus, the type of feature to be used varies with the kind of structure to be monitored and the type of damage to be found. These facts suggest the usage of several data transformation techniques to extract alternative features in order to reduce uncertainty and increase the likelihood of damage detection through data fusion.

However, if we can only detect damage when there is information in the raw data sets, why don’t we perform the statistical modeling for feature classification directly on the raw data? It is because of computational limitations and inefficiency: With a set of features obtained from the data, the statistical modeling for feature classification can be performed with less uncertainty and with minimal computational efforts.

Feature extraction domains

Many different techniques from alternative domains have been used for data transformation and feature extraction in SHM applications, ¹²⁶ depending on the type of raw data used. What is the nature of the raw data? Is it quasi-static (low-frequency sampling) or dynamic (high-frequency sampling)? The most common ones fall into the following domains ¹²⁷ :

• Time-domain features—mostly regarded to time-series processing. Common techniques use the coefficients of auto-regressive (AR) models and auto-regressive moving average (ARMA) models as input features ^128,129 Although providing stable results, they work best for slightly damped dynamic systems and require a significant number of time-domain samples.

• Frequency-domain features—conventionally related to the Fourier transform. Frequency components are estimated from raw data in order to decompose and analyze their spectral content. ¹³⁰ In this domain, the modal properties ^31,131–135 include vibration parameters (e.g., natural frequencies, mode shapes, modal curvature, and flexibility matrix) as features for damage detection, as they provide global on-site automated continuous monitoring due to their inherent physical properties (stiffness and mass).

• Time–frequency-domain features—often represented by approaches based on the short-time Fourier transform (STFT), providing an effective characterization of time–frequency patterns of non-stationary signals.

• Wavelet-based features—used to achieve a reasonable trade-off between frequency and time resolution with multiple choices of basic functions available. Conventionally performed using discrete wavelet transform (DWT) and wavelet packet transform (WPT). They can be very effective features when modeled to match specific characteristics of the structure being monitored.

• Empirical mode decomposition (EMD) features—performed by advanced signal processing algorithms, they are built to adaptively model time–frequency characteristics of non-stationary and/or nonlinear structural systems. Their end result is usually the decomposition of the input raw data into intrinsic mode functions (IMF).

Levels of feature extraction

Feature extraction techniques can be performed at different levels of complexity. ^136–138 The feature extraction itself can be performed at multiple feature fusion processes. It is possible to follow the conventional fusion operation level to divide the techniques into three levels as follows:

1. Data-level fusion (first level)—performed on the measured raw data by integrating multi-sensor signals through the several feature extraction domains. It comprises a first-level set of extracted features. Most features fall into this first category.

2. Feature-level fusion (second level)—the process is performed on the first-level features by compressing them into a reduced set of derived variables; these second-level features retain the essential information of the original subset of features with less data and in a lower dimensional space.

3. Decision-level fusion (third level)—intrinsically related to the feature classification step, the results of multiple classifiers are combined through majority voting approaches and used to perform further analysis. The output is not a new set of features, but some useful pattern or knowledge. At this level, other sources of information can be used, such as visual inspections.

Figure 10 shows a taxonomy of feature extraction methods conventionally used in SHM applications for bridges. In some cases, techniques used to derive first-level features can be used in a second-level feature extraction fashion (e.g., correlation techniques). Very often, the second and third levels can be often found in the literature as part of the fourth stage—statistical modeling for feature classification.

Figure 10.

Taxonomy of feature extraction methods for SHM.

The majority of the SHM studies have been focused on the first-level feature extraction methods, by identifying the static and dynamic properties of a monitored structure to relate changes in the vibration characteristics to the damage occurrence. ^{10,126,139–142} Examples of such features are: statistical moments (mean, standard deviation, skewness, and kurtosis), auto-regressive parameters and residual errors, modal properties (natural frequencies, mode shapes, and damping rations). Even though the first-level features may work well for intelligent fault diagnosis, they have deficiencies: (i) generally, the extracted features are handcrafted from time-domain signals and heavily depend on much prior knowledge and diagnostic expertise; (ii) those features compose very complex information under high influence of normal variability effects, which becomes difficult for the proper discrimination of damaged and undamaged conditions; and (iii) as the amount of data increases, more complex information must be learned, which enlarges the number of learned parameters to correctly perform feature classification and to identify the damage occurrence. ¹³²

For those reasons, the second-level feature extraction techniques have been applied for monitoring scenarios. ^143,144 Although each method can be applied for specific damage scenarios and with different goals, feature-level fusion may be more effective than data-level fusion, because the those features contain selected information about the structural condition, present physical interpretation, and have filtered out of some noise. ⁵⁹ However, technical and organizational issues have limited their application by structure managers in practice. Some reasons can be pointed out, for example, the lack of a framework or guideline for perform damage identification, which results in no universally accepted feature extraction method.

Examples of second-level features include those based on feature space transformation, such as the principal component analysis (PCA) which have been the most widely employed for SHM applications, ⁴⁰ those based on correlation techniques such as the Modal Assurance Criterion for mode shapes and those based on machine intelligence.

Considered to be the current breakthrough in artificial intelligence, deep learning-based techniques have the potential to overcome the aforementioned problems of current feature extraction methods. ^40,145 In particular, autoencoders are equipped with extremely powerful mapping operators, yielding an adaptive representation of the input data into its bottleneck layer. ^146,147 These networks can be stacked to form a deep architecture and to perform reliable mapping operations. An example of this framework for bridge SHM can be summarized as shown in Ref. 59, where the outputs at the bottleneck layer (second level) of a stacked autoencoder (SAE) that learns a compressed representation from natural frequencies (first level). Although this framework is specifically designed with deep autoencoders, in fact it summarizes the steps for any other application using different techniques to extract second-level features. These types of networks will be discussed in more detail in the Deep Learning.

Feature generation through numerical models

The most common features generated through calibrate numerical models are the modal properties. ³² Other type of features can be generated, such as modal coordinates, displacements, and stresses.

Statistical modeling for feature classification

The statistical modeling for feature classification generally includes data treatment and threshold definition. Several machine learning algorithms for data normalization and structural damage identification are described in the context of the SPR paradigm. These algorithms are especially relevant in cases where damage-sensitive features extracted from the structural responses are affected both by changes caused by operational and environmental variability and by changes caused by damage. Most of the machine learning algorithms are reviewed in the context of an outlier detection perspective.

An outlier detection perspective: Goals and challenges

In an outlier detection perspective for feature classification, one possible approach for data modeling is a clustering procedure combined with a (dis)similarity metric and a threshold. Thus, the goals of statistical modeling for damage detection are (1) learning clusters that correspond to the normal structural conditions (training phase); (2) estimating a threshold from the training data to classify new features (observations); (3) computing a damage indicator (DI) for each new observation considering the (dis)similarity between this observation and the centers of the clusters; (4) classifying each DI according to a defined threshold.

As shown in Ref. 64, the discovered clusters play a crucial role for mining patterns and extracting knowledge from the training data, as they can be correlated with the physical states of the structure under operational and environmental influences. Often, the better the clusters can represent the normal structural conditions, the better is the classification performance.

Some of the key tasks in the data modeling for feature classification are discussed as follows:

• Updating statistical models as new data becomes available.

• Choosing the machine learning algorithm for a specific task, which must be done as a function of the damage-sensitive features used as well as their distribution in the feature space.

• Posing the feature classification in the context of a binary classification, with a trade-off between false-positive and false-negative indications of outliers. It recognizes that false-positive classifications may have different consequences than false-negative ones. Therefore, analytical approaches to defining threshold levels must balance trade-offs between false-positive and false-negative indications, minimizing false-positives when economic concerns drive the SHM applications, and minimizing false-negatives when life safety issues drive the SHM systems.

Assessment of classification performance is a critical aspect of evaluating and comparing models, algorithms, or classifiers. For the two-class problem in SHM (binary classification), in which the two sets of cases are labeled as damaged (or positive, P) and undamaged (or negative, N), for each threshold there are four possible outcomes as summarized in Figure 11 and Table 1. For a positive outcome, the case can be either true positive (TP) if the observed is positive or false positive (FP) if the observed is negative. On the other hand, for a negative outcome the case can be either false negative (FN) if the observed is positive or true negative (TN) if the observed is negative. The shaded portion of Table 1 represents the confusion matrix (also known as contingency table), where the numbers along the major diagonal represent the correct classifications, and the numbers off the diagonal represent misclassifications, also known as Type I (FP) and Type II (FN) errors.

Figure 11.

Distributions from the undamaged and damaged conditions.

Table 1.

Accuracy of binary classification.

Outcome	Observed
	Positive	Negative	Total
Positive	True positive (TP)	False positive (FP)	TP+FP
Negative	False negative (FN)	True negative (TN)	FN+TN
Total	TP+FN	FP+TN	TP+FP+FN+TN

Receiver operating characteristic (ROC) curves provide a comprehensive and graphical way to summarize the performance of classifiers. ¹⁴⁸ The ROC curves were introduced in signal detection theory by electrical and radar engineers during the World War II for detecting enemy objects in battle fields. ¹⁴⁹ Since then, ROC curves have become increasingly common in fields such as finance, atmosphere science, and medicine. In the field of machine learning, these curves have become a standard tool to evaluate the performance of binary classifiers.

ROC curves focus on the trade-off between sensitivity and 1-specificity. As shown in Figure 12, the sensitivity is sometimes called the true-positive rate, TPR=TP/(TP+FN), and defines the fraction of true detection. The 1-specificity is sometimes called false-positive rate, FPR=FP/(FP+TN), and defines the fraction of false alarm. Each point on the ROC curve corresponds to a specific threshold, although the values of thresholds are not evident from the square plot. The diagonal line divides the ROC space into two regions and represents a classifier that performs random classifications. Any point in the upper-left triangle means that the classifier has some understanding of the classes; the closer the ROC plot is to the upper-left corner, the higher the overall accuracy of the classifier. On the other hand, any point in the lower-right triangle means that the classifier is performing worse than random; that is, the classifier has some underlying information about the classes but applies it in the opposite way.

Figure 12.

Example of a ROC curve; the diagonal line divides the ROC space into two regions and represents a classifier that performs random classifications.

Mahalanobis squared distance

The Mahalanobis squared distance (MSD) is a metric for multivariate statistics outlier detection. ⁵⁷ Taking a training matrix, with multivariate mean vector and covariance matrix, the MSD (a damage indicator) between feature vectors from the training matrix and any new feature vector (or observation) is computed. The assumption is that if a new observation is obtained from the data collected on the damaged condition, which might include sources of operational and environmental variability, the observation is further from the mean of the normal condition and might be considered an outlier. On the other hand, if an observation is obtained from a system within its undamaged condition, even with operational and environmental variability, this feature vector is closer to the mean of the normal condition. ¹⁹

Due mainly to its simplicity, the MSD has been extensively used in SHM as a distance metric for classification or as a machine learning algorithm. ¹⁵⁰

Principal component analysis

Traditional linear PCA has been adapted to model the effects of linear variations in order to act as a feature extraction and data normalization technique. ¹⁵⁰ In the case of bridges, PCA-based approaches are very often used to model linear environmental and operational influences from undamaged first-level features. ¹⁵¹

Nevertheless, in the SHM field, PCA might be used for four primary purposes which are:

(i) Evaluation of patterns,

(ii) Feature visualization,

(iii) Feature cleansing/normalization, and

(iv) Feature selection.

The evaluation and visualization of patterns in the data is achieved through a linear mapping of data from the original feature space into a transformed feature space, where one can identify patterns, define clusters, or identify outliers. For example, the sag at points along the span of a suspension bridge due to main cable heating alone exhibits a constant pattern that scales linearly with temperature so can be reduced to a single constant vector and a temperature dependent scalar. Feature cleansing is a process used to discard those linear combinations of the data that have small contributions to the overall variance, that is, the principal components with low eigenvalues. This process can be achieved by reversing the projection back to the original feature space using only the main principal components; back in the original space, the normalization step can be achieved by constructing a residual matrix. ³ Feature selection is the process of choosing a subset of the data matrix by constructing a new matrix composed of coordinates on the first principal components, whose dimension is smaller than the original matrix. The mapping of the data into a lower dimension space may result in a loss of some information that can be necessary to discriminate the undamaged and damaged conditions. However, note that PCA simply performs a coordinate rotation to align the transformed axes with the directions of maximum variance. Thus, there is no guarantee that those directions will preserve good features for discrimination. Additionally, a major difficulty with this technique is the limited applicability in real-world bridge SHM when nonlinearities are present in the monitoring data, as the linear PCA-based approach might fail to model the normal condition of a structure.

The auto-associative neural network (AANN) is a nonlinear version of the PCA intended to perform feature extraction, dimensionality reduction, and damage identification of multivariate data. ¹⁵² As demonstrated by Kramer, ¹⁵³ the AANN is capable of intrinsically performing a nonlinear mapping of the input variables, as it characterizes the underlying dependency of the identified features in terms of unobserved factors, that is, a nonlinear PCA (NLPCA). The main challenge in this approach is to determine the number of normal factors changing the actual behavior of the structure, as the number of factors retained by the model must be handcrafted discovered. ²⁷

An alternative manner to perform NLPCA is to employ kernel functions along with the traditional linear PCA. The kernel trick allows the mapping of features to high-dimensional spaces, which can provide room for adding nonlinear capabilities to the traditional PCA, leading to a kernel PCA (KPCA). ^38,154,155 Although this technique has achieved satisfactory results for data normalization and can be easily adapted for damage detection, ³⁹ it may also reveal some loss of information as the principal components are normally retained based on 99% of the data variability.

Moving principal component analysis (MPCA) is an improved method based on PCA which involves applying a moving time window of fixed size. Only the time series of observations inside the moving time window are used to construct the covariance matrix for the derivation of eigenvalues and eigenvectors. Several authors have already applied it to bridge SHM. ¹⁵⁶

Gaussian mixture models

One of the first publications on the use of Gaussian mixture models (GMMs) for bridge SHM was performed by Figueiredo and Cross ³¹ in 2013. The GMM is a probabilistic model that assumes all the features are generated from a mixture of a finite number of Gaussian distributions, aiming to capture the main clusters/components of features, which correspond to the normal and stable state conditions of a bridge, even when it is affected by extreme operational and environmental conditions Figure 13. Next, an outlier detection strategy is implemented in relation to the chosen components representing states. The damage detection is carried out based on multiples MSD-based algorithms, where the covariance matrices and mean vectors are functions of the main components.

Figure 13.

Feature space (first two natural frequencies) of the Z24 Bridge.

Later, in order to improve the parameter estimation of the GMM, Figueiredo et al. ⁶⁵ proposed an algorithm with a Bayesian approach based on a Markov-chain Monte Carlo method to cluster structural responses of the bridges into a reduced number of global state conditions, by taking into account eventual multimodality and heterogeneity of the data distribution. This approach stands as an improvement over the classical maximum likelihood estimation based on the expectation-maximization algorithm. Other works using GMM on bridge data sets can be found in Refs. 157–159.

Deep learning

In recent years, with the rapid progress of computing hardware, the deep learning-based data processing approach has added a new dimension for mining the massive data from an SHM system, moving towards autonomous, accurate, and robust processing of the monitoring data. Already several review papers and applications of deep learning for civil structures are available and a review on vibration-based damage detection in civil structures, from traditional methods to machine learning and deep learning, can be found in Ref. 22. Given enough computational resources, the biggest advantage of these types of neural networks is related to the capability to learn directly from the raw data. But what is the relation between neural networks and deep learning? A neural network is a biologically-inspired programming paradigm which enables a computer to learn from observational data; deep learning is a powerful set of techniques for learning based on neural networks.

Flah et al. ¹⁶⁰ performed a systematic review of machine learning algorithms in civil SHM. They concluded that despite their great success in some areas of research, neural networks are now outdated in SHM applications, while deep learning has emerged as a sophisticated subset of artificial intelligence, where CNNs have proved their great success especially for computer vision-based applications.

Ye et al. ¹⁶¹ also reviewed deep learning-based SHM of civil infrastructures. Azimi et al. ¹⁶² performed a state-of-the-art review on data-driven SHM and damage detection through deep learning. Avci et al. ²² performed a comprehensive review from traditional methods to machine learning and deep learning applications for vibration-based damage detection in civil structures.

Moisés et al. ⁴⁰ was one of the first studies to applied deep learning-based techniques for damage detection in bridges using the experimental natural frequencies on the Tamar and Z24 Bridges.

Transfer learning for domain adaptation

Whether supervised, unsupervised, or semi-supervised, machine learning algorithms for statistical modeling and feature classification have been developed assuming that both training and test data are drawn from the same underlying distribution. This assumption makes the machine learning algorithms specialized for a particular structure and application conditions. ²⁷

Unless test data are obtained from the same structure in the same operational and environmental conditions, the machine learning inferences from the training data will not provide accurate predictions when applied to the test data. Besides, new training data and labels must be measured for every new structure to train a robust model for classification, which is significantly expensive and almost impossible in the SHM context, especially for one-of-a-kind structures like bridges.

Transfer learning offers a novel solution to these problems by providing a method for mapping feature and label distributions from different structures, and labeled source and unlabeled target structures onto the same space. As a result, classifiers trained on a labeled structure in the source domain will generalize to a different unlabeled target structure. ¹⁶³ Transfer learning, a subfield of machine learning, aims to improve a learner from one domain by transferring knowledge from a related domain. Domain adaptation is a particular branch of transfer learning, where the focus is on reducing the distance between differing data distributions from source and target domains. A brief history and thorough survey on transfer learning and domain adaptation have been given in Pan and Yang. ¹⁶⁴

In bridges, this is particularly promising as the authorities can measure data from a small set of representative bridges, that is, not nominally identical but with structural similarities, to get enough samples to build machine learning algorithms to detect damage in bridges without long-time monitoring data. For instance, if the authorities own two main types of bridges, like simply supported beam bridge and three-span post-tensioned concrete box girder bridge, they should build only two algorithms trained on data sets from a small set of similar bridges. This fact may revolutionize the way we perform SHM, as we may move to population-based SHM, with the associated savings in investment and operation costs of permanent SHM systems. The foundations of population-based SHM for mechanical systems have been laid down in Refs. 34, 165, 166.

Transfer learning has already been applied for image classification. Matasci et al. ¹⁶⁷ provided a preliminary study of some methods able to generalize existing classification models’ predictive abilities to newly acquired target images. The authors applied a supervised classifier trained exclusively on labeled source pixels mapped in this common latent subspace when predicting a target image. The transfer component analysis (TCA) method was used to reduce the differences between domains while being discriminant for image classification.

Several authors have recently applied transfer learning for damage detection in the context of SHM. Gardner et al. ¹⁶³ demonstrated three-domain adaptation techniques on several laboratory case studies, namely TCA, Joint Domain Adaptation, and Adaptation Regularization based Transfer Learning. The work provided new frameworks for approaching the problem of population-based SHM. However, no real-world data was used.

The transfer learning capability would also mean that data from an experimental structure in a laboratory environment or numerical data from a finite element (FE) model could be used to label data for any bridge, providing significant benefits in applying SHM to most of the bridges. In particular, the use of FE models to generate source label data is of particular interest, as they are traditionally used for structural performance and damage identification.

Some authors have already tested the robustness of domain adaptation using data from FE models. ¹⁶³ The domain adaptation should work in situations where FE models have not been calibrated or contain deviations but capture the changes in the features due to damage. Even though we have already witnessed some laboratory structures applications, few publications have been made on real-world bridge data.

We have already trialled a domain adaptation approach in which a classifier has access to labeled training data (source) from numerical models and unlabeled monitoring test data (target), where both domains are from similar but statistically different distributions. The domain adaptation is performed over a well-known transfer knowledge approach, namely TCA. A numerical model is deemed to fail in detecting damage in real situations, as its data are generated under different conditions from those observed in real ones, mainly controlled by parameters that may not share the same underlying features. This type of mapping is expected to improve the classification accuracy for the target domain compared to a strategy that does not implement domain adaptation as a result of the reducing distributions mismatch between source and target domains. The domain adaptation approach’s effectiveness has been shown by evaluating the classification performance of unlabeled target data with and without domain adaptation, using real-world data sets from the Z24 Bridge in Switzerland.

Hybrid approach based on finite element models

FE models can be used as proxies for extreme operational and environmental conditions as well as for damage simulation to both unsupervised and supervised learning strategies.

The combination of finite element modeling and machine learning has been tackled by several researchers. In Ref. 168, the authors use FE models to establish probabilistic distributions of the physical parameters of bridge structures and to construct baseline stochastic FE models, for two ranges of temperatures. In Ref. 169, data from undamaged and damaged FE models are used to train an artificial neural network to diagnose damage in a fictitious railway bridge. However, no experimental data was used for either training or testing.

In Ref. 156, the authors propose an integrated hybrid framework to detect and identify the type of several damage scenarios. Numerical data from FE models are used for supervised learning and experimental data are used for testing. Parent FE models are generated for the baseline condition and selected damage scenarios, then offspring models are generated from the probability distributions related to sensitive parameters using Monte-Carlo simulation. The damage-sensitive features are based on the first principal component of a moving principal component analysis. A binary hypothesis test based on a Gaussian distribution is used to first decide whether a damage has occurred and then use successive comparisons between pairs of hypotheses to identify the type of damage. The framework is tested and validated on experimental data sets from a scaled-down four-span bridge in a laboratory environment, obtained for the same scenarios simulated in the FE models. It is shown that practical and efficient data-driven methods can be used to interpret data using the proposed hybrid approach. However, the laboratory structure is relatively small and the damage is particularly simple, as it is mainly induced by taking screws out.

A further development of the hybrid approach to SHM was reported by Figueiredo et al., ³² whereby FE models from a real bridge, that had been monitored for nearly one year, were used to generate data corresponding to scenarios that rarely occurred during the monitoring period. The results showed that FE models can be used to predict undamaged and damaged state scenarios of the bridge not observed by the monitoring systems. The undamaged data generated by the FE models were used in parallel with the monitoring data to improve the learning process of the machine learning algorithms, resulting in an improvement of their classification performance. However, the study remained mainly at a proof-of-concept level, as the numerical data was obtained strictly deterministically (and thus marred by considerable uncertainty), and the number of material parameters was very small, meaning that the amount of data obtained using the FE models was much inferior to that obtained from the monitoring campaign. Therefore, the study did not address three fundamental issues:

(i) Can FE data be used along with limited set of monitoring data if long-term monitoring data was unavailable, without compromising the performance of the damage detection algorithm?

(ii) How is the performance of the damage detection algorithm impacted by the number of observations, their density in the feature space and the choice of statistical parameters used in the training?

(iii) How to generate FE data in a systematic, reliable way?

To answer those questions, Bud et al. ⁴¹ assessed the performance of machine learning for damage detection based on a Gaussian mixture model trained only with numerical data from a FE model, which does not need to be precise, as the probabilistic variation of uncertain parameters is considered. The number of uncertain parameters far exceeds that considered in Ref. 32 and the FE model is shifted from deterministic to probabilistic to allow a greater coverage of their combinations. The Z24 Bridge in Switzerland is used as benchmark. This study shows the reliability of hybrid approaches for bridge SHM.

Seventekidis et al. ¹⁷⁰ used data derived from an optimal FE model to train a deep learning CNN classifier, which is experimentally validated on a benchmark structure, based on a supervised learning strategy. The structure is composed of a cantilever steel beam; the clamped edge is equipped with an electrodynamic shaker which serves as the base excitation, while the other end is free to oscillate. Artificial damage is introduced on the beam edge by first mounting a mass via a magnet and in a second instance inducing a cut on the beam in order to simulate a crack. In order to check the robustness of the proposed method, the damage magnitudes imposed on the benchmark are kept small and combined with random excitations. Acceleration time series are used as inputs to the network. The trained CNNs are validated on a real benchmark set-up, by experimental measurements in binary (undamaged and damaged conditions) and multiclass (undamaged and two damaged conditions) damage identification problems. The results show that (i) the multiclass damage identification is more demanding than the binary classification and (ii) the classification performance was linked with the calibration of the FE model.

Future trends

There are four major trends in line with digital technologies and resilient infrastructure. The more holistic pattern recognition paradigm and digital twin development are driven by the will to integrate technology to facilitate the interpretation and the reduction of human intervention in condition assessment. On the other hand, it seems that quite apart from extreme man-made or natural hazards, our transportation infrastructure (e.g., bridges) is not resilient under man-made or natural loads, and we need to leverage technology to better understand and respond to societal risks due to a lack of resiliency. ¹⁷¹ Finally, standardization arises as a normal step after the respective technological innovations and developments; that is, standards are seen as a “democratization” of SHM.

Holistic pattern recognition paradigm

In terms of future development, a broader approach should be considered—holistic pattern recognition paradigm—which takes into account physical modeling, structural monitoring, and information from visual inspections. In this data fusion approach, machine learning algorithms (especially, the ones based on deep learning) can learn from physics-, data- and visual-based sources, which potentially improves their knowledge from the structures to better identify damage at early stages.

Some authors have already started to tackle the paradigm. Arjomandi et al. ¹⁷² presented a hybrid SHM approach for condition assessment of cable-stayed bridges. Information from the structure’s quasi-static response, global vibration behavior, visual inspection, traffic data, global positioning system, and local vibration test of cables are incorporated into the assessment process. The framework was applied to a long-span case-study bridge in New Brunswick, Canada. Condition of primary structural members including stay-cables, bridge girders, and the orthotropic steel deck was successfully identified based on FE model updating. However, no machine learning strategy was used in this case.

In the forthcoming years, with the growing interest in video-based technologies for SHM (to support the digital representations of bridges and to extract full-scale damage-sensitive features), the digital twin development may combine mathematical models describing the physics of the bridge’s operation, with multivariate sources of data collected and processed from real assets during real world operations.

Digital twin development

Recent advances in digitization and big data have accelerated the development and application of digital twins. A digital replica of any structure is currently denoted its digital twin that allows for assessing the structural behavior and, eventually, its health condition over time.

The concept of digital twin was first proposed by Michael Grieves at the University of Michigan in 2003. In 2010, the National Aeronautics and Space Administration (NASA) introduced the concept of digital twin in the space technology roadmap, intending to use it to implement comprehensive diagnosis and prediction functions for flight systems to ensure continuous and safe operation during their service life. ¹⁷³

Digital twins have become the new futuristic obsession of the construction sector. ¹⁷⁴ This concept has now come into the spotlight as a technology that is even more forward-looking than building information modeling (BIM). It is seen as an opportunity to enhance asset management, help authorities anticipate adverse impacts during the life cycle stages, forecast maintenance activities based on sensor data (and other sources of data), and optimize costs of the entire infrastructure.

In 2019, Ye et al. ¹⁷⁵ presented an overview of the necessary capabilities required to develop a digital twin of bridges for SHM purposes. A FE model was developed to help establish a performance baseline and a data-driven approach was developed to perform short-term prediction and to detect anomalies in the data. SHM datasets were incorporated into a dynamic BIM environment for visualizing real-time sensor data and associated bridge behavior. The authors concluded that integrating large and heterogenous data resources (e.g., various types of SHM data) and integrating different analytical approaches (e.g., physics-based and data-driven approaches) are key to the successful development of a digital twins of bridges. For future work for successful digital twin, the authors proposed integrating monitoring data, BIM, FE modeling, and statistical modeling to improve SHM and thus to improve whole-life management of the bridge structures.

Ye et al. ¹⁶ show the importance of integrating different systems and technologies to process and share diverse and huge data about a bridge in the life cycle. It is not enough for us to judge the structural condition of bridges only based on periodic visual inspections (say every two or five years) or other forms of heuristic knowledge. On the other hand, no SHM system can replace those visual inspections, even if it has the best artificial intelligent algorithms.

Digital twins of bridges are not just about the 3D geometry model, the model must contain descriptions of all relevant bridge components. The information collected and processed over the years can then be attached to these components. The model then becomes a knowledge base about the bridge. The digital twin must also be related to the structural modeling, for instance, through a FE model.

In this respect, use of UAVs will be fundamental for developing a detailed 3D model of a structure, as has been elaborated upon by several researchers. Relevant recommendations for automated UAV-based inspections, image acquisition, 3D model reconstruction, and the subsequent image analysis algorithms for decision support have been discussed in Ref. 176. For instance, Khaloo et al. ¹⁷⁷ performed a UAV-based inspection on a bridge in Alaska via the generation of a dense 3D point cloud by using captured images and a hierarchical dense structure-from-motion algorithm.

Pushed by the fourth Industrial Revolution, a digital twin is nowadays a means to link between the physical and digital world. Digital twins have the potential to facilitate pattern recognition in a more holistic perspective. If the condition of a bridge changes several times during its service life, the prediction model (i.e., digital twin) can be constantly modified and updated with monitoring data, for instance, to reflect the actual conditions of the structure. ¹¹³

Resilience, sustainability, and climate change

The challenge to improve infrastructure resilience and sustainability has led to major infrastructure research initiatives that are relevant to the case of bridges. In the United States, the Federal Highway Administration (FHWA) created the Long-Term Bridge Performance Program, while in the United Kingdom, the Engineering and Physical Sciences Research Council (EPSRC) promotes two themes of resilient infrastructure and monitoring and field investigation of existing infrastructure.

Why resilience and sustainability? SHM has a major role in both. In the former, it reduces loss of capacity in post-trauma downtime; in the latter, it informs on future design with low embodied energy as part of the objective function. Both are major incentives for SHM.

The use of SHM for resilient and sustainable infrastructure management has already been investigated by several authors ¹⁷⁸ to aid infrastructure managers when prescribing maintenance and repair schemes as well as to provide quantitative and real-time evaluation of structural damage after a disaster that would support emergency managers in resource allocation.

Moreover, studies about the influence of operational and environmental conditions must go beyond daily traffic load, temperature, and humidity influences. We are experiencing climate change which may have multidimensional adverse impacts on the operational performance, longevity, and safety of existing bridges, which are particularly vulnerable to climate variables such as higher temperatures, increased evaporation, and increase precipitation.

Generally, changes in average climate are not noticeable, but it is the extremes of climate (floods, storms, and heat waves) which provide the largest impacts on our bridges. Model projections indicate much increased frequency and intensity of such events as the 21^st century progresses. For instance, in Figure 14, a schematic diagram shows the effects on extreme temperatures when both the mean and the variance increase, leading to much more record hot weather. An outstanding example is the heat wave in central Europe in 2003. ¹⁷⁹ A comprehensive review of the potential risks of climate change on bridges was made by Nasr et al. ¹⁸⁰ Therefore, if most of the bridge failures take place during or after an extreme natural event and if climate change is triggering more extreme events, are our bridges more vulnerable to failures in the near future?

Figure 14.

Changes in climate extremes. ¹⁷⁹

Climate change scenarios have to be incorporated into the SHM process, with the aim to predict the health state of existence bridges under unknown future environmental conditions. ¹⁸¹ We need to develop a systematic framework for the probabilistic assessment of the impact of climate change on the structural health of existing bridges, by coupling climate risk evaluation and health monitoring techniques; that is, we need to integrate existing procedures for the assessment of climate threats into the SHM procedures of bridges leading to a probabilistic SHM framework.

Standardization

Designing and implementing an effective SHM system is a process that must be carried out following a logical sequence of steps, in order to be technically efficient and to address the owner and authorities needs for management decisions. Standards and codes are an important milestone to set a basic SHM platform and to increase the transfer of professional knowledge and techniques to practical applications. They can also provide practical information for professionals and researchers involving the SHM techniques. ¹⁸²

One should note that codes are prescriptive, with weight of law (sometimes criminal) and standards provide information, for example, on terminology and interpretation. There are also guidance documents that are likely to be industry or interest-group derived. It is worth noting that normally a standard takes a previous guideline (or recommendation) and makes it a requirement for legal operation.

In the field of SHM for civil engineering structures, only a few technical guidelines exist. They were developed and published by national or international scientific or technical organizations. The first guideline for SHM, with focus on bridges, was probably released in 2001 by ISIS Canada—Intelligent Sensing for Innovative Structures. ¹⁸³

A guideline proposal especially for bridge monitoring was developed in 2003 to the US Federal Highway Administration Research and Development. ¹⁸⁴ The guide was designed to serve as a starting point for reaching consensus standards to help the bridge engineering community, especially bridge owners to make informed decisions about the value of various technologies such as instrumentation, monitoring, load testing, and field-calibrated analytical modeling.

Another comprehensive guideline was developed by the European thematic network SAMCO (Structural Assessment, Monitoring and Control) in 2006. ¹⁸⁵ The activities of the network were mostly related to bridges, buildings, power plants, and industries under seismic and other environmental loads.

In the United Kingdom, the CIRIA C788 (Structural Health Monitoring in Civil Engineering) guide released in 2020 shows how SHM can become an integral part of civil infrastructure asset management and describes the role of different stakeholders in making the necessary SHM design and management processes effective. The guide aims to provide a summary of “what to think of” and “how to think about it” with sufficient technical detail and help to support business cases to adopt SHM. ¹⁸⁶

Up to now, no standard is published or under development in this field by an international standardization organization. ¹⁸⁷ Nevertheless, in 2014 China was one of the first countries in the world to implement a national code for SHM. ¹⁸² The “Technical Code for Monitoring of Building and Bridge Structures (GB50982-2014)” is an exclusive national SHM code with mandatory requirements in China. The main content includes basic requirements, monitoring methods for environmental temperature, humidity and structural responses, and specifications for various types of structures (i.e., high-rise buildings and structures, long-span spatial structures, bridge structures, seismically isolated structures, and crossing construction) involving sensor arrangement, monitoring parameter selections, and condition assessments for construction and post-construction as well as technical requirements. This code provides the primary framework of structural monitoring. The outstanding characteristics of this code is that detailed monitoring requirements for various structures during both construction and post-construction stages are put forward, and parameters that need to be measured during construction and post-construction stages are clarified. ¹⁸⁸ Additionally, the ASTM International released in 2019 a standard guide for application of acoustic emission for SHM. ¹⁸⁹

Therefore, it is recognized that the lack of guidelines and minimum standards are impeding bridge owners to recognize and take full advantage of the available experimental and analytical technologies. Worldwide, more recommendations and drafts codes for the implementation of SHM systems are appearing, which is certainly an important step towards a mature SHM industry. ⁴³ Although the SHM standards from these institutions might not be enforced by their governments, the standards still give regulatory instructions for the implementation of SHM, which is supportive toward SHM code development. ¹⁹⁰ In Europe, there are developments to transform a set of technical recommendations into a code.

Standardization may be the trigger we need to move SHM from research to practice.

Summary and conclusions

Past, present, and future of structural health monitoring of bridges

Table 2 summarizes the chronology of the condition assessment of bridges. SHM has moved from being an on-site tool for condition assessment in the advent of catastrophic bridge failures to becoming a remote bridge assessment tool in parallel with the BMSs, and it is beginning to appear as an integrated tool to support the concept of infrastructure asset management, where risk management, financial efficiency, and sustainability are the focus of bridge owners and authorities.

Table 2.

Past, present, and future of SHM of bridges.

	Where has it come from?	Where is it?	Where is it going?
Big picture	Some sort of SHM was used to assess the structural condition when there was suspicions of damage in the bridge in a discrete manner in time.	It is now implemented in some one-off situations or as research projects. Although countries like China are installing very large monitoring systems, nobody has been able or prepared to show a positive rate of return on investment for bridge applications	To permanent, online, and autonomous damage identification procedure.
Specific topics	-Construction boom after the World War II	-Implementation of SHM systems to continuously identify damage	-Integration of FE models and artificial intelligence to identify damage
	-Occurrence of bridge collapses in developed societies (1960s) due to the lack of maintenance	-Sensing technology: fiber-optic systems and wireless sensors	-Sensing technology: video- and laser-based technologies (no contact sensors)
	-Implementation of BMS (in 1980s and 1990s)	-Complex FE models to predict structural responses	-Digital twin (and augmented and virtual realities) to support the bridge inspections, BMS, and SHM
	-Static field tests and temporarily monitoring of special bridges	-Big Data era: artificial intelligence techniques (e.g., machine learning algorithms) to identify damage	-Infrastructure asset management for a holistic approach
	-Sensing technology: electrical wired systems

Summary of the statistical pattern recognition paradigm

This section summarizes the main evolutions observed in each stage of the SHM paradigm, in order to contribute to the optimization of each stage of the SPR paradigm and to the development of more efficient bridge SHM.

In terms of operational evaluation, we have moved from a global damage identification strategy to more localized applications in order to increase the rate of return on investment.

However, the SHM community remains ambitious to identify damage based on full-field measurements in a global approach. NDE has shown potential for detecting hidden damage but the scale of large structures presents a significant challenge to implement such local inspection methods. ⁹⁰ As SHM technologies and potential applications arise, developers may not possess the knowledge about “value of information” and the required skill set to estimate the benefit of their technology ahead of the actual application. Therefore, the benefits of using SHM systems must be identified, and the information obtained must be understood and effectively used for decision support.

In terms of data acquisition, we have observed developments moving from wire, fiber-optic, and wireless to non contacting sensors such as cameras and vibrometers.

To address the limitations of current sensor systems for field applications, the research community has been actively exploring new technologies that can advance the state-of-the-practice in SHM. Electrical wired systems are still the most robust sensing technology for long-term SHM and while wireless sensor networks have emerged as a powerful low-cost platform for connecting large networks of sensors, they may suffer unexpected constraints. Their applicability has been largely constrained to temporary monitoring as the power consumption is still the major limitation of wireless sensors operating on batteries. Fiber-optic systems are still limited by the cost, particularly of the loggers. They have advantages for static measurements but may have limitations in terms sensitivity if used as accelerometers. Nonetheless, distributed fiber-optic sensing offers a capability to monitor long lengths through a single cable, allowing full structure coverage over several kilometers, with the potential to simplify installation, and overcoming the issue of optimal sensor location. In the last decade, vibrometers have shown capabilities, but are still too expensive for mainstream; on the other hand, vision-based techniques have been pushed by advances in computers and digital imaging technology, which have allowed 2D and 3D DIC methods to measure full-field deformation of vibrating structures, providing very high spatial resolution measurements where every pixel effectively becomes a measurement point on the structure. Even though the sensitivity is still limited for real-world bridge monitoring, it stands as an opportunity for the near future particularly when coupled with artificial intelligence technology. As with increasing capabilities in consumer-grade systems, professional systems feeding demand for motion capture in other domains (e.g., sports) as well as smartphones with sensing and signal capabilities may propagate to bridge applications.

In terms of feature extraction and generation, even though new techniques have been proposed to generate damage-sensitive features for long-term SHM, damage identification is still mainly focused on the modal properties. FE models have been proposed to generate structural responses not observed by the monitoring systems.

Data fusion and hybrid techniques, where measurements from different sensors and sources are integrated with finite element modeling data, have also emerged. In particular, besides the extraction of features directly from the response data, FE models are being used to generate new data sets from undamaged and damaged structural conditions.

In terms of statistical modeling for feature classification, machine learning algorithms have had a key role: we observed the evolution from neural networks as black boxes to deep learning capable to learn from scratch. The complexity and the mystery of deep learning lie in the amount of data it processes to get the fascinating results. In terms of training strategies, we are evolving from unsupervised to supervised learning strategies tackled by using numerical models.

In the big data era we face currently, machine learning algorithms have had the capacity to simplify massive and complex monitoring data into simple damage indices and graphical representations. They have an important role to deal with past measured data, resembling the human brain. Depending on the formulation, some of these algorithms are able to filter linear and nonlinear patterns and therefore enhance the characterization of structural parameters and behaviors. Machine learning algorithms have been mainly used in an unsupervised learning strategy. As long as these algorithms are fed with monitoring data collected during all the operational scenarios, they should be able to detect anomalies or patterns unseen during normal operation, without the need to measure the operational and environmental variations such as temperature and humidity. Experience has shown that there is not yet a machine learning algorithm capable to be appropriate for every type of data, but deep learning has shown signs of great capability as long as there are enough computer resources. Supervised learning strategies have been hampered by the lack of experimental damaged data from bridges, which have led to the advent of the FE models for data generation in undamaged and damaged conditions.

Main limitations and challenges of the bridge structural health monitoring

Sensing technology and statistical methods that run, successfully, on mechanical structures in a certain deterministic environment might not be applicable on civil structures at all. Based on the promises given by the rotating machinery, bridge SHM has been the focus of research for several decades. During that time, it has seen a number of firsts and advances that have kept it a thriving research field.

However, for the transition of SHM from research to practice as a widely used tool in bridges, there are a couple of grand challenges to overcome:

1. The new technology and trends have unveiled new perspectives to observe the response data, but we still strive to obtain reliable information on the presence of damage!

In any real-world bridge, the separation of changes in damage-sensitive features caused by damage from those caused by changing operational and environmental conditions is still one of the biggest challenges to transition the SHM technology from research to practice. We must develop better SHM solutions at the data acquisition stage, which includes the development of cost effective and robust sensors as well as assessing these sensor technologies in real world settings of civil engineering. Signal-to-noise ratios need to be boosted because small damage produces changes that are often below the noise from environmental effects. ¹⁹¹

2. As a multidisciplinary field, SHM needs T-shape civil engineers!

Students are turned on by designing and analyzing new structures, not preventing old ones from failing. If only they knew how interesting and multidisciplinary the topic can be! More, there are not many researchers and engineers who combine experience in experiment design and information technology know-how along with a deep understanding of bridge structural engineering. ¹⁶ In Canada, Mufti et al. ¹⁹² have promoted the concept of “civionics” in order to integrate civil engineering and electrophotonics, in the development of SHM, to provide engineers with the means and the knowledge to build “smart” structures containing the necessary equipment to provide much needed information related to the health of a bridge before things go wrong. Therefore, we need to change the curricula in order to combine deep specialist knowledge (the down bar of the T) with a broad understanding of other disciplines (the crossbar) and how they combine for greatest effect.

3. Curricula emphasize sustainability but should also do resilience!

Sustainability is a prescribed component of United Kingdom engineering taught programs, ¹⁹³ but resilience is not. The former addresses the role of engineers in reducing carbon footprint and achieving net zero while the latter addresses management of ongoing and future impact of climate change which, through extreme weather, generate major hazards affecting infrastructure resilience. We believe concepts of SHM should infuse civil engineering curricula since it offers the capability to manage risk and reduce immediate and ongoing impact of extreme loads on bridges whose post-traumatic condition is best established by SHM.

4. Identifying applications where monitoring can have an impact!

It is fundamental to change the current mindset, ¹⁹⁴ by performing a risk analysis beforehand in order to produce a ranking of damage risks. Some risks should be retained for long-term SHM, others may be addressed by regular bridge inspection, and the remaining will be dropped because of low impact and/or probability. Scouring has been shown to be one of the most common causes of bridge failure; therefore, any SHM system of bridges over rivers should start from sensing the foundations.

5. In the hierarchical structure of damage identification, most research has been driven by incipient vibration-based damage detection methods!

As a subset of SHM, vibration-based monitoring persistently has an unhelpful association with historic failed “damage detection” exercises. ¹⁹⁵ We must move from simplistic “cracked beam” views to real problems in bridges. What does actual damage look like and what is the best way to spot it? Several times it looks like we start with a spanner and try to find a nut that fits!

Besides, efforts still must be made in order to address other levels of damage identification, namely damage diagnosis (localization, type, and severity) and damage prognosis. However, several limitations can be pointed out: How to produce reliable numerical data? How to reliably model damage? How to best balance monitoring and numerical data? Also, there is the need to find damage-sensitive features for damage severity assessment (e.g., mode shapes in the damaged area and mode shape curvatures).

6. Unless there are public domain exemplars showing a positive rate of return on investment, bridge SHM will continue at best to be one-off demonstration!

It is very hard to show the return on investment unless we can provide the owner an accurate definition of the probability of damage detection, and it is hard to provide it unless we can define the damage to be detected and the probability of future loading that may produce that damage. This also goes to secrecy and liability issues; there are confidential reporting mechanisms mimicking civil aviation practice that should be used and mined in order to support the implementation of the SHM technology.