Abstract
This paper presents a novel hybrid structural health monitoring (SHM) framework for damage detection in bridges using numerical and experimental data. The framework is based on the hybrid SHM approach and combines the use of a calibrated numerical finite element (FE) model to generate data from different structural state conditions under varying environmental conditions with a machine learning algorithm in a supervised learning approach. An extensive experimental benchmark study is performed to obtain data from a local and global sensor setup on a real bridge under different structural state conditions, where structural damage is imposed based on a comprehensive investigation of common types of steel bridge damage reported in the literature. The experimental data are subsequently tested on the machine learning model. It is demonstrated that relevant structural damage can be established based on the hybrid SHM framework by separately evaluating different cases considering natural frequencies, mode shapes, and mode shape derivatives. Consequently, the work presented in this study represents a significant contribution toward establishing SHM systems that can be applied to existing steel bridges. The proposed framework is applicable to any bridge structure in which relevant structural damage can be simulated and experimental data obtained.
Keywords
Introduction
The aim of structural health monitoring (SHM) systems is to identify damage at an early stage to avoid the failure of structural components or systems. For bridges, SHM systems can be used to increase life-safety benefits through continuous monitoring and economic benefits through improved inspection efficiency and minimized unplanned downtime. The many bridges in need of lifetime extension comprise a major concern worldwide, while demands on operational conditions such as traffic loads, speed, and intensity are increasing. For steel and composite bridges, the most common damage types are related to fatigue occurring in or below the bridge deck. 1 Furthermore, variability in the operational and environmental conditions affects the structural response and can mask changes caused by damage. 2 With the large number of existing steel and composite bridges in the infrastructure, there is a need for SHM systems that can detect both local and global structural damage while taking the variability in the operational and environmental conditions into consideration. To further develop the possibilities of employing SHM systems for such bridges, research should be focused on tests of real structures in their operating and environmental conditions rather than numerical or laboratory studies of representative structures.2–5
SHM, defined as the process of implementing an automated and online strategy for damage detection in structures, 6 has traditionally been performed using two main approaches: model-based and data-based.7,8 In the model-based approach, a numerical finite element (FE) model is established, and model updating is performed in two stages. In the first stage, an initial numerical model is updated based on data from the undamaged condition to obtain a validated numerical model or reference model. In the second stage, the reference model is updated based on data from the damaged condition. Damage detection can then be performed based on the change in the updating parameters. Although according to the hierarchical structure of damage states,9,10 the model-based approach allows for level I (existence), level II (location), and level IV (extent) damage detection, there are several disadvantages with this approach. First, a numerical model must be developed, parameterized, and validated. Parameterizing the numerical model can be challenging, particularly for the selection of the parameters associated with damage. Second, there are inherent uncertainties in the application of numerical models for damage detection due to modeling inaccuracies, including model simplifications and uncertainties in the structural properties. Third, variability in operational and environmental conditions affecting the structural response may mask the changes caused by damage, inducing uncertainty in the model updating process. Last, the model updating process depends on the measured outputs including, but not limited to, the modal properties. The sensitivities of the damage detection results to the number of measured outputs and the accuracy and weighting of these outputs in the model updating are limitations. Applications of the model-based approach have been reported by considering relevant structural damage in concrete bridges.11–15 Furthermore, Behmanesh and Moaveni successfully applied the model-based approach using simulated damage in a composite steel-concrete footbridge. 16 From these applications, large damage resulting in a significant global effect on the structures has been considered. However, due to the many disadvantages and limited possibilities for detecting local structural damage, the model-based approach is considered impractical in the context of SHM for large and complex structures such as bridges.
In the data-based approach, however, a statistical model based only on experimental data is established using machine learning algorithms. Damage detection can be performed by analyzing the distribution of damage-sensitive features using unsupervised or supervised learning, referred to as statistical model development. 17 In the context of SHM, unsupervised learning refers to the situation where data are available only from the undamaged condition of the structure, while supervised learning refers to the situation where data are available from both undamaged and damaged conditions. Although there are no requirements for developing and validating a numerical model in the data-based approach, there are two main challenges that remain, both of which are related to the lack of data. First, unsupervised learning is often required. This challenge is evidenced by the lack of data from the damaged conditions, which are rarely available for bridges. Second, data normalization, referred to as the process of separating changes caused by operational and environmental conditions from changes caused by structural damage, 18 must be considered. This challenge is evidenced either by the lack of baseline data from the undamaged condition, which requires response measurement data where all operational and environmental variations are included, or by the lack of direct measurement data from the parameters related to the operational and environmental conditions. There are several applications of the data-based approach that use numerical models or test structures that are relevant to bridges and where variability in the operational and environmental conditions are considered.19–22 Moreover, applications have been applied to the Z24 prestressed concrete bridge by considering relevant structural damage23–25; however, these are currently the only examples of applications to real bridges. From the applications of the data-based approach, it is evident that unsupervised learning algorithms are primarily applied, allowing for level I (existence) and, to some extent, level II (location) damage detection. Furthermore, the performance of unsupervised learning algorithms strongly depends on the amount and variability of the data. These observations can be viewed as limitations of the data-based approach. Consequently, to further develop the field of SHM for application to bridges, a framework that can overcome these limitations is needed.
The hybrid SHM approach, which takes principles from the model-based and data-based approaches into consideration, can be used to overcome such limitations. There are different ways to utilize this approach; however, a common understanding is that numerical model(s), experimental data, and machine learning are integrated in some way. There are few studies in the literature that have considered this approach. Malekzadeh et al. 26 proposed a hybrid approach using supervised learning, but the experimental study was carried out using a bridge structure in the laboratory, where variability in the operational and environmental conditions was not included. On the other hand, Figueiredo et al. 27 presented a hybrid approach for damage detection in bridges by considering the variability in the operational and environmental conditions. This approach consisted of using a numerical model to enrich the monitoring data from the undamaged condition to improve damage detection using an unsupervised learning algorithm. This study was based on data from the Z24 bridge. In general, most practical applications of the SHM approaches are based on concrete bridges, particularly on the Z24 benchmark study. Consequently, in the literature, there is a need for studies in which SHM approaches are tested experimentally on steel bridges.
This paper presents a novel hybrid SHM framework for damage detection in bridges. The framework is based on the hybrid SHM approach and combines the use of a numerical model to generate data from different structural state conditions under varying environmental conditions with machine learning algorithms. As such, the numerical model is used as a proxy for the experimental structure to generate training data on which statistical model development can be performed; the test data acquired experimentally can subsequently be used to diagnose the structure. An extensive experimental benchmark study is carried out on a full-scale steel bridge where relevant structural damage is implemented. Supervised learning is performed, allowing for level I (existence), level II (location), and level III (type) damage detection. The feasibility of the framework is discussed with respect to its potential for detecting local and global structural damage and its applicability to similar bridges that are currently in service.
The hybrid structural health monitoring framework
Operational and environmental variability
Variability in the operational and environmental conditions is one of the main challenges in deploying an SHM system for bridges in operation. For bridges, the operational conditions mainly include live loads that are typically caused by traffic, while environmental conditions include wind loading, temperature effects, and humidity.
The most influential source of variability in the modal properties is due to temperature effects. Several studies have reported the variation in the modal properties of bridges caused by temperature. In general, variations within 5–10% of the fundamental natural frequencies can be expected on a daily and seasonal basis.5,28–31 However, several studies in the literature have reported larger variations in the natural frequencies.32–36 The bridges experiencing large variations in the natural frequencies include certain concrete and composite steel-concrete highway and railway bridges located in cold climates. A step-like variation in the natural frequencies can be obtained for such bridges during a yearly season, particularly around or below the freezing point. Such changes in the natural frequencies are explained by a contribution to the global stiffness of the structure, generally generated by the asphalt layer in highway bridges and the ballast in certain types of railway bridges.
Commonly, changes in the lower modes of the structure are reported, typically the first 2–6 modes, and the general observation is that the natural frequencies decrease with increasing temperatures. With the variability in operational and environmental conditions presenting a challenge for deploying SHM systems for bridges, this effect and the subsequent influence on damage detection must be addressed in the hybrid SHM framework.
The hybrid structural health monitoring framework
The proposed hybrid SHM framework consists of several steps that are conducted with the aim of determining whether damage is present in the structure. The framework concept, shown in Figure 1 simulates all possible outcomes for the relevant damage states of the structure based on a calibrated numerical FE model, including simulations of the variations in the operational and environmental conditions, and uses the outcome of the simulations as an input to obtain a machine learning model. Experimental data are subsequently tested on the machine learning model to provide decision support or structural diagnosis of the structure, that is, to determine whether damage is present in the structure. Herein, a damage state includes a relevant damage type at a specific location in the structure. Flowchart of the hybrid SHM framework. The dashed lines indicate a possible connection.
The hybrid SHM framework assumes the existence of an initial period for structural monitoring. The purpose of the structural monitoring step is twofold. First, structural monitoring provides data as an input for model updating to obtain a calibrated FE model. Such a calibrated FE model that accurately represents the structure is needed to produce meaningful data. Second, structural monitoring provides data to the machine learning model for subsequent structural diagnosis. Such data can be obtained from a global or a local sensor system, or both, on the structure. It is acknowledged that data enrichment, although not considered in this study, can be applied to improve the machine learning model. As such, the presented framework can be used as a dynamic learning process if the machine learning model is updated continuously with undamaged data, that is, if the structural diagnosis results in true negative (TN) indications of damage. The presented framework is applicable to new and existing bridge structures.
In this study, the hybrid SHM framework is demonstrated through damage detection of a bridge. As such, the remainder of the paper is organized into three parts. In the first part, represented by the left side of the hybrid SHM framework flowchart, an extensive experimental benchmark study performed on a full-scale steel bridge to obtain acceleration time series from a global and local sensor setup is presented. Measurements are obtained before and after relevant structural damage is imposed on the bridge. Operational modal analysis (OMA) is performed as a part of the feature extraction process to obtain the modal parameters of the bridge. The data obtained from the experimental study provide samples for the test set in the machine learning model. The second part, represented by the right side of the hybrid SHM framework flowchart, describes the process for generating training data and obtaining the machine learning model. A numerical FE model is first established, and model updating is performed to obtain a calibrated FE model. Different damage types are imposed on all the relevant locations in the FE model, and numerical simulations that take variations in the environmental conditions into consideration are carried out to obtain data from the damage states. Importantly, numerical simulations of the baseline, or undamaged condition, are performed. Furthermore, the feature extraction process and the machine learning algorithm applied are described. In the final part, the results from the structural diagnosis are presented, and an evaluation of the hybrid SHM framework is provided.
Experimental study
The Hell Bridge Test Arena benchmark study
The Hell Bridge Test Arena, shown in Figure 2, is a steel riveted truss bridge formerly in operation as a train bridge. The bridge is used as a full-scale damage detection test structure.37,38 Figure 3 shows an overview of the bridge structural system, which is composed of the two bridge walls, bridge deck, and lateral bracing. The bridge was used for an extensive experimental benchmark study carried out in 2020, where it was damaged in a number of damage scenarios while structural monitoring was performed. Table 1 summarizes the different structural state conditions, including the environmental conditions experienced during the benchmark study period. The Hell Bridge Test Arena. Overview of the Hell Bridge Test Arena: (a) bridge structural system, sensor layout and location of the modal vibration shaker (MVS) and (b) damage states introduced to the bridge deck and lateral bracing. Overview and descriptions of the structural state conditions, including the environmental conditions, during the experimental benchmark study period. aLocal: damage to the secondary steel structure. Global: damage to the primary steel structure.

Experimental setup
Figure 3(a) shows an overview of the sensor setup used in the experimental benchmark study. An instrumentation system from National Instruments consisting of multiple cRIO-9036 controllers was used to acquire data from accelerometers in a local and global sensor setup involving sensor groups 1 and 2, respectively. Synchronous sampling was ensured by global positioning system (GPS) timing of the individual controllers. Sensor group 1 (SG1) consisted of 40 single-axis accelerometers (Dytran 3055D3) for measuring the local response of the bridge deck in the vertical direction (global z-direction), whereas sensor group 2 (SG2) consisted of 18 triaxial accelerometers (Dytran 3583BT and 3233A) for measuring the global response of the bridge in both the lateral and vertical directions (global y- and z-directions). Data were sampled at 400 Hz. The data were detrended, low-pass filtered, and resampled to 100 Hz prior to analysis. The bridge was excited in the vertical direction using a modal vibration shaker (APS 420) located at approximately one-third of the bridge span to excite as many structural vibration modes as possible. The modal vibration shaker was operated in the sine sweep mode by applying a band-limited sinusoidal load in the range of 2–55 Hz with a logarithmic sweep speed of 1 Oct/min.
Evaluation of damage states based on Ref. 1.
Damage was introduced by replacing existing rivets with bolts. Moreover, the damage types were imposed by temporarily removing the bolts. Each damage state comprised highly progressed damage, representing large cracks or loose connections that open and close under dynamic loading, typically caused by traffic or large environmental loads. Consequently, the damage was considered by removing all bolts in each state condition. This damage progression leads to a redistribution of forces that would be demanding on the structure over time; however, it is not considered critical to the instant structural integrity due to the structural redundancy of the bridge. One test was performed for each state condition. For each test, time-series data were generated from 75 sensor channels, that is, 40 channels from sensor group 1 and 35 channels from sensor group 2. Note that one channel in the lateral direction was excluded from sensor group 2.
Of the four damage types considered in the experimental study, damage types 1 and 4, that is, the stringer-to-floor-beam connections and the connections between the floor beams and main load-carrying members, are of major importance with respect to traffic loading and structural integrity. Furthermore, damage types 2 and 3, that is the stringer cross beams and lateral bracing connections, ensure the stability of the bridge against lateral and torsional loads, mainly during environmental loading, and are of minor importance with respect to traffic loading and immediate structural integrity. Therefore, damage types 1 and 4, shown in Figure 4, are considered in this study. Details of the damage types, including the underlying mechanisms, can be found in Ref. 1. The damage types imposed on the bridge before and after damage: (a) damage type 1 and (b) damage type 4.
Operational and environmental variabilities impose difficulties on the damage detection process. The environmental conditions for the different structural state conditions are summarized in Table 1. Considerable variability in the weather occurred during the experimental study, with moderate variabilities in the temperature and wind speed range. These variabilities, which are highly representative of the climate where the bridge is located, can affect the structural response and mask changes caused by damage. Furthermore, such variabilities complicate the damage detection process and challenge the machine learning model, which is established based on numerical simulations only. No sources of variability were considered for the operational condition, which was limited to the operation of the modal vibration shaker.
Operational modal analysis
Operational modal analysis was performed to obtain the modal properties, such as the natural frequencies and mode shapes, of the bridge in the different damage states. Apart from the general aim of establishing as many modes as possible, there were two main challenges in modal identification: first, distinguishing all relevant modes between the different damage states and second, correctly identifying closely spaced modes in frequency. To solve these challenges, three methods for output-only system identification were considered: frequency domain decomposition,39,40 data-driven stochastic subspace identification (data-SSI), 41 and covariance-driven stochastic subspace identification (cov-SSI). 42 The cov-SSI was found to be the most appropriate method, particularly for distinguishing the relevant modes between the different damage states.
Data from SG1 and SG2 (75 channels) were used in the analyses. The results from the cov-SSI analyses are highly dependent on the input parameters and the selection of such parameters, including the number of blockrows, order, and stabilization criteria. Satisfactory results were obtained by setting the number of blockrows equal to 200 and defining the range of order between 0–200 with 100 evenly spaced values and a maximum order of 200. Furthermore, for the selection of stable poles to include in the stabilization diagram, a frequency deviance of 1%, a damping deviance of 5%, and a modal assurance criterion (MAC) threshold of 95% were used. Additionally, a stability level of four was used. For further details of the stabilization criteria, refer to Kvåle et al.43,44 The cov-SSI parameters were set to be equal for all damage states.
Damage categorized as global is expected to have a larger influence on the structural response than damage categorized as local. The stabilization plots from the different damage states in the 0–25 Hz frequency range are presented in Figure 5. The stabilization plot changes with different damage states, which is a challenge in the system identification process. This is observed particularly for DS8, where new modes appear due to the global damage imposed on the structural system. Hence, a manual evaluation was performed to ensure that the correct modes were established. With reference to the undamaged state, more than 40 physical modes up to a frequency of approximately 40 Hz were identified from the cov-SSI analysis. From these modes, the six most relevant modes within the 0–25 Hz frequency range were chosen to be used in the hybrid SHM framework. Figure 6 shows the mode shapes along with the natural frequencies, damping ratios, and MAC values between the experimental and numerical mode shape vectors from the calibrated FE model for SG1 and SG2. The reasoning behind the choice of these modes is included in the Calibrated FE model section. The OMA can be considered as feature extraction to obtain the experimental test set, which is then input to the machine learning model. Stabilization plots for the cov-SSI analysis of the different damage states. Mode shapes used in the hybrid SHM framework obtained from the cov-SSI analysis in the undamaged state. The following abbreviations are used: H (horizontal), V (vertical) and T (torsional). Linear interpolation of the mode shapes is performed for illustration purposes.

Numerical model and simulations
Calibrated FE model
The purpose of model updating is to obtain a validated numerical FE model that is in good agreement with the measurements and a validated FE model being computationally efficient for numerical simulations. The initial FE model and the calibration of the FE model using an improved sensitivity-based finite element model updating procedure are described in a separate study by Svendsen et al.45,46 A description of the FE model is included below for convenience.
The FE model is established using the FE software ABAQUS with site inspections and specifications from technical drawings as the basis. The primary structure of the bridge is included in the model, that is, the vertical walls, the bridge deck, and the lateral bracing. The secondary structure is included in the modeling of the bridge deck to obtain an accurate structural representation for damage detection purposes. Other secondary structure and nonstructural items are represented by lumped point masses on the bridge deck to obtain a proper mass distribution. A beam element model representation is used, and the bridge is modeled as simply supported on one end (global translational x-, y-, and z-direction constrained) and rolled boundary conditions on the opposite end (global translational x-direction partly constrained by spring elements and y- and z-direction constrained). As such, the FE model is obtained with reduced complexity. Figure 7 shows the FE model of the bridge. The calibrated FE model is referred to as the reference model. FE model.
Comparison of the modal properties between the experimental results and the calibrated FE model.
Modeling of damage
The damage types are each represented using no connectivity between the elements in the numerical model at the specific damage locations. There are two main advantages of this damage representation: first, the complexity of the numerical model is minimized by avoiding spring elements or similar complex linear or nonlinear damage modeling; and second, this is an accurate and realistic representation of loose structural connections. A comparison of the results obtained from the numerical and experimental analyses of the damage states is performed to visualize the effect of introducing damage in the structure and verify the introduction of damage in the numerical model. Figure 8 shows the percentage change in the natural frequencies for all damage states from the undamaged state, as evaluated for all modes. Furthermore, Figures 9 and 10 show the MAC values between the mode shape vectors of the undamaged state and the damage states for the local and global sensor setups. The percentage change of the natural frequencies for all damage states based on the undamaged state (UDS) for all modes, considering the experimental and numerical analyses. Modal assurance criterion values between the mode shape vectors of the undamaged state (UDS) and the damage states in the experimental study. Modal assurance criterion values between the mode shape vectors of the undamaged state (UDS) and the damage states in the numerical study.


Figures 8–10 provide information on how the imposed damage changes the natural frequencies and mode shapes of the bridge compared to the baseline condition (undamaged state). It is observed that increasing the damage severity of DT1 (DS1 and DS2) induces increasing changes in both the natural frequencies and MAC values for the experimental and numerical analyses. Furthermore, the local damage, represented by DT1, generally influences the changes in the natural frequencies and MAC values less than the global damage, represented by DT4 (DS8). Consequently, it is concluded that (1) there is consistency in the results for the different damage states upon considering the experimental and numerical analyses separately and (2) there is adequate correspondence in the change of the natural frequencies and MAC values between the experimental and numerical analyses. Furthermore, an important observation is that the most significant changes in the natural frequencies are generally within a range that can be expected due to variations in operational and environmental conditions.
Numerical simulations
Simulation information.
aSimulated variation in the environmental conditions.
In the datasets, the variation in each damage type (class label) is based on (1) the number of simulated damage locations and (2) the simulated change in the environmental conditions. For each damage type, damage is successively introduced in all possible locations in the bridge. The stringer-to-floor-beam connections (DT1), which are introduced with damage in both single and multiple connections to represent increasing severity, are denoted as DT1 (1) and DT1 (2), respectively. Altogether, 1744 simulations are performed for each dataset. For each simulation, an eigenvalue analysis is performed, and the natural frequencies along with the normalized mode shape values for each mode from the local (SG1) and global (SG2) sensor setups are extracted. A total of 250 modes are extracted for each simulation. To ensure the correct identification of modes for each simulation, a mode match index is used
Environmental variation in the numerical simulations
Variability in the environmental conditions must be included to obtain realistic training data as an input to the machine learning model. Considering environmental conditions is particularly important for the baseline condition. The most influential source of variability in the modal properties is due to temperature effects. In general, changes in temperature affect the global material properties and system characteristics and have a considerable influence on the natural frequencies of a bridge structure.
Variation in the environmental conditions is simulated by varying the steel density parameter
Minimum and maximum changes in natural frequencies from variation from the environmental conditions in the baseline condition.
Feature extraction and machine learning
Feature extraction
In the context of SHM, machine learning is applied to associate the damage-sensitive features with the state of the structure to distinguish between the undamaged and damaged states. In this study, modal properties such as the natural frequencies and mode shapes, including mode shape derivatives, are used as damage-sensitive features in the machine learning model. The underlying assumption associated with the use of modal properties as damage-sensitive features is that the imposed damage will somehow change the stiffness, mass, or energy dissipation characteristics of the structure.
Natural frequencies provide a high-level assessment of the structural dynamic characteristics, mainly allowing for level I (existence) damage detection. Mode shapes, including the mode shape derivatives, provide spatial information about the structural dynamic characteristics and allow for level I (existence) and level II (location) damage detection. Mode shape information is particularly useful for structures with regular geometry, such as bridges, that are equipped with an appropriate number of sensors.
The natural frequencies and mode shapes, including mode shape derivatives, are considered useful as damage-sensitive features in the hybrid SHM framework for three main reasons: first, the natural frequencies and mode shapes are straightforward to establish both experimentally and numerically; second, for a numerical model, the natural frequencies and mode shapes are obtained by solving the eigenvalue problem, which is computationally efficient and consequently suitable for a large number of simulations; and third, the possibility of level I (existence), level II (location), and level III (type) damage detection is enabled when combined with supervised learning.
Mode shape derivatives
The mode shape derivatives can be used to enable a high sensitivity to localized damage. The mode shape curvature
The following clarifications should be noted. First, the mode shape curvature, which is based on a beam formulation, is applied to the local sensor setup (SG1). The setup is divided into four parts in the longitudinal direction of the bridge to establish the full formulations for all sensors. The formulation for each part is applied separately and subsequently concatenated. Second, the FE model is used to represent the undamaged state of the bridge for the experimental (test) and numerical (training) data. This representation is equivalent to assuming that the SHM system can be installed at any time during the bridge lifetime. This approach is beneficial for existing bridges; however, it requires a precise FE model. Furthermore, since the undamaged state is represented in the calculations of the mode shape curvature, this method is used here only for predicting the damage states in the structural diagnosis of the structure. Consequently, in the structural diagnosis of the structure, the prediction of the undamaged state is excluded.
Machine learning
In supervised training, labeled data are available. Supervised learning is made possible through the numerical generation of training data with the relevant damage types included. Supervised learning allows for level I, II, and III damage detection and is considered for group classification only. A training matrix
In this study, the support vector machine (SVM) algorithm is applied.
48
The SVM algorithm is computationally efficient and capable of performing linear and nonlinear classification, making it highly suitable for SHM applications. The training data are divided into 80% for training and 20% for testing. A grid search with 5-fold cross-validation is performed on the training data to find the optimal hyperparameters and increase the ability of the SVM algorithm to generalize to unseen data. The regularization parameter C, the kernel functions, and the corresponding kernel coefficients
Results
General
The hybrid SHM framework for damage detection is evaluated using three different cases. The cases are used to consider natural frequencies, mode shapes, and mode shape derivatives separately as damage-sensitive features. All sensor channels from each sensor group are taken into consideration. Furthermore, a total of 1744 numerical and four experimental observations are included. Hence, the training matrix
In the cases considering natural frequencies and mode shapes, the damage detection performances are evaluated in terms of level I and III damage detection. For the level I damage detection performance, the evaluation is performed in terms of Type I and Type II errors, which are referred to as false positive (FP) and false negative (FN) indications of damage, respectively. For the level III damage detection performance, the evaluation is performed in terms of the correct damage predictions for the different damage types. The damage detection performances are also evaluated in terms of level II damage detection in the case considering mode shape derivatives.
Additionally, the machine learning performances are included in the evaluation of the results. Machine learning performance can be a possible source of uncertainty in the results and is evaluated by means of the cross-validation score (CV score), the area under the curve (AUC), and the fraction of correct predictions (score). The AUC value refers to the area under the receiver operating characteristics (ROC) curve, which represents the relative trade-offs between true positives (TPs), or the probability of detection, and FPs, or the probability of false alarms. 49
Case 1: Natural frequencies
In the case considering natural frequencies, damage detection is investigated by considering datasets 1 and 2 with different ranges of environmental variability in the undamaged state (baseline). Figures 11 and 12 show the results from the prediction of the experimental test set and the corresponding performances of the machine learning algorithm for datasets 1 and 2, respectively. The true label is denoted on the x-axis of the result matrices. The correct predictions are highlighted in the figures, whereas the Type I and Type II errors, including the total number of errors, are summarized in Table 6. Prediction results including the performances of the machine learning algorithm for dataset 1 in the case considering natural frequencies. Prediction results including the performances of the machine learning algorithm for dataset 2 in the case considering natural frequencies. The UDS (±1%) is excluded due to the variation of ±2% for the different damage types in dataset 2. Damage detection results obtained by considering Type I and Type II errors for different ranges of environmental variability in the undamaged state (baseline) of datasets 1 and 2.

Three general observations are made from the results presented in Figures 11 and 12 and Table 6. First, the total number of Type I and Type II errors decreases with decreasing variability in the environmental conditions. Hence, it becomes more difficult to perform level I and level III damage detection with increasing variability in the environmental conditions. Second, it is observed from the results of dataset 1 that all the damage types are established. DT1 was detected with the lowest variability in the environmental conditions (±1%), DT4 was detected with a slightly higher variability (±2%), and UDS was detected for the remaining variability in the environmental conditions. Third, the machine learning performances are generally considered acceptable. However, the performances are lower for dataset 2 than dataset 1. As such, it becomes more difficult for the machine learning algorithm to make correct predictions when large changes in environmental conditions affect the damage states. However, the generally high machine learning performances indicate that the different damage states can be separated under variability in the environmental conditions. Overall, the results clearly show that changes in the environmental conditions mask damage, resulting in Type II errors. Consequently, using natural frequencies as damage-sensitive features in this study results in low damage detection capabilities for high variability in the environmental conditions.
Case 2: Mode shapes
To investigate the damage detection capabilities in the case considering mode shapes, analyses are performed separately for each mode. Figures 13 and 14 show the results from the prediction of the experimental test set and the corresponding performances of the machine learning algorithm for the local (SG1) and global (SG2) sensor setups of datasets 1 and 2, respectively. Here, the damage prediction results obtained per mode are arranged horizontally, whereas those obtained per sample in the test set are arranged vertically. The Type I and Type II errors are summarized in Table 7, and the damage predictions for the different damage types are summarized in Table 8. Prediction results including the performances of the machine learning algorithm for the local (SG1) and global (SG2) sensor setups with dataset 1 in the case considering mode shapes. Prediction results including the performances of the machine learning algorithm for the local (SG1) and global (SG2) sensor setups with dataset 2 in the case considering mode shapes. Damage detection results obtained by considering Type I and Type II errors for datasets 1 and 2. Correct damage prediction results for the different damage types in datasets 1 and 2.

From the results obtained for Case 2, two important observations are made. First, all the damage types can be predicted. In general, although Type I errors are reported, there are few Type II errors that indicate strong level I damage detection capabilities. Furthermore, local damage represented by DT1 is best identified by the higher modes, whereas global damage represented by DT4 is best identified by the lower modes. Second, the local sensor setup (SG1) performs better than the global sensor setup (SG2). SG1 obtains better prediction results than SG2 in terms of the total number of correct damage type predictions and the ability to detect different damage types (including the undamaged state). Furthermore, by comparing the results obtained for the different datasets, it is observed that SG1 is not affected by a change in the environmental conditions of the damage types. Conversely, SG2 obtains lower damage prediction results with increasing variability in the environmental conditions of the damage types. It should also be noted that the machine learning performances are generally better for the higher modes (modes 4–6) than the lower modes (modes 1–3), which might be a possible explanation for the generally low damage detection capability of the lower modes. Overall, from the results obtained, using mode shapes as damage-sensitive features provides strong damage detection capabilities, with the best performance obtained using the local sensor setup (SG1).
Case 3: Mode shape derivatives
Based on the results obtained for Case 2, the damage detection capabilities are investigated by considering only the local sensor setup (SG1) in the case considering mode shape curvatures. Therefore, level I damage detection is assumed. Figures 15 and 16 show the prediction of the experimental test set and the corresponding performances of the machine learning algorithm with datasets 1 and 2, respectively. Similar results are presented in Figures 17 and 18 for three different configurations of summed absolute mode shape curvature difference. Since the undamaged state is represented in the mode shape curvature calculations, the prediction of the undamaged state is excluded. Consequently, the results presented consider only the damage types with imposed damage on the structure. Prediction results including the performances of the machine learning algorithm for each mode in dataset 1 in the case considering mode shape curvatures. Prediction results including the performances of the machine learning algorithm for each mode in dataset 2 in the case considering mode shape curvatures. Prediction results including the performances of the machine learning algorithm for the configurations of summed modes in dataset 1 in the case considering mode shape curvatures. Prediction results including the performances of the machine learning algorithm for the configurations of summed modes in dataset 2 in the case considering mode shape curvatures.



From the results obtained in Figures 15 and 16, it is observed that the damage prediction is strong. DT1 and DT4 are predicted well, with particular prediction improvements for the lower modes. Furthermore, modes 1 (H1), 4 (V2), and 5 (T2) predict all damage types correctly when considering both datasets. It is also observed that the mode shape curvatures are little affected by the difference in the datasets, and the machine learning performances are nearly optimal for all modes. The configurations of the summed absolute mode shape curvature difference provide an effective means of evaluating several modes together. From the results obtained in Figures 17 and 18, it is clear that all damage types are correctly predicted in the vertical configuration when considering both datasets, whereas only DT4 is correctly predicted in the other configurations. Consequently, when the damage is not severe, as with DT1, inaccurate predictions are made if modes that are not significantly affected by damage are included in the calculations of the mode shape curvatures. It should also be noted that the machine learning performances are optimal.
To evaluate the performance in terms of level II damage detection, the experimental and numerical inputs to the machine learning model for the summed absolute mode shape curvature difference are compared in Figure 19 considering the vertical and horizontal configurations. The inputs are normalized between 0 and 1. The relevant peaks of the curvature related to the damage types are labeled with the corresponding sensor names. The experimental and numerical input conform for DT1 and DT4 in the vertical configuration and for DT4 in the horizontal configuration. These results correspond well with the results presented in Figures 17 and 18. Furthermore, and most importantly, the sensors representing the peaks are located nearest to the damaged areas of the bridge deck. These sensors are also shown in Figure 20. Consequently, the damage types are successfully localized. Comparison of the numerical and experimental input to the machine learning model considering the summed absolute mode shape curvature difference for the vertical and horizontal configurations. Sensor overview for the level II damage detection. The sensors located nearest to the damaged areas that are indicating damage are highlighted in red.

Two additional important observations are made. First, the experimental results are jagged and unclear compared to the numerical results. This observation is explained by the differentiation process, which amplifies high-frequency noise. Normalization is performed after the differentiation process, resulting in higher mean values of the experimental curvature than the numerical curvature. The experimental results are noisy, and the damage is less distinct from these results. On the other hand, the numerical results are less noisy and more distinct where the damage is represented for both configurations, clearly illustrating the difficulties in working with experimental data. Second, the machine learning model is able to recognize DT1 (1) in the vertical configuration despite the noise in the data from the experimental study. This observation clearly demonstrates the superior abilities of the machine learning model in distinguishing damage.
Discussion
From the results obtained for the different cases investigated in the hybrid SHM framework, the following general observations are made: 1. Relevant structural damage in steel bridges can be found under a simulated variation in the environmental conditions. All the damage types considered, including the undamaged state (baseline), can be established in this way. Moreover, level I (existence), level II (location), and level III (type) damage detection can be performed by separately evaluating different cases of natural frequencies, mode shapes, and mode shape derivatives. 2. The case considering natural frequencies generally provides Type II errors or false negative indications of damage, whereas the case considering mode shapes generally provides Type I errors or false positive indications of damage. Furthermore, using natural frequencies provides low damage detection capabilities for high variability in environmental conditions. Consequently, using mode shapes is more reliable for detecting damage. Moreover, the cases considering mode shapes and mode shape derivatives increase the damage detection capabilities. 3. The local sensor setup (SG1) performed better than the global setup (SG2) with respect to the damage detection of both local and global damage types. Using a local sensor setup that consists of a grid of sensors covering the part of the bridge where damage is most commonly experienced, that is, the bridge deck, allows for increased damage detection capabilities. As such, the local sensor setup should be used for damage detection purposes, whereas the global sensor setup should be used for system identification and model updating purposes. The results clearly demonstrate the effect that a dense sensor setup has on the increased damage detection capabilities, particularly for performing level II damage detection. Consistent level II damage detection covering all possible damage locations of the imposed damage types cannot be performed without a systematic, and preferably dense, sensor network. A sparse sensor network, however, is not considered a limitation for applying the hybrid SHM framework. 4. Datasets 1 and 2 consider low and high variations in the environmental conditions of the damage types compared to the baseline, respectively. The datasets thus represent how long damage is assumed to persist in the system: dataset 1 assumes that damage has been present for a short time period, while dataset 2 assumes that damage has been present for a long time period. From the cases considering mode shapes and mode shape derivatives, the local sensor setup (SG1) was not affected by the difference in the datasets. However, from the case of mode shapes, the global sensor setup (SG2) obtained lower damage detection capabilities with dataset 2.
The deviations between the numerical and experimental results, resulting in reduced classification results, are caused by uncertainties. There are several uncertainties related to the different parts of the hybrid SHM process. In particular, such uncertainties are related to (1) the representation of the FE model and numerical simulations, (2) the estimation of modal parameters using OMA and the corresponding system identification method(s), (3) the performance of the machine learning algorithm, and (4) the statistical representation of the experimental data. Furthermore, for the representation of the FE model and numerical simulations, the uncertainties are related to simplifications in the numerical modeling, the model updating procedure, the modeling of damage, and the representation of environmental variability in the numerical simulations. As such, this part of the hybrid SHM process is essential. Reducing the identified uncertainties in the different parts of the hybrid SHM process can further enhance the damage detection process and enable improved classification results.
The advantage of using natural frequencies as damage-sensitive features is that they are straightforward to establish both numerically and experimentally. Furthermore, few sensors are required. However, natural frequencies are sensitive to changes in the operational and environmental conditions and insensitive to local structural damage (particularly associated with the global modes of the structure). Consequently, the low sensitivity of the frequency shifts to damage requires either a very precise numerical model, precise measurements, or significant levels of damage for them to be an effective damage indicator. These requirements were not fulfilled in this study. However, there are several advantages of using mode shapes and mode shape derivatives as damage-sensitive features. Mode shapes and mode shape derivatives are less affected by variation in the environmental conditions than natural frequencies, and they can perform level I, II, and III damage detection. Although more effort and more sensors are needed to establish mode shapes and mode shape derivatives than natural frequencies, they provide increased damage detection capabilities, which are also demonstrated in this study.
From the hybrid SHM framework presented, experimental data can be applied by considering the OMA from ambient vibration or vibration from regular loading conditions. Traffic-induced vibration is not necessary but can be beneficial with respect to operational variability. Using OMA, or output-only system identification, requires only response measurements; this is clearly beneficial for large structures such as bridges, structures in service, and structures with an SHM system that typically provide data based on undamaged conditions.
Conclusion
This paper presented a novel hybrid SHM framework for damage detection in bridges. The framework is based on the hybrid SHM approach and combines the use of a numerical model to generate data from different structural state conditions under varying environmental conditions with machine learning algorithms to obtain a machine learning model. Based on the machine learning model, experimental test data can be applied to provide decision support or diagnose the structure in a supervised learning approach. The supervised learning approach allows for level I, II, and III damage detection.
The hybrid SHM framework was demonstrated through damage detection of a full-scale steel bridge. A machine learning model was established by simulating all possible outcomes of the relevant damage states for the structure using a calibrated FE model, including simulations of environmental conditions. An extensive experimental benchmark study of the bridge was performed to obtain data from a local and global sensor setup under different structural state conditions. The data obtained from the experimental study provided samples to the test set in the machine learning model. Three different cases of damage-sensitive features were evaluated. Based on the evaluation of the different cases, different levels of damage detection were performed. All the damage types considered were established. As such, by considering the uncertainties related to the hybrid SHM framework, damage detection and structural diagnosis were successfully performed. In conclusion, the hybrid SHM framework is demonstrated to work.
The main limitation of the hybrid SHM framework is the need for a precise numerical model for successful classification. There is a trade-off between a calibrated numerical model in good agreement with measurements and a calibrated numerical model being computationally efficient for numerical simulations. Nevertheless, the hybrid SHM framework presented in this paper demonstrates that damage detection can be successfully performed using a numerical model with reduced complexity. Relevant structural damage can be found in steel bridges. Moreover, the framework is applicable to any bridge structure in which relevant structural damage can be simulated and experimental data obtained. Further investigation should be conducted to (1) reduce the identified uncertainties in the hybrid SHM framework for sharper classification, (2) incorporate data enrichment into machine learning, and (3) implement more damage types and increase the statistical representation of the experimental data.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Hell Bridge Test Arena is financially supported by Bane NOR and the Norwegian Railway Directorate.
