Deep neural network for damage detection in Infante Dom Henrique bridge using multi-sensor data

Abstract

This paper proposes a data-driven approach to detect damage using monitoring data from the Infante Dom Henrique bridge in Porto. The main contribution of this work lies in exploiting the combination of raw measurements from local (inclinations and stresses) and global (eigenfrequencies) variables in a full-scale structural health monitoring application. We exhaustively analyze and compare the advantages and drawbacks of employing each variable type and explore the potential of combining them. An autoencoder-based deep neural network is employed to properly reconstruct measurements under healthy conditions of the structure, which are influenced by environmental and operational variability. The damage-sensitive feature for outlier detection is the reconstruction error that measures the discrepancy between current and estimated measurements. Three autoencoder architectures are designed according to the input: local variables, global variables, and their combination. To test the performance of the methodology in detecting the presence of damage, we employ a finite element model to calculate the relative change in the structural response induced by damage at four locations. These relative variations between the healthy and damaged responses are employed to affect the experimental testing data, thus producing realistic time-domain damaged measurements. We analyze the receiver operating characteristic curves and investigate the latent feature representation of the data provided by the autoencoder in the presence of damage. Results reveal the existence of synergies between the different variable types, producing almost perfect classifiers throughout the performed tests when combining the two available data sources. When damage occurs far from the instrumented sections, the area under the curve in the combined approach increases $50\%$ compared to using local variables only. The classificatoin metrics also demonstrate the enhancement of combining both sources of data in the damage detection task, reaching close to $97\%$ precision values for the four considered test damage scenarios. Finally, we also investigate the capability of local variables to localize the damage, demonstrating the potential of including these variables in the damage detection task.

Introduction

Structural health monitoring (SHM) is gaining importance for bridge managers and researchers.^1,2 In the last decades, many bridge structures have been instrumented to monitor their response during operation.^3–10 Instrumentation systems are often deployed in structures to track specific processes (e.g., construction steps and concrete hardening phenomena). Sometimes, these devices are removed from the structure after completing the target task and turn unprofitable. In other cases, the instrumentation systems remain installed but are inactive or disregarded. Under such scenarios, different sensor types coexist in the same structure, offering redundant and complementary information on the structural condition. From a value of information perspective, a continuous SHM strategy based on measurements obtained from these devices efficiently exploits their assessment potential throughout adequate processing.

One of the most critical tasks in SHM is damage detection.^1,11 Civil engineering structures are subject to several deterioration processes, including material degradation, aging, and extreme loading conditions. ¹² Damage detection is an inverse problem that indicates the structural health condition from measurements.^13,14 Two main approaches exist in damage detection: model-based and data-driven. ¹² Model-based approaches employ computational models to account for the physics governing structural behavior. ¹³ Model-based SHM is extensively employed given its ability to reach high damage identification levels according to Rytter scale, ¹⁵ allowing to locate and quantify the damage. The Rytter scale classifies SHM algorithms based on the reached level of damage characterization, where four levels exist: (I) detection, (II) location, (III) quantification, and (IV) prognosis.^15,16 These techniques require strong physical knowledge and modeling skills.^14,17 Besides, the iterative procedure required to solve the inverse problem of damage identification involves running thousands of simulations with the corresponding computational expense. This situation prevents model-based approaches from yielding a fast assessment close to real-time once new data are measured.^14,18

By contrast, data-driven techniques assess the health condition of structures based uniquely on monitoring data.^11,12 These approaches skip the need for complex models based on engineering knowledge and the associated computational cost to solve the identification problem. However, monitoring data regarding possible damage conditions is rarely measurable in real practice, and only healthy data are available. When only healthy measurements are employed, the SHM approach consists of characterizing the baseline condition (healthy) and detecting departures from this reference, which corresponds to level I in the Rytter scale. ¹⁵ The core idea of data-driven methods is thus to characterize the reference (undamaged) behavior of the structure and detect an outlier when a novelty (i.e., damage) occurs.^11,19

One of the most challenging issues in data-driven methods is tackling the effect of changing environmental and operational conditions (EOCs).^14,20,21 These phenomena (e.g., temperature, humidity, traffic) may mask the presence of damage and hinder the ability of the algorithms.^22,23 Numerous statistical methods have been developed to address this problem, such as autoregressive models ¹⁷ or Principal component analysis (PCA).^23–26 PCA is a statistical data compression tool widely employed in outlier detection for SHM.^7,27–29 This method allows isolating environmental and operational variability by projecting the data onto the most relevant principal components (eigenvectors) of the covariance matrix. ³⁰ However, PCA is mainly restricted to linearly correlated data spaces, which limits its applicability.

The emergence of artificial intelligence introduces new techniques to handle large amounts of data and account for complex nonlinear relationships.^31,32 Machine learning has progressively extended to many application areas, including the identification of structural damage.^33–36 Neural networks (NNs), and particularly deep neural networks (DNNs) have demonstrated to be powerful techniques in SHM applications.^32,37–46 convolutional neural networks (CNNs) have demonstrated powerful performance as image processing techniques to detect external damage. ³² For example, in Cha et al., ⁴⁷ authors employ a CNN that automatically learns image features to identify cracks in concrete. It incorporates a sliding window technique to accommodate different image sizes. They design the CNN to classify images and detect the presence of cracks robust to image noise sources. Cha et al. ⁴⁸ propose an enhanced approach to identify different damage types (cracks, corrosion, and delamination) by applying a faster region-based CNN. In contrast to traditional statistical methods, NNs require a training phase to learn relationships from large amounts of data. ³¹

In the line of damage detection accounting for varying EOCs, many DNN-based works have been proposed. ³² One of the most powerful methods in this task is the autoencoder. ⁴⁹ Autoencoders are a specific neural network type based on the same idea of data compression as PCA but include nonlinearities. ⁵⁰ An autoencoder reconstructs the input data by following two steps: encoding and decoding. ⁵¹ The encoding step compresses the data onto a lower-dimensional latent representation (dimensionality reduction), and the decoding step reconstructs the original measurements from the lower-dimensional features. ⁵² During training, the autoencoder learns to reconstruct data from the healthy condition of the structure. The healthy condition consists of long-term monitoring data that includes environmental and operational variability. ⁴⁰ By learning the existing relationships in the long-term monitoring data that is affected by EOCs, we ensure that this variability is considered to be “normal,” thus establishing a reliable description of the reference behavior.^8,38 The reconstruction error measures the discrepancy between a measurement and its reconstruction. When new measurements are obtained that belong to a similar (undamaged) condition, we expect that the autoencoder will adequately reconstruct them with a small error value. We assume that the damage-induced variability affects the data differently than environmental and operational phenomena. ⁵³ Thus, we expect that the autoencoder yields a high reconstruction error with the appearance of damage that sufficiently affects the measurements.^8,38 Several works exist in the literature employing reconstruction error in SHM applications.^{8,36,37,49,54} For example, Bao et al. ⁵⁵ propose a two-step damage detection approach that transforms raw data signals into grey-scale images and trains a DNN based on stacked autoencoders. The recent work ⁵⁶ employs autoencoders to produce two reconstruction error metrics indicative of damage. Article ⁵⁷ proposes an adversarial autoencoder trained with vehicle acceleration responses and uses the reconstruction error as the damage indicator. In Lee et al., ⁵⁸ authors employ a convolutional autoencoder for damage detection on a PSC-I bridge under vehicle-induced excitation using acceleration signals. In Amarbayasgalan et al., ⁵⁴ authors propose a deep autoencoder for an efficient dimensionality reduction and then apply a density-based clustering for outlier detection. In this work, we employ an autoencoder DNN to reconstruct monitoring data subject to EOCs and employ the reconstruction error as the damage-sensitive indicator.

Another critical aspect of data-driven approaches is to identify the variables to measure. Vibration-based are the most extensively employed data.^{9,17,46,53,59} In this context, raw vibration measurements (e.g., acceleration signals) are often processed through operational modal analysis (OMA) ⁶⁰ techniques to produce the modal properties of the structure, mainly eigenfrequencies and eigenmodes. ²³ These are strongly sensitive to the presence of damage^23,61 Avci et al. ¹⁷ reviewed the extensive literature on different data-driven approaches to detect structural damage using vibration monitoring data. For example, in Bakhary et al., ⁶² authors employ eigenfrequencies and eigenmodes as the input features to estimate the location and severity of damage by applying a two-stage NN incorporating system substructuring. They validate the method in small-scale concrete slab and steel-frame structures. Particularly on bridge applications, Xu et al. ⁶³ propose a two-step approach employing a modal energy-based damage indicator to locate damage and estimate its location with a NN. In Jayasundara et al., ⁶⁴ authors employ modified modal strain energy and modal flexibility versions as the damage-sensitive features to feed a four-layer NN that estimates damage location and severity. Azim and Gül ⁶⁵ employ two parallel approaches to detect and locate damage in the beams and the deck of a bridge. They employ the modal strain energy as the feature to find a damage indicator for the steel beams and the relative flexibility change for the reinforced concrete deck. Nick et al. ⁶⁶ also employ a damage-sensitive index based on modal strain energy calculated from the first three bending eigenmodes to detect and estimate the location of damage in steel girder bridges. They apply a NN to quantify the damage that is trained with synthetic data from a computational model. In Wickramasinghe et al., ⁶⁷ authors present a damage detection technique relying on the modal flexibility to design two damage indicators (vertical and lateral), which allow locating damage at critical elements of a suspension bridge. Another relevant work is, ⁶⁸ where authors compare three modal-based SHM strategies (strain energy, flexibility, and curvature) in noisy data.

While post-processing measurements to estimate eigenfrequencies is a well-established task (see, e.g.,^69–72), the approximation of eigenmodes (a.k.a., mode shapes) presents additional difficulties. Only a few OMA techniques address this challenge,^69,73 yet the accuracy of higher-order modes can be deficient. For this reason, some vibration-based approaches employ only eigenfrequencies as damage-sensitive features. For example, Comanducci et al. ²⁶ investigate the ability of various data-driven algorithms (dynamic regression, linear and local PCA, and cointegration) using time histories of eigenfrequencies measured in the Infante Dom Henrique bridge. Magalhães et al. ⁶ also employed Infante Dom Henrique as the target structure to investigate the potential of eigenfrequencies in damage detection. However, eigenfrequencies are global variables, and their ability to provide information regarding damage location is limited. ¹⁸ This implies they reach only a level I (detection) diagnostic in the Rytter scale. ¹⁵

Recent developments in both sensing technology and computational resources have aroused a growing interest in the exploitation of raw time-domain signals, for example, displacements, rotations, or strains.^74,75 Compared to the value of modal coordinates, which are often unit-scaled, local variables own some physical meaning, and their value might be directly insightful about the structural condition. These local variables have demonstrated detection capabilities, mainly when damage occurs close to their emplacement. For example, Teng et al. ⁴⁴ applied one-dimensional CNNs to raw short-term acceleration signals. They validated the method using a truss bridge and a frame structure in a laboratory environment. Authors’ previous work ⁷ presents a case of applying data-driven methods to long-term time histories measured in a full-scale operating structure. Azim and Gül ⁷⁶ applied PCA to strain measurements to detect damage in truss railway bridges. They validated the method using a finite element model to simulate stiffness reductions, achieving successful detection, location, and relative damage quantification.

However, local variables lose sensitivity when the damage lies far from their location. This restricts the detectable damage to the monitored region, and very dense sensor arrays are required to cover the whole structural system. In this sense, including global variables (eigenfrequencies) seems helpful to solve the detection problem. After extensive review, the authors have yet to find any other work in the literature that addresses and analyzes the combination of global and local variables in SHM practices. Only the work presented by Alten et al. ⁷⁷ was found to study the detection ability of eigenfrequencies, strains, and rotations separately in a post-tensioned reinforced concrete bridge. In Soman et al., ⁷⁸ two local sources of data are fused, namely accelerometers and strain gauges, for damage detection in long-span bridge structures. However, no global variables (i.e., eigenfrequencies) are considered to complement the lack of sensibility of local variables when damage occurs far from their specific emplacement. Combining experimental data sources (i.e., local and global) in full-scale bridges still needs to be explored in the existing literature.

This work presents a strategy to adequately handle multivariate data from different instrumentation systems coexisting in a bridge, including local and global variables. We employ a data-driven approach based on a particular autoencoder architecture to reconstruct long-term monitoring measurements for novelty detection (level I assessment ¹⁵ ) in full-scale bridges. The autoencoder learns the underlying relationships in experimental measurements affected by EOCs to characterize the reference (healthy) condition. The novelty indicator is the reconstruction error. We assume that damage affects the meaurements differently from EOCs, thus resulting in higher values of the reconstruction error. We extensively analyze the advantages and limitations of dynamic global variables (eigenfrequencies) and quasi-static local variables (inclinations and stresses) for bridge condition assessment. The main contribution of this work lies in unveiling the synergies in data from different coexistent instrumentation systems, including local and global variables, to provide a more insightful assessment. We demonstrate that the combination extends the damage detection capabilities of local-only or global-only approaches. For that purpose, we analyze the receiver operating characteristic (ROC) curves and the data transformation into the latent space of reduced dimensions. We also explore the potential of local variables to enrich the diagnostic by providing location information as long as the damage occurs nearby any of the installed sensors.

We select the Infante Dom Henrique bridge as the target structure for the analysis. The bridge contains long-term monitoring data from accelerometers, strain gauges, and inclinometers. Acceleration signals are processed to obtain time series of the eigenfrequencies of the structure, which are the global variables. Stresses and rotations are directly used as the local variables. Information about the exact sensor location can be found in Refs. 79 and 80. We employ a deep autoencoder DNN as the damage detection method. Its architecture follows the scheme developed in our previous work, ⁸¹ where we proposed a partially explainable deep autoencoder to outperform PCA in the outlier detection task. This work also supports the robustness and flexibility of the methodology described in Fernandez-Navamuel et al. ⁸¹ by feeding the DNN with data from different sources and dimensions. Three DNN architectures are built to accommodate the different input data for the analysis: (i) local variables, (ii) global variables, and (iii) the combination of both variable types. The DNN allows to accommodate a wide range of input dimensions, being adaptable to more complex problems with denser instrumentation systems. The encoding dimension of each architecture is fixed based on the level of information captured during training. The resulting compressed features lack a physical meaning but are an optimal representation of the data in terms of dimensionality.

To test the performance of the proposed methodology, we explore the capability of detecting any damage with impact on the vertical bending stiffness of the structure. ⁶ This damage can appear due to long-term structure aging or extreme loading, such as earthquakes. Since real data regarding damage scenarios are unavailable, the detection ability of the method is tested by affecting the undamaged measurements synthetically. For that purpose, a finite element (FE) model of the bridge is employed to simulate damage at four different locations. The damage scenarios are generated by reducing the vertical bending stiffness of the affected FE model elements. From the FE model, we obtain the relative change between the healthy and damaged response for each damage scenario. We assume this shift to be sufficiently representative of the effect of real damage. We transfer the relative change induced by damage to some monitoring measurements preserved for testing, which incorporate the effect of EOCs. For each damage scenario, the testing dataset contains two subsets: undamaged and damaged. The ROC curves ⁸² evaluate the detection ability of each DNN and allow us to investigate the potential of combining local and global variables. The capability of the reconstruction error to locate damage is also analyzed by calculating the contribution of each instrumented section to the total error.

Monitoring data

Infante Dom Henrique bridge is a complex reinforced concrete structure that constitutes one of the main entrances into Porto (Portugal). It combines two elements: a pre-stressed box beam deck and a thin arch spanning 280 m between abutments. ⁸⁰ A static instrumentation system containing strain gauges, inclinometers, and thermometers was installed to control construction and was subsequently kept to assess the structural behavior during operation. ⁷⁹ This system records one measurement every hour. Besides, a complementary long-term dynamic monitoring system was installed in 2007 to obtain the first 12 eigenfrequencies of the structure. It measures acceleration signals from 12 force balance accelerometers installed inside the deck box girder. The system creates an ASCII file with 30-min time series sampled at 50 Hz every half an hour. A downsampling to 12.5 Hz removes the offset and reduces the data amount. This sampling frequency is enough to accurately obtain the desired eigenfrequencies, given that they are below 5 Hz. ⁶ The data are subsequently post-processed using DynaMo ⁸³ software to obtain estimates of the first 12 eigenfrequencies of the structure: $\mathbf{f}=\{f_1,f_2,...,f_{12}\}$ . A full description of the monitoring system (devices, acquisition process, and obtained data) can be found in Magalhães et al.^6,84

This work employs 3 years of continuous monitoring data acquired with these two instrumentation systems. The monitoring period extends from September 2007 to September 2010. Measurements from inclinometers and strain gauges are used as the local variables. Figure 1 depicts the location of the sensors, distributed in six instrumented sections. Sections one, two, five, and six contain two inclinometers each, but one of the sensors in section six (S61) was discarded due to low signal quality. Figure 2 shows the evolution of the seven selected inclination signals over the monitoring period. Sections three and four include four strain gauges, but one is discarded (S34) due to low signal quality. Figure 3 shows the evolution of the selected stress time series during the monitoring period. Each sensor is denoted as $S_{i,j}$ , with i indicating the instrumented section where it is located and j being the sensor number. More details regarding the exact location of the sensors can be found in Fonseca et al. ⁷⁹ The eigenfrequencies are produced with DynamMo software ⁸³ as the global variables. Figure 4 shows the evolution of the first 12 eigenfrequencies (Hz) of the structure. The figure reveals that relatively small changes affect the measurements in the long term (around 0.005 Hz variation), which mainly correspond to temperature variability. ⁶ The identification algorithm accurately identifies the eigenfrequency values throughout the entire monitoring period. Table 1 summarizes the statistical properties of the long-term eigenfrequency measurements, including the mean and the standard deviation (SD) values.

Figure 1.

Instrumented sections with local variables and damage locations.

Figure 2.

Evolution of the seven inclinometers over the monitoring period.

Figure 3.

Evolution of the seven strain gauges over the monitoring period.

Figure 4.

Evolution of the first 12 eigenfrequencies over the monitoring period. The squared window represents a subset of 1000 measurements of the first eigenfrequency to show the variability over time.

Table 1.

Statistical properties of the long-term eigenfrequency data.

Statistic	$f_1$	$f_2$	$f_3$	$f_4$	$f_5$	$f_6$	$f_7$	$f_8$	$f_9$	$f_{10}$	$f_{11}$	$f_{12}$
Mean	0.778	0.821	1.147	1.416	1.752	2.012	2.225	3.042	3.338	3.523	3.761	4.380
SD	0.002	0.004	0.003	0.004	0.005	0.008	0.006	0.011	0.009	0.010	0.001	0.015

Since the local variables yield one measurement per hour while eigenfrequencies are produced every half an hour, we compute the mean of every two eigenfrequency measurements to use data from both monitoring systems with a time resolution of 1 h. After removing null values, a total of 16,005 measurements is obtained for each variable. From these data, the first $72\%$ of the samples (obeying chronological order) are employed for training and $18\%$ for validation. The final monitoring period (remaining $10\%$ of the data) is kept to investigate the damage detection ability using different variable combinations.

DNN approach

We employ a data-driven method based on autoencoder DNNs. We intend to exploit the advantages of combining multiple sources of monitoring data in the task of damage detection. The architecture follows the scheme developed in our previous work. ⁸¹ It provides adequate results according to the results obtained in Fernandez-Navamuel et al. ⁸¹ However, there may exist other architectures that provide adequate results as well. Optimizing the architecture compared to other existing approaches is out of the scope of this work.

The architecture contains two adding blocks at the encoder and the decoder stages, namely a linear and a nonlinear one. Its main advantage lies in allowing to employ a simple linear architecture (analogous to PCA), which accelerates the training phase over deeper architectures. ⁷ Besides, this strategy enhances the explainability of the neural network, since the contribution of nonlinear transformations can be explicitly separated from the linear ones. ⁷ Figure 5 shows the block-wise architecture of the autoencoder.

Figure 5.

Block diagram of the autoencoder with parallel connections.

Let $\mathbf{M}\in {\mathbb{M}}^{n\times s}$ be a set of raw measurements from s variables. Since variables from different sources are used, a linear standard rescaling is applied. For any variable $\mathbf{x}$ , let $\mu$ and $\sigma$ be the mean and the SD of the variable, respectively. The rescaling function $\mathcal{R}$ is defined as:

$$\mathcal{R}(x)=\frac{\mathbf{x}-\mu }{\sigma }.$$

(1)

The rescaled measurements are obtained as ${\mathbf{M}}_{\mathrm{resc}}=\mathcal{R}(\mathbf{M})$ . For notation simplicity, we refer to the rescaled measurements as $\mathbf{M}$ . Let $\mathbf{m}=(m_1,m_2,...,m_s{)}^T$ be a sample from $\mathbf{M}$ at a given time. The encoder operator ${\mathcal{F}}_E$ compresses the data to a lower-dimensional feature or latent space ( $\mathbf{z}\in {\mathcal{R}}^k$ , with $k<s$ ) by adding the following two terms:

$$\mathbf{z}={\mathcal{F}}_E(\mathbf{m})={\mathcal{F}}_a(\mathbf{m};{\mathbf{\theta }}_a)+{\mathcal{F}}_b(\mathbf{m};{\mathbf{\theta }}_b).$$

(2)

Here, ${\mathcal{F}}_a$ is an affine transformation and ${\mathcal{F}}_b$ is a nonlinear operation, with ${\theta }_a$ and ${\theta }_b$ being their corresponding learnable parameters. Subsequently, the decoder operator $\left({\mathcal{F}}_D\right)$ follows a similar architecture to yield the reconstructed measurements $\hat{\mathbf{m}}\in {\mathcal{R}}^s$ :

$$\hat{\mathbf{m}}={\mathcal{F}}_D(\mathbf{z})={\mathcal{F}}_c(\mathbf{z};{\theta }_c)+{\mathcal{F}}_d(\mathbf{z};{\theta }_d),$$

(3)

where ${\mathcal{F}}_c$ is an affine transformation and ${\mathcal{F}}_d$ a nonlinear one. ${\mathbf{\theta }}_c$ and ${\mathbf{\theta }}_d$ gather the learnable parameters involved at each operation. The misfit between an original measurement and its reconstruction is computed using the squared $l_2$ norm:

$$\rho (\mathbf{m})=\parallel \mathbf{m}-({\mathcal{F}}_D°{\mathcal{F}}_E{)}_{\theta }(\mathbf{m}){\parallel }_2^2,$$

(4)

where $\mathbf{\theta }=\{{\mathbf{\theta }}_a,{\mathbf{\theta }}_b,{\mathbf{\theta }}_c,{\mathbf{\theta }}_d\}$ , and ° represents the composition operator.

The optimal parameter set ${\theta }^{*}$ results from minimizing a loss function based on the reconstruction error for the training dataset $\mathbf{M}$ :

$${\theta }^{*}:=\underset{\theta }{\arg \min }\Vert \mathbf{M}-{({\mathcal{F}}_D°{\mathcal{F}}_E)}_{\theta }(\mathbf{M}){\Vert }_2^2.$$

(5)

The final goal of the autoencoder is to detect abnormal responses in the structure from the value of the reconstruction error. If damage alters the structural response, the latent representation $\mathbf{z}$ will differ with respect to that learned for the undamaged observations, yielding a high reconstruction error $\rho$ (outlier). Given the interest in investigating the potential of combining local and global monitoring variables in damage detection, three different DNN architectures are designed to accommodate the input dimension for (i) local variables $\left(\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}\right)$ , (ii) global variables $\left(\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}\right)$ , and (iii) both local and global variables $\left(\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}\right)$ .

The three DNNs follow the scheme sketched in Figure 5. All the architectures are fully connected with only dense layers. The linear encoder ${\mathcal{F}}_a$ applies the affine transformation using a single dense layer with linear activation to compress the input to the k-dimensional latent vector. The same holds for the linear decoder ${\mathcal{F}}_c$ , which expands the latent vector to the original input space. The nonlinear encoder ${\mathcal{F}}_b$ contains six hidden layers using the ReLu activation function and an output layer linearly activated to reach the latent dimension. The architecture of the nonlinear decoder ${\mathcal{F}}_d$ is symmetric with respect to that of ${\mathcal{F}}_b$ . The activation function is ReLu for all the hidden layers and linear at the output layer.

The architectures of $\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}$ and $\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$ are identical except for their input dimension, that is, 12 (the 12 eigenfrequencies) and 14 (seven inclinometers and seven strain gauges considered), respectively. The hidden layers in the nonlinear encoder contain the following number of neurons: $[32,36,48,36,48,24]$ . The encoding dimension is $k=3$ . In the case of $\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$ , which has a 26-dimensional input, the hidden layers have twice as many units as those in $\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}$ or $\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$ , and the encoding dimension is $k=8$ . This expansion broadens the search space (more trainable parameters) when working with the combination of variables. The decision on the number of hidden layers in the nonlinerar connections determines the number of parameters of the networks. We select the encoding dimension after exploring its effect in the detection task. Figure 6 shows the accuracy and precision metrics ⁸⁵ for an increasing encoding dimension within the interval $[2,12]$ for $\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$ . This figure was obtained from a test dataset with damage D1 with $10\%$ severity. The testing datasets are described in section “Damaged scenarios.” Similar results were obtained for the other damage cases. The curves reveal an optimal solution for an encoding dimension of eight, with an accuracy value of $98.08\%$ and a precision of $96.31\%$ .

Figure 6.

Sensitivity analysis of encoding dimension on accuracy and precision metrics.

The proposed architectures provided adequate results while avoiding overfitting problems. Other architectures could be employed, but optimizing the architectures is out of the scope of this work. ⁸⁶ The hyperparameter selection found in previous studies on gradient-based training of DNNs.^87,88 We employ Adam optimizer given its ability to prevent getting trapped in local minima. ⁸⁹ A small learning rate may result in slow training and, in some cases, getting stacked in a local minimum. ⁹⁰ Contrarily, a significant learning rate may cause instability during training or even divergence of the loss function. ⁹¹ In this work, a learning rate of ${10}^{-03}$ yields adequate training and validation phases. We fix the number of epochs once no significant improvement is observed in the validation loss, preventing overfitting phenomena. We employ a batch size of 1024 for the three architectures, which provides an adequate trade-off between training speed and memory usage constraints. Table 2 summarizes the main characteristics of each architecture and the training properties.

Table 2.

Summary of architecture and training specifications.

ID	Input dimension	k	Parameters	Optimizer	Epochs	Batch size	Learning rate
$\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}$	14	3	16,582	Adam	800	1024	${10}^{-03}$
$\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$	12	3	16,438	Adam	800	1024	${10}^{-03}$
$\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$	26	8	65,380	Adam	1800	1024	${10}^{-03}$

DNN: deep neural network.

The networks are trained and validated with long-term monitoring data corresponding to the healthy state of the bridge (unsupervised learning). This period covers 2 years of environmental and operational variability. New (unseen) measurements corresponding to the healthy state are expected to produce a feature vector $\mathbf{z}$ similar to those of the training observations and thus yield a small reconstruction error. However, if the new measurements correspond to a damaged scenario, the network is expected to reconstruct them poorly, delivering a high reconstruction error. The network architectures and training procedures were implemented using Tensorflow 2.8. ⁹² Figure 7 shows the evolution of the loss function during training and validation for the three networks. We observe that local and global variables independently reach convergence before 800 epochs. An epoch represents an iteration over the entire dataset to minimize the loss function. For $\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$ , since the number of trainable parameters is much higher, the minimization process achieves convergence after approximately 1800 epochs, yielding a better solution.

Figure 7.

Loss evolution of the designed DNNs: (a) loss evolution for $\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}$ , (b) loss evolution for $\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$ and (c) loss evolution for $\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$ .

Simulation of damage for testing

Once the autoencoder is trained to reconstruct healthy monitoring data under varying EOCs, we must test its performance under new unseen scenarios, including damage. From the available long-term monitoring data, $10\%$ is preserved for testing purposes, as described in section “Monitoring data.” Given the lack of experimental data in the presence of damage, we employ numerical simulations to represent some damage scenarios and test the detection ability of the methodology. We follow a two-step process to generate damaged measurements: First, we employ FE model simulations to obtain the shift in the damage-induced response compared to the healthy state. Second, we construct damaged time-domain measurements by applying the relative changes to the testing monitoring subset, which is affected by varying EOCs.

Damaged scenarios

Let us assume that the target structure, under specific EOCs, has an undamaged response ${\mathbf{m}}_{\mathrm{und}}^{\mathrm{true}}$ . This response includes the local and global variables available from the monitoring system. When damage occurs, it affects the structural response, resulting in ${\mathbf{m}}_{\mathrm{dam}}^{\mathrm{true}}$ . We obtain the relative change between the healthy and the damaged conditions as:

$${\mathbf{d}}_{\mathrm{rel}}^{\mathrm{true}}=\frac{({\mathbf{m}}_{\mathrm{dam}}^{\mathrm{true}}-{\mathbf{m}}_{\mathrm{und}}^{\mathrm{true}})}{{\mathbf{m}}_{\mathrm{und}}^{\mathrm{true}}}$$

(6)

However, since the damaged response is unavailable experimentally, we employ a FE model that represents the structural behavior. The model is built in ANSYS^® using 3D elastic beam-type (BEAM4) elements. It is an uniaxial element that allows tension, compression, torsion, and bending. Figure 8 shows an extruded view of the FE model (not the real geometry of the cross-sections).

Figure 8.

Extruded view of the FE model built in ANSYS^®.

The FE model is built based on engineering knowledge and prior information but includes simplifications and assumptions that make its response differ from that of the true structure. ⁸⁴ In order to reduce that discrepancy, the model undergoes an updating process to match the response measured during an ambient vibration test with specific EOCs. This approach is a standard practice in the scientific community since a denser sensor array is deployed during the ambient vibration test compared to long-term instrumentation, allowing to characterize the shape of the eigenmodes with higher resolution. The updating process is fully described in Magalhães et al., ⁸⁴ where the relative errors with respect to the dynamic response in the ambient vibration test were calculated. Table 3 summarizes the updating results and compares the numerical response of the final model with the ambient vibration test, revealing errors below $5\%$ . These errors correspond to the relative change between the simulated dynamic feature (eigenfrequencies) and the experimental one measured during the ambient vibration test.

Table 3.

Model updating results obtained by Magalhães et al. ⁸⁴

Mode	Ambient vibration test	FE model	Error (%)
1	0.770	0.794	3.12
2	0.810	0.810	−0.12
3	1.135	1.149	1.23
4	1.405	1.467	4.24
5	1.734	1.768	1.96
6	1.993	2.086	4.72
7	2.212	2.185	−1.18
8	3.013	3.011	−0.07
9	3.309	3.287	1.45
10	3.490	3.357	−3.81
11	3.734	3.641	−2.49
12	4.339	4.129	−4.84

FE: finite element.

Accurately simulating damage demands a substantial computational effort and human expertise. These resources could be unavailable or unprofitable, and simpler approximations are needed. In this work, damage is feasibly simulated by reducing the inertial properties to represent a loss of vertical bending stiffness at small-length segments of the structure. When simulating damage, we employ BEAM4 elements, which directly modify the inertial properties and preserve the mass value (fixed area and material). In these elements, the flexural inertia for the vertical bending can be reduced by affecting the corresponding real constant of the element, that is, without affecting the mass, material properties or other inertial properties of the bridge. For that purpose, we first alter the geometry in separate cross-section models to achieve the flexural intertia value corresponding to the damage level. The geometrical modification consists of slightly reducing the bottom slab height. Magalhaes et al. ⁹³ provide descriptive information of these geometries. Figure 9 shows the geometry modification introduced at $S_3$ (see Figure 1) to achieve a $10\%$ bending intertia reduction.

Figure 9.

Comparison of the original and modified geometry at $S_3$ to generate a $10\%$ damage level. The bottom slab height is reduced to produce a flexural inertia decrease that is applied as a constant to the BEAM4 affected elements in the bridge FE model: (a) original geometry at cross-section $S_3$ and (b) altered geometry at cross-section $S_3$ .

We then introduce the new (damaged) intertia value into the BEAM4 elements of the affected sections of the bridge FE model. It was first proposed and applied to the same bridge by Magalhães and Cunha ⁷³ to reproduce possible consequences of aging phenomena, corrosion, or extreme events. The same damage scenarios were also employed for validation in our previous work. ⁸¹ Other works, such as,^94,95 also employ stiffness reductions to reproduce damage scenarios, considering reduction levels up to $30\%$ . Figure 1 indicates the approximate locations of the studied damage cases (D1, D2, D3, and D4). Scenarios D1, D2, and D3 occur nearby instrumented deck sections, while D4 affects one column connecting the arch and deck, and it is relatively far from the positions where local variables are recorded. For scenarios D1 and D3, we affect the vertical bending inertia of two elements (one at each side of cross-sections $S_1$ and $S_6$ , respectively), covering a total length of ten meters. Analogously, damage D2 affects two elements in the central region (both sides of the instrumented section $S_3$ ), with a nine meters length. Damage D4 affects a 6.5 m long element in the corresponding arch-deck connection pier. We investigate three severity levels, namely $5\%$ , $10\%$ , and $20\%$ , which correspond to the stiffness reduction at the affected region. As an illustrative example, Table 4 summarizes the vertical bending inertia value at the affected elements in damage D2. The synthetic global variables are derived from modal analyses using ANSYS^® eigensolver. The rotations are directly obtained from static self-weight simulations. The vertical bending moments from the self-weight simulation are obtained to generate the stresses provided by the strain gauges. Then, the bending stress variations are calculated using the Navier formula for the considered inertia reductions.

Table 4.

Vertical bending inertia reductions for damage D2.

Severity level	Healthy	5%	10%	20%
Bending inertia $(m^4)$	71.78	68.19	64.60	57.42

Analogous to Equation (6), we calculate the relative differences between the undamaged and the damaged responses for the FE model as:

$${\mathbf{d}}_{\mathrm{rel}}^{\mathrm{FE}}=\frac{{\mathbf{m}}_{\mathrm{dam}}^{\mathrm{FE}}-{\mathbf{m}}_{\mathrm{und}}^{\mathrm{FE}}}{{\mathbf{m}}_{\mathrm{und}}^{\mathrm{FE}}},$$

(7)

where ${\mathbf{m}}_{\mathrm{und}}^{\mathrm{FE}}$ and ${\mathbf{m}}_{\mathrm{dam}}^{\mathrm{FE}}$ correspond to the numerical responses obtained in the undamaged and damaged states, respectively. Tables 5 and 6 gather the relative differences between local and global variables. In Table 5, the sensors with a null relative difference $(d_{\mathrm{rel}}=0)$ for all the considered damaged scenarios have been omitted, namely $S_{21},S_{22},S_{41},S_{42},S_{43}$ , and $S_{44}$ .

Table 5.

Relative differences $d_{\mathrm{rel}}(\%)$ for local variables.

Damage	Severity	$S_{11}$	$S_{12}$	$S_{31}$	$S_{32}$	$S_{33}$	$S_{51}$	$S_{52}$	$S_{62}$
D1	$5\%$	−1.61	1.61	0.00	0.00	0.00	0.00	0.00	0.00
	$10\%$	−3.42	3.42	0.00	0.00	0.00	0.00	0.00	0.00
	$20\%$	−7.54	7.54	0.00	0.00	0.00	0.00	0.00	0.00
D2	$5\%$	0.00	0.00	4.94	4.94	4.94	0.00	0.00	0.00
	$10\%$	0.00	0.00	10.43	10.43	10.43	0.00	0.00	0.00
	$20\%$	0.00	0.00	23.47	23.47	23.47	0.00	0.00	0.00
D3	$5\%$	0.00	0.00	0.00	0.00	0.00	0.00	0.00	−1.76
	$10\%$	0.00	0.00	0.00	0.00	0.00	0.00	0.00	−3.61
	$20\%$	0.00	0.00	0.00	0.00	0.00	0.00	0.00	−7.95
D4	$5\%$	0.00	0.00	0.00	0.00	0.00	0.23	0.23	0.00
	$10\%$	0.00	0.00	0.00	0.00	0.00	0.21	0.21	0.00
	$20\%$	0.00	0.00	0.00	0.00	0.00	1.02	1.02	0.00

Table 6.

Relative differences $d_{\mathrm{rel}}(\%)$ for global variables.

Damage	Severity	$f_1$	$f_2$	$f_3$	$f_4$	$f_5$	$f_6$	$f_7$	$f_8$	$f_9$	$f_{10}$	$f_{11}$	$f_{12}$
D1	$5\%$	0.00	−0.08	−0.02	−0.07	0.00	−0.06	−0.03	0.00	−0.01	0.00	0.00	0.00
	$10\%$	0.00	−0.17	−0.03	−0.14	0.00	−0.12	−0.06	0.00	−0.01	0.00	−0.00	−0.01
	$20\%$	0.00	−0.38	−0.05	−0.31	0.00	−0.26	−0.13	0.00	−0.02	−0.00	−0.01	−0.02
D2	$5\%$	0.00	0.00	−0.03	−0.03	0.00	0.00	−0.03	0.00	−0.00	−0.01	0.00	0.00
	$10\%$	0.00	0.00	−0.06	−0.07	0.00	0.00	−0.07	0.00	−0.00	−0.03	0.00	−0.00
	$20\%$	0.00	0.00	−0.13	−0.14	0.00	−0.00	−0.16	0.00	−0.00	−0.06	−0.00	−0.00
D3	$5\%$	0.00	−0.09	−0.02	−0.08	0.00	−0.07	−0.04	0.00	−0.01	0.00	−0.00	−0.00
	$10\%$	0.00	−0.19	−0.03	−0.16	0.00	−0.14	−0.08	0.00	−0.01	−0.00	−0.00	−0.01
	$20\%$	0.00	−0.41	−0.06	−0.35	0.00	−0.31	−0.17	0.00	−0.02	−0.01	−0.01	−0.03
D4	$5\%$	0.00	−0.06	−0.01	−0.07	0.00	−0.01	−0.01	0.03	−0.12	−0.07	0.01	−0.00
	$10\%$	0.00	−0.11	−0.01	−0.14	0.00	−0.03	−0.03	0.03	−0.25	−0.12	0.01	−0.00
	$20\%$	0.00	−0.24	−0.02	−0.29	0.00	−0.06	−0.08	0.03	−0.53	−0.23	0.01	−0.01

Given the impossibility to obtain $d_{\mathrm{rel}}^{\mathrm{true}}$ under any real damage scenario, we assume that $d_{\mathrm{rel}}^{\mathrm{true}}\approx d_{\mathrm{rel}}^{\mathrm{FE}}$ . Hence, we can affect a true undamaged response to embed damage by applying ${\mathbf{d}}_{\mathrm{rel}}^{\mathrm{FE}}$ as:

$${\mathbf{m}}_{\mathrm{dam}}^{\mathrm{true}}\approx {\mathbf{m}}_{\mathrm{und}}^{\mathrm{true}}(1+{\mathbf{d}}_{\mathrm{rel}}^{\mathrm{FE}})$$

(8)

In this work, we assume that modeling error affects the structural response differently than damage and, thus, that the shift due to modeling error will not overlap with the shift produced by damage. Figure 10 schematically represents this assumption. With this assumption, the minimum detectable damage depends on the sensitivity of the response and not on the modeling error.

Figure 10.

Schematic representation of modeling error and damage-induced changes.

Damaged datasets

The proposed FE simulations disregard the effect of environmental and operational variability. Since the monitoring system provides one observation per hour, running time-domain simulations becomes unfeasible to generate a representative dataset for evaluating the proposed methodology. This modeling scope poses a limitation to the proposed methodology since the change between the healthy and the damaged responses (Equation (6)) may slightly change under varying EOCs. However, it is believed that this second-order phenomenon barely influences the ratio and can be neglected, as supported, for example, in Magalhães ⁶ and Comanducci et al. ²⁶

In order to incorporate the effect of EOCs in the testing phase, we apply Equation (8) to the measurements in the monitoring subset preserved for testing. These measurements are subject to the effect of varying EOCs unseen during training and thus incorporate this variability during the testing phase. We assume that the damage evolves slowly enough to consider that it remains constant during the testing period. We build 12 damaged datasets (one for each scenario and severity level). The healthy testing dataset contains $N_{\mathrm{und}}=1439$ observations corresponding to $10\%$ of the monitoring data (unseen during training). Accordingly, each damaged dataset contains $N_{\mathrm{dam}}=N_{\mathrm{und}}=1439$ observations. With this strategy, we produce time-domain damaged measurements affected by changing EOCs to be evaluated during testing. For each damage scenario, the entire testing dataset includes the $N_{\mathrm{und}}$ healthy observations and the corresponding $N_{\mathrm{dam}}$ samples to evaluate the ability of the autoencoder as an outlier detector. Figures 11(a) and (b) show the effect of damage D1 with $10\%$ severity on sensor $S_{11}$ and eigenfrequency $f_4$ , respectively.

Figure 11.

Effect of damage D1 with 10% severity on: (a) effect of damage D1 with 10% severity on $S_{11}$ and (b) effect of damage D1 with 10% severity on $f_4$ .

Results

Damage detection

As introduced in section “Deep neural network approach,” the reconstruction error $\rho$ (see Equation (4)) is employed as the indicator to detect abnormal structural behavior. If damage alters the structural response, the latent representation $\mathbf{z}$ will differ from that learned for the undamaged observations, yielding a high reconstruction error $\rho$ (outlier). To evaluate the performance in detecting structural damage, the ROC curves ⁸² are calculated. ROC curves are widely employed to measure the performance of classifiers. ⁴² These curves represent the rate of false-positive predictions (FPRs) against the rate of true-positive predictions (TPRs).

In the context of SHM, a false positive indicates healthy data misclassified as damage, and a true positive indicates a correctly classified damaged scenario. Unsupervised learning results in a single-class classification, where undamaged data belong to the healthy class and any damaged scenario corresponds to an outlier. ⁹⁶ The detection ability consists of classifying new unseen healthy observations as undamaged and detecting any departure as an outlier. False-positive and true-positive rates are obtained according to a threshold value $\alpha$ that establishes the maximum $\rho$ value corresponding to the undamaged state. Measurements that produce error values above $\alpha$ are classified as damaged. We calculate the threshold $\alpha$ as the k-percentile over the training reconstruction error dataset. For example, seting $\alpha$ to the 95-percentile means that $5\%$ of the training reconstruction errors exceed the threshold and are false positives. When building the ROC curves, we vary the threshold $\alpha$ to sweep the FPR within the interval [0,1] and display it against the TPR. A perfect classifier operates along the horizontal line at TPR = 1, and a random classifier would follow the diagonal line ( $45°$ ) that crosses the ROC space. ⁴²

Let ${\rho }_{t_{\mathrm{und}}}$ and ${\rho }_{t_{\mathrm{dam}}}$ denote the reconstruction errors for the undamaged and damaged testing subsets, respectively. The FPR and TPR for a threshold value ( $\alpha$ ) are obtained as:

$$\mathrm{FPR}=\frac{\sum _{j=1}^{N_{\mathrm{und}}}({\rho }_{t_{\mathrm{und}}^j}>\alpha )}{N_u},$$

(9)

$$\mathrm{TPR}=\frac{\sum _{j=1}^{N_{\mathrm{dam}}}({\rho }_{t_{\mathrm{dam}}^j}>\alpha )}{N_d},$$

(10)

Figures 12 to 14 compare the performance of the three designed DNNs for each type of damage to demonstrate the benefit of combining local and global variables. In the figures, the random classifier is represented with a grey dashed line. The performance increases as the curve moves away from the grey line and approximates the top of the ROC space.

Figure 12.

ROC curves for damage severity 5%: (a) damage D1 5% severity, (b) damage D2 5% severity, (c) damage D3 5% severity and (d) damage D4 5% severity.

Figure 13.

ROC curves for damage severity 10%: (a) damage D1 10% severity, (b) damage D2 10% severity, (c) damage D3 10% severity and (d) damage D4 10% severity.

Figure 14.

ROC curves for damage severity 20%: (a) damage D1 20% severity, (b) damage D2 20% severity, (c) damage D3 20% severity and (d) damage D4 20% severity.

Local variables exhibit good performance for damage scenarios D1–D3, mainly for severity levels above $10\%$ (see Figures 13 and 14). In the case of damage D2, where three strain gauges are installed in the region affected by the damage, local variables yield a perfect classifier even for a slight severity level (see Figure 12). Since D3 occurs close to only one sensor (at S6), the sensitivity is lower than with damage D1 (similar damage at S1), where two inclinometers participate. But when damage occurs far from any instrumented section (D4), local variables turn insensitive and yield a poor classifier even for a high severity level (see Figure 14). Hence, the performance of local variables is adequate as long as the damage occurs nearby an instrumented section but strongly worsens for other damage locations.

Given the limited sensitivity to damage severity of $5\%$ , we perform further analysis based on the $10\%$ damage scenarios. Table 7 gathers the quantitative results for a damage severity of $10\%$ in terms of four metrics: area under curve (AUC), precision, recall and F1-score. These are standard metrics to evaluate the performance of classifiers. ⁸⁵ The AUC metric corresponds to the area under the classifier ROC curves, with value in the interval $[0.5,1]$ .The obtained results reveal the significant enhancement in the performance attained when combining local and global variables. Even if local variables demonstrate an elevated sensitivity to damage scenarios $\mathrm{D}1$ , D2, and D3 given their proximity to the instrumented sections, they are highly blind to any other damage. This situation fosters considering eigenfrequencies in the search for more global detection capabilities. We observe that the performance of global variables rises when complemented with local information, as in the case of damage D3.

Table 7.

Performance metrics for damage severity $10\%$ .

Damage	DNN	AUC	Precision (%)	Recall (%)	F1-score (%)
D1	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}$	0.9873	80.43	100.00	89.15
	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$	0.8204	91.60	42.61	58.16
	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$	0.9889	96.82	100.00	98.39
D2	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}$	0.9889	80.43	100.00	89.15
	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$	0.5887	63.40	6.76	12.22
	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$	0.9889	96.83	100.00	98.39
D3	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}$	0.9035	80.43	100.00	89.15
	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$	0.8647	93.44	55.65	69.76
	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$	0.9888	96.83	100.00	98.39
D4	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}$	0.5191	51.46	25.80	34.37
	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$	0.9693	96.21	99.02	97.59
	$\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$	0.9878	96.82	99.93	98.35

AUC: area under curve; DNN: deep neural network.

We also explore the latent space for each damage scenario. Figure 15 compares the latent space representation of local variables for a $10\%$ severity. Figure 15(b) highlights the sensitivity of strain gauges to damage with a huge difference between the healthy and the damaged latent representation. Figure 15(d) evidences that damage D4 barely affects local variables, making the healthy and the damaged latent representations overlap throughout the testing period.

Figure 15.

Comparison of the healthy (blue) and damaged (red) 3-dimensional latent space for $\mathrm{DN}{\mathrm{N}}_{\mathrm{Local}}$ : (a) damage D1 with 10% severity, (b) damage D2 with 10% severity, (c) damage D3 with 10% severity and (d) damage D4 with 10% severity.

On the other hand, global variables adequately detect the four damage scenarios when the severity reaches $20\%$ (see Figure 14). However, the performance decays for lower severity levels, indicating a limited sensitivity to slight damage, as shown in Figures 12 and 13. Figure 16 compares the latent space representation for each damage with $10\%$ severity for global variables. Results are particularly deficient in the case of damage D2 (according to Figure 16(d), only latent variable $z_2$ shows some variation with respect to the undamaged state). To explain this, the eigenmodes associated with the eigenfrequencies are analyzed. The shape of an eigenmode represents the vibration amplitude of each structural node when the bridge is excited at the corresponding eigenfrequency. ⁹⁷ The curvature at a certain node describes its bending level in the mode shape. When damage occurs close to a node with a small curvature value for a certain eigenmode, the shift induced by damage will also be small, resulting in a poor sensitivity of the corresponding eigenfrequency. Thus, eigenfrequencies are more sensitive to damage that occurs close to nodes with higher curvature values.

Figure 16.

Comparison of the healthy (blue) and damaged (red) 3-dimensional latent space for $\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$ : (a) damage D1 with 10% severity, (b) damage D2 with 10% severity, (c) damage D3 with 10% severity and (d) damage D4 with 10% severity.

Since damage was introduced as a vertical stiffness reduction, it mainly affects the vertical bending eigenmodes. In this case, only four out of the 12 global variables represent vertical bending eigenmodes and thus participate in detecting the proposed damage scenarios. The rest of the mode shapes are lateral and torsional. Figure 17 shows the four vertical bending mode shapes adopted in this work. For damage cases D1–D3, (occuring at the deck of the bridge), damage D2 presents practically null curvature, yielding the worst results. Damage D2 occurs in the middle deck (see Figure 1) and it slightly affects global variables due to the low curvature of the involved eigenmodes at this position (see mainly Figure 17(a) and (d)). Hence, although global variables increase the range of detectable damage compared to local variables alone, they still present some limitations, mainly for reduced severity levels. It is worth noting that $\mathrm{DN}{\mathrm{N}}_{\mathrm{Global}}$ achieves the best performance for damage D4 (see Figure 13(d)), which presented worse results when using local variables only. Global variables are more sensitive to damage D4, which occurs at a different location (in an arch-deck connection pier; see Figure 1).

Figure 17.

Vertical bending eigenmodes: (a) first vertical eigenmode, (b) second vertical eigenmode, (c) third vertical eigenmode and (d) fourth vertical eigenmode.

In light of the previous observations, it becomes interesting to combine both sources of monitoring data. The red dashed line in Figures 12 to 14 reveals that combining local and global variables outperforms the classification ability of local or global variables alone. The ROC curves denote a multiplicative enhancement of the results that goes beyond the sum of the individual contributions of local and global variables. This becomes more evident mainly for lighter damages (see, e.g., Figure 12(c)). In the case of damage D4, since the contribution of local variables is very low (see, e.g., Figure 14(d)), the combined solution practically coincides with the performance of global variables only. Figure 18 compares the 8-dimensional latent space representations of the damage scenarios ( $10\%$ severity) with respect to the undamaged condition produced by $\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$ . Results show the discrepancies between damaged and undamaged latent representations (e.g., damage D1 mainly affects the second, seventh, and eighth dimensions, whereas damage D2 affects all of them).

Figure 18.

Comparison of the healthy (blue) and damaged (red) eight-dimensional latent space during testing: (a) damage D1 with 10% severity, (b) damage D2 with 10% severity, (c) damage D3 with 10% severity and (d) damage D4 with 10% severity.

In general, the obtained ROC curves for the combined approach are very good for severity levels of $10\%$ (see Figure 13). The minimum detectable damage can thus be established within the interval $5\%-10\%$ , depending mainly on the contribution of local variables.

Damage location

In addition to detecting damage occurring nearby the sensor emplacement, local variables also contribute to determining the damage location. It is expected that the autoencoder yields an outlier ( $\rho >\alpha$ ) when an abnormal behavior that affects the structural response registers through monitoring. It is assumed that this abnormal behavior corresponds to damage at any bridge location. Once the reconstruction error exceeds the threshold value (α) indicating damage, the contribution of local variables to the reconstruction error may be informative of the damage location. Here, the four damage scenarios considered for testing are investigated, whose location is known beforehand, to verify the capability of local variables in damage location.

It is assumed that if damage occurs nearby an instrumented section, it will mainly affect the corresponding local variables, and the autoencoder will reconstruct these variables more deficiently. Hence, the highest contribution to the reconstruction error would result from variables measured with the sensors located nearby the damaged region. Here, the reconstruction error of local variables is analyzed using the combined autoencoder ( $\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$ ). For a particular (randomly chosen) observation in the damaged testing datasets, the individual reconstruction error of the local variables is obtained using Equation (3). The contribution to the reconstruction error (%) is calculated at each instrumented section and results are compared. Tables 8 to 11 gather the reconstruction error at each section for each of the considered damage scenarios. When the instrumented section contains more than one local variable, the average value of the involved variables is computed. For damage scenarios D1–D3, the error values corresponding to the instrumented section nearby the damage (see Figure 1) are highlighted in bold.

Table 8.

Contributions to reconstruction error for D1.

Instrumented section	Severity level
	5%	10%	20%
$S_1$	16.44	22.48	30.77
$S_2$	0.09	0.16	0.32
$S_3$	1.55	0.96	1.05
$S_4$	1.38	1.58	1.70
$S_5$	0.60	0.86	0.99
$S_6$	3.44	6.78	0.03

Table 9.

Contributions to reconstruction error for D2.

Instrumented section	Severity level
	5%	10%	20%
$S_1$	0.08	0.00	0.09
$S_2$	0.00	0.10	0.08
$S_3$	18.96	24.57	23.03
$S_4$	8.53	2.96	1.75
$S_5$	0.25	0.40	0.27
$S_6$	1.07	0.11	1.19

Table 10.

Contributions to reconstruction error for D3.

Instrumented section	Severity level
	5%	10%	20%
$S_1$	0.07	0.33	0.72
$S_2$	0.24	0.60	0.90
$S_3$	3.51	2.47	1.74
$S_4$	2.00	3.06	3.76
$S_5$	0.68	0.38	0.21
$S_6$	33.21	43.90	51.29

Table 11.

Contributions to reconstruction error for D4.

Instrumented section	Severity level
	5%	10%	20%
$S_1$	3.52	1.65	0.16
$S_2$	0.73	0.36	0.01
$S_3$	3.44	1.70	0.54
$S_4$	3.79	1.15	0.20
$S_5$	4.36	1.58	1.62
$S_6$	0.21	0.16	0.10

Results demonstrate that damage D1–D3, which occur close to instrumented sections that measure local variables, are adequately located even for reduced severity levels (see Tables 8 to 10). The contribution of the affected section is considerably higher than those of the rest of the instrumented sections. By contrast, local variables are unable to inform about the location of D4, since no local variables are recorded close to it. This means that local variables only aid to locate damage occurring close to their emplacement.

SHM strategy

The final goal of the methodology is to detect abnormal behavior of Infante Dom Henrique bridge during operation from experimental monitoring data. Given the successful results observed in section “Results,” we employ the combination of local and global variables as the input data and thus employ the corresponding DNN ( $\mathrm{DN}{\mathrm{N}}_{\mathrm{Combined}}$ ), developed and trained in section “Deep neural network approach.” The damage detection strategy employs control charts of the reconstruction error ⁹⁸ to produce an alert system that detects structural damage early.

We employ the cumulative error after five measurements to reduce the risk of occurrence of false positives and produce a more robust alert system. ⁶ This means that the detection time is 5 h, since the monitoring system provides one observation per hour including the measurements of the local and global variables. The threshold value $\alpha$ for the reconstruction error is established at the 97-percentile of the training dataset to obtain an adequate trade-off between the number of false positives and false negatives. Five testing subsets are concatenated: healthy, damage D1, damage D2, damage D3, and damage D4. After the analysis developed in section “Results” for three increasing damage severity levels, here we focus on a $10\%$ severity to evaluate the performance of the methodology.

Damage detection

Figure 19 shows the control chart of the reconstruction error. The logarithmic scale is applied to the reconstruction errors to better visualize the results. The grey-shadowed region indicates the part of the testing with damage, and the vertical dashed lines delimit the subset associated with each damage. Measurements exceeding the threshold value (blue horizontal line) are red-colored, and those below the threshold are green-colored. The figure reveals that undamaged measurements are properly classified as healthy, with a $3.48\%$ of false positives. We also observe that the four damage scenarios are adequately detected. In the case of damage D2, since three strain gauges exist at the affected region, the reconstruction error is very sensitive and robustly detects it. This means that lighter damage occurring at that location will be successfully detected. Figures 12 and 18(b) already manifested this sensitivity. Damage scenarios D1 and D3, which occur close to inclinometers, are adequately detected with some slack for lighter severity levels, specially damage D1, where two sensors participate. On the other hand, when analyzing damage D4, a $2.02\%$ of false negatives is attained, indicating that lower severity levels will be more difficult to detect. Hence, when only global variables participate since no local variables exist nearby the damage, the minimum detectable severity is $10\%$ .

Figure 19.

Testing control chart with damage severity of $10\%$ .

Damage location

We finally analyze the four damaged testing datasets to explore/investigate the ability of the method in damage location. We obtain the contribution to the reconstruction error at each section instrumented with local variable sensors ( $S_1-S_6$ in Figure 1). Figure 20 displays the contributions to the reconstruction error $\left(\rho \right)$ for the testing datasets affected by damage scenarios D1–D4. Figure 20(a) to (c) demonstrate that when damage occurs nearby an instrumented section, we can successfully locate it given the high contribution of the corresponding sensors to the reconstruction error. As expected, for damage D4, which occurs far from the monitoring devices, we obtain very small contributions from all the local variables (see Figure 20(d)). In such case, the damage is detectable but its location is unknown.

Figure 20.

Contribution of local variables to the reconstrution error in presence of damage: (a) local variable contributions to $\rho$ under damage D1, (b) local variable contributions to $\rho$ under damage D2, (c) local variable contributions to $\rho$ under damage D3 and (d) local variable contributions to $\rho$ under damage D4.

Conclusion

An unsupervised learning approach was developed and tested to detect damage at the Infante Dom Henrique bridge in Porto. A particular DNN based on an autoencoder with two adding blocks at the encoder and decoder was employed to account for nonlinear relationships. The reconstruction error, which measures the discrepancy between original and predicted measurements, is the damage-sensitive feature for outlier detection. Reaching a higher identification level (quantification and location) requires adopting a supervised learning strategy where simulated damaged data participates in the training phase of the data-driven model, which is a topic of interest to our research. Three DNN architectures were created to accommodate different inputs and investigate the potential of combining measurements from various sensor types, namely, local and global variables. The testing phase included healthy and damaged measurements to produce the ROC curves. The damage was simulated using a FE model of the structure by reducing the vertical bending stiffness to produce the relative change between the healthy and damaged responses. This relative shift affected a portion of the monitoring data preserved for testing purposes, subject to environmental and operational variability.

The classification performance of local, global, and a combination of both variables was compared and examined by analyzing the resulting ROC curves and the latent space representation. Some standard metrics (AUC, precision, recall, and F1-score) evaluated the classification performance. Results revealed that local variables successfully detect damage occurring nearby the sensor emplacement but strongly fail for distant damage, such as D4, which practically yields a random classifier. Global variables enlarge the detectable space but with limited sensitivity for slight severity levels. Besides, they still present some gaps at locations where the associated eigenmodes have a small curvature that restricts the sensitivity of the corresponding eigenfrequency. It is the case of damage D2, which provides a low-performance classification.

After analyzing the combination of variables, an enhancement in the detection ability of the reconstruction error was observed, allowing us to detect lighter severity levels. When damage occurs far from local variables, the combined DNN reaches only the same solution as global variables alone since there is no contribution from local variables. Results reflect the enhancement attained when combining both data sources in terms of damage sensitivity.

We also investigated the ability of local variables to inform about the location of damage. By calculating the contribution of local variables at each instrumented section to the total reconstruction error, we attained that local variables successfully locate damage that occurs in the vicinity of the sensor emplacement. We concluded that local variables aid in the detection task and contribute to finding the possible damage location. However, the method is unable to locate damage when it occurs far from instrumented sections, which poses a limitation to the approach in locating damage.

Results suggested that an important issue when implementing SHM systems lies in the adequate disposal of the sensors. Deploying many sensors helps to improve diagnostics, but it results in inadmissible costs. An intelligent positioning strategy might yield a more efficient solution. Essential locations are, apart from those more susceptible to suffering damage (e.g., joints, pier foundations), those where eigenmodes present small curvature values since global variables lose sensitivity at these positions. Since eigenmodes can be obtained beforehand (through an ambient vibration test or numerical simulations), those nodes that are more “blind” to global variables can be identified and supported with local variable sensors.

The proposed method presents some modeling limitations that must be mentioned and addressed. The employed FE model is subject to assumptions and simplifications that introduce a discrepancy with respect to the true structural response. Although we overcome this discrepancy by tuning the boundary conditions in the model, a more sophisticated method based on iterative updating could provide more accurate results. Besides, the employed FE model disregards the effect of varying EOCs since it is updated for the specific EOCs occurring during an ambient vibration test. We consider future work incorporating EOCs in the damage simulations to obtain more realistic shifts between healthy and damaged scenarios. Using bayesian approaches both in the updating process and in the autoencoder architecture contributes to handling these variability sources mainly when combining numerical simulations with experimental data, posing a key topic for our future developments.

We also consider as future research the investigation of other damage scenarios to extend the validation of the proposed methodology and to demonstrate the potential of monitoring data in the early assessment of the structural behavior of bridges. Representing degradation phenomena in the system or subcomponents to identify performance trends and prevent catastrophic failures poses a future study of interest focused on predictive maintenance.

Finally, in line with our previous works on supervised learning,^99,100 we also contemplate as future work investigating the discrepancy between the experimental and numerical healthy state and its effect in the damage identification task. We will explore domain adaptation techniques to minimize the distance between both spaces and accurately classify damage location and severity.