Data-driven structural health monitoring for early corrosion detection in the oil and gas pipelines: enhancing sensitivity with unsupervised dimensionality reduction
Restricted accessResearch articleFirst published online July, 2026
Data-driven structural health monitoring for early corrosion detection in the oil and gas pipelines: enhancing sensitivity with unsupervised dimensionality reduction
Corrosion presents significant threats to structural integrity within the oil and gas industry, increasing maintenance demands and associated costs. However, traditional inspection techniques often face limitations in detecting early-stage corrosion, particularly under challenging environmental and operational conditions. This study investigates structural health monitoring using low computational cost data-driven approaches, specifically principal component analysis, t-distributed stochastic neighbor embedding, and locally linear embedding to enable early and autonomous corrosion detection capabilities in a scalable manner for embedded systems. Corrosion was induced in a steel pipe using an ionic solution, with nearly 400 GB of guided wave signal data recorded across well-defined corrosion stages via an array of piezoelectric transducers. The results demonstrate that the proposed approach enables the detection of corrosion with a sensitivity of up to 0.39% reduction in the cross-sectional area. Furthermore, the results validate the feasibility of establishing autonomous damage detection thresholds, mitigating the need for periodic inspections. The methodology proved robust, effectively isolating environmental effects without the need for environmental and operational condition compensation techniques. Unsupervised dimensionality reduction effectively detects early structural changes, such as corrosion and transduction loss, with guided waves and optimized parameters, offering strong support for traditional inspection methods.
SharmaVBTewariSBiswasS, et al. A comprehensive study of techniques utilized for structural health monitoring of oil and gas pipelines. Struct Health Monit2024; 23(3): 1816–1841.
2.
FarrarCRWordenK.Structural health monitoring: a machine learning perspective. Chichester: John Wiley Sons, 2012.
3.
RoseJL.Ultrasonic guided waves in solid media. Cambridge: Cambridge University Press, 2014.
4.
MollJKexelCPötzschS, et al. Temperature affected guided wave propagation in a composite plate complementing the Open Guided Waves Platform. Sci Data2019; 6(1): 1–9.
5.
AttarianVACeglaFBCawleyP.Long-term stability of guided wave structural health monitoring using distributed adhesively bonded piezoelectric transducers. Struct Health Monit2014; 13(3): 265–280.
6.
LanzaraGYoonYKimY, et al. Influence of interface degradation on the performance of piezoelectric actuators. J Intell Mater Syst Struct2009; 20(14): 1699–1710.
7.
ParkGFarrarCRDi ScaleaFL, et al. Performance assessment and validation of piezoelectric active-sensors in structural health monitoring. Smart Mater Struct2006; 15(6): 1673–1673.
8.
MarianiSHeinleinSCawleyP.Compensation for temperature-dependent phase and velocity of guided wave signals in baseline subtraction for structural health monitoring. Struct Health Monit2020; 19(1): 26–47.
9.
AzimiMEslamlouAPekcanG.Data-driven structural health monitoring and damage detection through deep learning: state-of-the-art review. Sensors2020; 20(10): 2778.
10.
BishopCM.Pattern recognition and machine learning. New York, NY: Springer, 2006.
11.
LiuXYooCXingF, et al. Deep unsupervised domain adaptation: a review of recent advances and perspectives. APSIPA Trans Signal Inform Process2022; 11(1): 25–25.
12.
HassanMALeeCG.Alleviating confirmation bias in perpetually dynamic environments: continuous unsupervised domain adaptation-based condition monitoring (CUDACoM). Eng Appl Artif Intell2024; 137: 109057.
13.
LeeK. Trends of modern processors for AI acceleration. In: 2021 18th International SoC design conference (ISOCC), Jeju, South Korea, 2021. pp. 227–227.
14.
BatzolisEVrochidouEPapakostasGA.Machine learning in embedded systems: limitations, solutions and future challenges. In: 2023 IEEE 13th Annual computing and communication workshop and conference (CCWC), Las Vegas, 2023. pp. 0345–0350.
15.
PatilSBanerjeeSTallurS.Smart structural health monitoring (shm) system for on-board localization of defects in pipes using torsional ultrasonic guided waves. Sci Rep2024; 14: 24455.
16.
TusunMEMetinEMTigrelF, et al. Embedded machine learning system design for post-earthquake structural health assessment. In: 2024 32nd signal processing and communications applications conference (SIU), Mersin, Turkey, 2024. pp. 1–4.DOI:10.1109/SIU61531.2024.10600859.
RébillatMMechbalN.Damage localization in geometrically complex aeronautic structures using canonical polyadic decomposition of lamb wave difference signal tensors. Struct Health Monit2020; 19(1): 305–321.
19.
AbbassiARömgensNTritschelFF, et al. Evaluation of machine learning techniques for structural health monitoring using ultrasonic guided waves under varying temperature conditions. Struct Health Monit2022; 22(2): 1308–1325.
20.
SchneiderPXhafaF.Anomaly detection and complex event processing over IoT data streams. Amsterdam: Elsevier, 2022.
21.
WollschlaegerMSauterTJasperneiteJ. The future of industrial communication: automation networks in the era of the internet of things and industry 4.0. IEEE Ind Electron Mag2017; 11(1): 17–27.
22.
OliveiraHTHd. Projeto de um colar de ondas guiadas para aplicação em tubulação enterrada. Dissertation, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil, 2017.
23.
CarbasRJCMarquesEASda SilvaLFM, et al. Effect of cure temperature on the glass transition temperature and mechanical properties of epoxy adhesives. J Adhes2014; 90(1): 104–119.
24.
JacquesRCde OliveiraHHdos SantosRWF, et al. Design and in situ validation of a guided wave system for corrosion monitoring in coated buried steel pipes. J Nondestr Eval2019; 38(3): 65.
25.
RyanH.Ricker, Ormsby and Klander, Butterworth, a choice of wavelets. CSEG Recorder Mag1994; 19(7): 8–9.
26.
de Sá RodriguesFGiannakeasINKhodaeiZS, et al. Probability based damage detection on a composite fuselage panel based on large data set of guided wave signals. NDT E Int2023; 139: 102924.
27.
KerschenGGolinvalJC.Dimensionality reduction using linear and nonlinear transformation. In: Encyclopedia of Structural Health Monitoring, vol. 33. Chichester: John Wiley & Sons, Ltd, 2009, pp. 1–13.
28.
LangerHFalsaperlaSHammerC. Chapter 6 - a posteriori analyses—advantages and pitfalls of pattern recognition techniques. In: LangerHFalsaperlaSHammerC (eds.) Advantages and pitfalls of pattern recognition, computational geophysics, vol. 3. Elsevier, 2020. pp. 237–259.
29.
JungA. Machine learning: the basics (Machine Learning: Foundations, Methodologies, and Applications). 1st ed. Singapore: Springer, 2022.
30.
CanonacoGRoveriMAlippiC, et al. Corrosion prediction in oil and gas pipelines: a machine learning approach. In: 2020 IEEE international instrumentation and measurement technology conference (I2MTC). Dubrovnik, Croatia: IEEE, pp. 1–6.
31.
FangJChengXGaiH, et al. Development of machine learning algorithms for predicting internal corrosion of crude oil and natural gas pipelines. Comput Chem Eng2023; 177: 108358.
32.
EltounyKGomaaMLiangX.Unsupervised learning methods for data-driven vibration-based structural health monitoring: a review. Sensors2023; 23(6): s23063290.
33.
WangZChaYJ.Unsupervised deep learning approach using a deep auto-encoder with a one-class support vector machine to detect damage. Struct Health Monit2021; 20(1): 406–425.
34.
ChaYJWangZ.Unsupervised novelty detection–based structural damage localization using a density peaks-based fast clustering algorithm. Struct Health Monit2018; 17(2): 313–324.
35.
WangZChaYJ.Unsupervised machine and deep learning methods for structural damage detection: a comparative study. Eng Rep2022; 7(1): e12551.
36.
HastieTTibshiraniRFriedmanJ.The Elements of statistical learning. 2 ed. Springer Series in Statistics, New York, NY: Springer New York, 2009.
37.
DervilisNShiHWordenK, et al. Exploring environmental and operational variations in SHM data using heteroscedastic Gaussian processes. In: PakzadSJuanC (eds.) Dynamics of civil structures, Volume 2. Cham: Springer International Publishing, pp. 145–153.
38.
MaleklooAOzerEAlHamaydehM, et al. Machine learning and structural health monitoring overview with emerging technology and high-dimensional data source highlights. Struct Health Monit2022; 21(4): 1906–1955.
39.
ChaYJAliRLewisJ, et al. Deep learning-based structural health monitoring. Autom Constr2024; 161: 105328.
40.
BellmanRE.Adaptive control processes. A Guided Tour. Princeton: Princeton University Press, 1961.
41.
BanksDLFienbergSE.Data mining, statistics. In: MeyersRA (ed.) Encyclopedia of Physical Science and Technology, 3 ed. New York: Academic Press, 2003, pp. 247–261.
42.
KokJNKoronackiJLopezdeMantarasR, et al. (eds.) Machine learning: ECML 2007: 18th European conference on machine learning, Warsaw, Poland, September 17-21, 2007; proceedings. Number 4701 in Lecture notes in computer science Lecture notes in artificial intelligence, Berlin Heidelberg: Springer, 2007.
43.
LeeJAVerleysenM.Nonlinear dimensionality reduction. 1 ed. Information Science and Statistics, New York, NY: Springer, 2007.
44.
LespinatsSColangeBDutykhD.Nonlinear dimensionality reduction techniques: a data structure preservation approach. Cham, Switzerland: Springer Nature Switzerland AG, 2022.
45.
Fernandez PiernaJABaetenVDardenneP, et al. Spectroscopic imaging. Compr Chemom2009; 4: 173–196.
46.
GhojoghBCrowleyMKarrayF, et al. Elements of dimensionality reduction and manifold learning. Cham: Springer International Publishing, 2023.
47.
GarzonMYangCCVenugopalD, et al. Dimensionality reduction in data science. Cham, Switzerland: Springer, 2022.
48.
De SilvaVTenenbaumJB. Global versus local methods in nonlinear dimensionality reduction. In: BeckerSThrunSObermayerK (eds.) Advances in Neural Information Processing Systems, British Columbia, Canada, vol. 15, pp. 705–712. Cambridge, Massachusetts, USA: MIT Press, 2002.
ZhouYSharpeeTO.Using global T-SNE to preserve intercluster data structure. Neural Comput2022; 34(8): 1637–1651.
51.
AnowarFSadaouiSSelimB.Conceptual and empirical comparison of dimensionality reduction algorithms (PCA, KPCA, LDA, MDS, SVD, LLE, ISOMAP, LE, ICA, t-SNE). Comput Sci Rev2021; 40: 100378.
52.
JolliffeIT.Principal component analysis. New York: Springer: Springer Series in Statistics, 2002.
53.
PearsonK.LIII. On lines and planes of closest fit to systems of points in space. London Edinburgh Dublin Philos Mag J Sci1901; 2(11): 559–572.
54.
HotellingH.Analysis of a complex of statistical variables into principal components. J Educ Psychol1933; 24(6): 417–441.
55.
MaJTangZLvF, et al. High-sensitivity ultrasonic guided wave monitoring of pipe defects using adaptive principal component analysis. Sensors2021; 21(19): 6640.
56.
RoweisSTSaulLK.Nonlinear dimensionality reduction by locally linear embedding. Science2000; 290(5500): 2323–2326.
57.
RoweisSSaulLHintonGE.Global Coordination of Local Linear Models. In: DietterichTBeckerSGhahramaniZ (eds.) Advances in Neural Information Processing Systems, British Columbia, Canada, vol. 14, pp. 889–896. Cambridge, Massachusetts: MIT Press, 2001.
58.
SaulLKRoweisST.Think globally, fit locally: unsupervised learning of nonlinear manifolds. Technical report, Technical Report MS CIS-02-18, University of Pennsylvania, 2003.
59.
PraetzelRClarkeTSchmidtD, et al. Monitoring the evolution of localized corrosion damage under composite repairs in pipes with guided waves. NDT E Int2021; 122: 102477.
60.
LiQLiuFLiP, et al. Acoustic data-driven framework for structural defect reconstruction: a manifold learning perspective. Eng Comput2024; 40(4): 2401–2424.
61.
MaatenLvdHintonG. Visualizing data using t-SNE. J Mach Learn Res2008; 9(86): 2579–2605.
62.
MaatenLvd. Learning a parametric embedding by preserving local structure. In: Proceedings of the twelfth international conference on artificial intelligence and statistics, Clearwater, Florida, pp. 384–391. ISSN: 1938-7228. PMLR.
63.
KobakDBerensP.The art of using t-SNE for single-cell transcriptomics. Nat Commun2019; 10(1): 5416.
64.
AgisDPozoF.A frequency-based approach for the detection and classification of structural changes using t-SNE. Sensors2019; 19(23): 5097.
65.
WattenbergMViégasFJohnsonI.How to use t-SNE effectively. Distill2016. Epub ahead of print October2016. DOI: 10.23915/distill.00002.
66.
GarrettRG.The chi-square plot: a tool for multivariate outlier recognition. J Geochem Exploration1989; 32(1–3): 319–341.
67.
Tibaduiza BurgosDAGomez VargasRCPedrazaC, et al. Damage identification in structural health monitoring: a brief review from its implementation to the use of data-driven applications. Sensors2020; 20(3): 733.
68.
DaviesJO.Inspection of pipes using low frequency focused guided waves. PhD Thesis, Department of Mechanical Engineering, Imperial College London, 2008.
69.
HalkoNMartinssonPGTroppJA.Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev2011; 53(2): 217–288.
70.
AbbassiARömgensNTritschelFF, et al. Evaluation of machine learning techniques for structural health monitoring using ultrasonic guided waves under varying temperature conditions. Struct Health Monit2023; 22(2): 1308–1325.
