Sage Journals: Discover world-class research

Abstract

Current environmental challenges for water resources include guaranteeing good ecological status of water bodies, promoting sustainable water use and protection of water resources. A key aspect in the achievement of these objectives is the application of a consistent and efficient monitoring strategy. Implementation of continuous water quality measurement systems is allowing to capture the dynamics in water systems for identification of critical events, cause-effect relationships and trends among others. Huge amounts of data are then being generated with uncertain quality. Water quality monitoring networks will only be useful in practice if careful quality assessment, of the data is carried out. With a practical vision, this paper presents a method for automatic data quality assessment extracting information from individual water quality time series from on-line sensors. Data mining techniques based on forecasting models are used to detect and remove unreliable data from the raw data sets. A posterior analysis is applied to remove noise and detect abnormal situations and potential sensor faults. The proposed tool has been successfully tested on water quality time series collected from different water and wastewater systems.

Keywords

Data quality assessment forecasting techniques on-line water systems monitoring fault detection

Get full access to this article

View all access options for this article.

References

Alferes and

P.A.

Vanrolleghem, Automated data quality assessment: Dealing with faulty on-line water quality sensors, in: Proceedings 7th International Congress on Environmental Modelling and Software (iEMSs2014), San Diego, USA, 2014.

J.L.

Bertrand-Krajewski and

Muste, Data Validation: Principles and Implementation. Data Requirements for Integrated Urban Management, Unesco and Taylor & Francis Publishing, Paris, France, 2008.

Branisavljevic,

Prodanovic and

Pavlovic, Automatic, semi-automatic and manual validation of urban drainage data, Wat. Sci. Tech.62(5) (2010), 1013–1021. doi:10.2166/wst.2010.350.

Clement,

Thas,

Ottoy and

P.A.

Vanrolleghem, Data management of river water quality data: a semi-automatic procedure for data validation, Water Resources Research43(8) (2007).

Copp,

Belia,

Hübner,

Thron,

P.A.

Vanrolleghem and

Rieger, Towards the automation of water quality monitoring networks, in: Proc. 6th IEEE Conference on Automation Science and Engineering (CASE 2010), Toronto, Canada, 2010.

Dochain and

P.A.

Vanrolleghem, Dynamical Modelling and Estimation in Wastewater Treatment Processes, IWA Publishing, London, UK, 2001.

Garcia,

Quebedo,

Puig and

M.A.

Cuguero, Sensor data validation and reconstruction in water networks: A methodology and software implementation, in: 9th International Conference CRITIS, Limassol, Cyprus, 2014.

Garcia,

Pedregal and

Roberts, Time series methods applied to failure prediction and detection, Reliability Engineering & System Safety95(6) (2010), 698–703. doi:10.1016/j.ress.2009.10.009.

D.J.

Hill and

B.S.

Minsker, Anomaly detection in streaming environmental sensor data: A data-driven modeling approach, Environmental Modelling & Software25 (2010), 1014–1022. doi:10.1016/j.envsoft.2009.08.010.

10.

Kiruluta,

Eizenman and

Pasupathy, Predictive head movement tracking using a Kalman filter, Systems, Man, and Cybernetics, IEEE Transactions27(2) (1997), 326–331. doi:10.1109/3477.558841.

11.

Maimon and

Rockach, Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers, Springer, London, UK, 2010.

12.

Metadier and

J.L.

Bertrand-Krajewski, The use of long-term on-line turbidity measurements for the calculation of urban stormwater pollutant concentrations, loads, pollutographs and intra-event fluxes, Wat. Res.46 (2012), 6836–6856. doi:10.1016/j.watres.2011.12.030.

13.

Mourad and

J.L.

Bertrand-Kralewski, A method for automatic validation of long time series of data in urban hydrology, Water Sci. Technol.45(4–5) (2002), 263–270.

14.

Rieger and

P.A.

Vanrolleghem, monEAU: A platform for water quality monitoring networks, Water Sci. Technol.57(7) (2008), 1079–1086. doi:10.2166/wst.2008.135.

15.

Sandoval and

J.L.

Bertrand-Krajewski, Identification of errors in high temporal resolution rainfall time series by model based approaches, in: Proceedings of the 10th UDM – International Conference on Urban Drainage Modelling, Mont Sainte Anne, Quebec, Canada, 2015.

16.

M.G.

Schimek, Smoothing and Regression, Approaches, Computation and Application, John Wiley & Sons, New York, USA, 2013.

17.

Sharma,

Golubchik and

Govindan, Self-organizing systems, in: 7th IFIPTC International Workshop IWSOS, Palma de Mallorca, Spain, May 2013, 2013.

18.

Takahama and

Sakai, A comparative study on kernel smoothers in differential evolution with estimated comparison method for reducing function evaluations, in: Proceedings of the Eleventh Conference on Congress on Evolutionary Computation, Trondheim, Norway, May 2009, 2009.

19.

Taylor, Triple seasonal methods for short-term electricity demand forecasting, European Journal of Operational Research204 (2010), 139–152. doi:10.1016/j.ejor.2009.10.003.

20.

Thomann, Quality evaluation methods for wastewater treatment plant data, Water Sci. Technol.57(10) (2008), 1601–1609. doi:10.2166/wst.2008.151.

21.

B.F.

Thomas, Exponential Smoothing Models. Lecture, Department of Economics Southern Methodist University Dallas, 2008.

22.

J.L.

Viola, Double exponential smoothing: An alternative to Kalman filter-based predictive tracking, in: Proc. Workshop on Virtual Environments, Zurich, Switzerland, May 2003, 2003.

23.

Wagner,

Boulger,

Oblinger and

Smith, Guidelines and Standard Procedures for Continuous Water-Quality Monitors: Station Operation, Record Computation, and Data Reporting, U.S. Geological Survey, Virginia, USA, 2006.

24.

C.K.

Yoo,

Villez,

S.W.V.

Hulle and

P.A.

Vanrolleghem, Enhanced process monitoring for wastewater treatment systems, Envirometrics19(6) (2008), 602–617. doi:10.1002/env.900.

Efficient automated quality assessment: Dealing with faulty on-line water quality sensors

Abstract

Keywords

Get full access to this article

References