Reduce or remove: Individual sensor reliability profiling and data cleaning

Abstract

Environmental sensing using multitudes of wirelessly connected sensors is becoming critical for resolving environmental problems, given recent technology advances in the Internet of Things (IoT). Current environmental sensing projects typically deploy commodity sensors, which are known to be unreliable and prone to produce noisy and erroneous data. Moreover, the majority of current sensor data cleaning techniques have not moved beyond using the mean or the median of spatially correlated readings, thus providing unsatisfying accuracies. In this paper, we propose a sensor reliability-based cleaning method, called Influence Mean (IM), which uses weighted aggregation based on individual sensor reliabilities. We investigate whether reducing or removing unreliable sensors can be more effective to provide accurate cleaning results, by designing and testing respective algorithms on synthetic and real datasets. The experimental results show that our method generally improves the data cleaning accuracy, particularly when the behaviors of unreliable sensors vary drastically from reliable sensors.