Local linear regression-based unsupervised truth discovery

Abstract

In the era of big data, the data provided by multiple sources for the same entity may result in conflicting information. Because sources with higher reliability degrees provide true information more frequently, the truth can be obtained by estimating the weights or reliability degrees of the sources. Due to hostile websites or faulty sensors, some sources may occasionally provide outliers that deviate significantly from the truth. However, in the majority of existing truth discovery methods, each source should be uniformly assigned a weight at the initial stage. Therefore, the accuracy of truth estimation is degraded by outliers. Several previous studies have proposed kernel density estimation-based truth discovery algorithms to solve the problems caused by outliers. These approaches aim to estimate the probability distribution of observation values to assess the reliability of sources. Unfortunately, the data with outliers that are smoothed by Gaussian kernels may cause more deviations from the truth. Thus, we propose a local linear regression (LLR) method for addressing the problems caused by outliers. The proposed method can effectively estimate the source reliability and the truth of the datasets with outliers. Experiments on two real-world datasets demonstrate that the proposed method yields more accurate results than existing state-of-the-art methods.