Machine Cleaning of Online Opinion Spam: Developing a Machine-Learning Algorithm for Detecting Deceptive Comments

Abstract

Humans are not very good at detecting deception. The problem is that there is currently no other particular way to distinguish fake opinions in a comments section than by resorting to poor human judgments. For years, most scholarly and industrial efforts have been directed at detecting fake consumer reviews of products or services. A technique for identifying deceptive opinions on social issues is largely underexplored and undeveloped. Inspired by the need for a reliable deceptive comment detection method, this study aims to develop an automated machine-learning technique capable of determining opinion trustworthiness in a comment section. In the process, we have created the first large-scale ground truth dataset consisting of 866 truthful and 869 deceptive comments on social issues. This is also one of the first attempts to detect comment deception in Asian languages (in Korean, specifically). The proposed machine-learning technique achieves nearly 81% accuracy in detecting untruthful opinions about social issues. This performance is quite consistent across issues and well beyond that of human judges.