Citizens’ appeals classification using a mixed method: A case study of Yichang government public service hotline data

Abstract

The study investigated the automatic classification of citizens’ appeals from the “12345” service hotline in a Chinese local government. The approach employed a mixed method with Large Language Models (LLMs) and word embeddings, specifically using ChatGlm2-6b and Moka Massive Mixed Embedding (M3E) respectively. Firstly, taxonomy was developed with Delphi method, and classification categories were determined by experts. The standard element library was built by incorporating weighting factors assigned to elements within each category. Secondly, ChatGlm2-6b was employed to extract the “topics” and “problems” from each new appeal. Next, word embeddings were utilized to compute the similarities between “topics”, “problems”, and standard elements. Finally, the categories of new appeals were determined with weighted voting. The results indicate that the top five categories of citizens’ appeals are “Housing”, “Daily life”, “Urban management”, “Transportation”, and “Employment”. The accuracy of this method is 0.83. The results can promote early warning systems for emergencies, mining and identification of citizens’ appeals, and supervision of social governance.