Can Human Reading Validate a Topic Model?

Abstract

Validation is at the heart of methodological discussions about topic modeling. The authors argue that validation based on human reading hinges on distinctive words and readers’ labeling of a topic, and it overlooks the probability of conflicting results from semantically similar models, such as regressions or other methods. This runs counter to the presumption that topic modeling can reveal features of documents that have some measurable association with social aspects outside the text. The authors develop a similar topic identifying procedure to verify that semantically similar solutions yield similar results in further analysis. The authors argue that future validations of topic modeling must consider such procedures.