Topical Indexing and Questions to Represent Text for Retrieval and Browsing

Abstract

AI systems for text analysis and retrieval face a double-edged problem of knowledge representation and ergonomics. On the one hand, the content of a text must be explicitly represented, and at a level of abstraction that is plausible and descriptively useful. On the other hand, the effort to encode the knowledge must be in line with the benefits that that provides. There are no ideal solutions, but a good scheme would be one where the content of texts were usefully represented, accurate retrievals were easily made, and the knowledge encoding was naturally done. This article describes a methodology for representing texts in terms of the questions that are raised and answered by them: a natural and efficient way of abstracting and capturing knowledge. The questions arising from the texts are, in turn, classified according to a theory of topical indexing. This indexing scheme relates the questions to each other, and this enables the automatic generation of a prunable network of associated texts. This article details the question posing methodology and the underlying theory of indexing, using examples from an implemented prototype, TaxOps, a story-based advisor for tax consultants.