Text mining and topic modelling in English teaching: Extracting key themes and concepts for effective curriculum development

Abstract

Nowadays, English teachers have more options to improve curriculum development through the rapid rise of textual data from feedback, student interactions, and educational resources. This study introduces a novel Intelligent Mined Text-Hierarchical Dirichlet Process Modelling + Quantitative Term-Frequency Matrix (IMT-HDPM + QTFM) methodology that effectively integrates topic modeling and text mining techniques to enhance curriculum development to refine understanding of key English language instruction themes and concepts. This study proposes a methodology that enables teachers to improve curriculum and teaching methods through in-depth textual analysis. The dataset was gathered from various educational sources, including lessons, essays, and exercises, categorized by difficulty level (Easy, Medium, and Hard), with each entry containing a brief text sample, source, and category. The IMTs are text cleansing, tokenization, stop word removal, and stemming/lemmatization, which are used for removing punctuation, numbers, special characters, and common words from the text sample to build cleaned text data. The HDPM provides a flexible probabilistic framework that enhances the capture of topic distributions across cleaned text sample documents, leading to more accurate identification of meaningful topics. QTFM examines how topics relate to each other in the text data. The IMT-HDPM + QTFM resulted in the most accurate topic identification and significantly improved the understanding of key themes and concepts in English language instruction texts and then reduced the time spent on manual curriculum refinement. The evaluation metrics included accuracy (94%), recall (95%), and precision (96%), indicating the robustness of the proposed methodology. The IMT-HDPM + QTFM novel analyzes unstructured textual data from blogs, social networks, and forums; important trends and insights can be extracted from a variety of online content, improving comprehension across platforms.