An Exploratory Study on Two Automated Item Generators for Generating L2 Reading Test Items

Abstract

Given the increasing interest in automated item generation in the second language assessment field, this study investigated the potential of two automated item generators for L2 reading assessment. The first generator, KR-Item-Generator, was developed by the authors, who used a free chatbot builder. The second, Q-Craft, was developed using GPT-4 API and employs an all-in-one method to generate questions and passages. A total of 83 pre-service teachers at a college of education in South Korea were asked to generate English reading passages and test items using both generators. They were then given a post-task survey on varying aspects of the two generators. The results of the study demonstrated that both generators were positively perceived regarding the naturalness of the sentences in the passages and the level of completion of the test items, although Q-Craft was rated significantly more positively in terms of the latter. Given these findings, we discuss the pedagogical implications and offer key directives for further L2 AIG research.