Effects of Generative AI Feedback and Interactive Video Assessment on Student Learning Achievement in Philological Content Creation Courses

Abstract

Despite the increasing integration of generative Artificial Intelligence (GenAI) in education, a significant research gap exists in its comparative effectiveness against traditional instructor-led guidance in specialized humanities like philology. This quasi-experimental study compares an experimental group (EG, n = 26), using GenAI chatbots for conceptual understanding and GenAI platforms to create interactive materials from classical texts, with a control group (CG, n = 26) receiving standard instruction. This study investigates the impact of GenAI-assisted instructional design on students’ learning achievements and empowerment in using GenAI within real-world contexts. It specifically examines if there is a statistically significant difference (a) in philological content mastery—defined as understanding classical or modern texts/themes—and (b) in perceived empowerment in using GenAI (self-efficacy, impact, and meaningfulness). The findings indicate that the EG significantly outperformed the CG in mastering philological content, with a notably higher adjusted post-test mean score and a large effect size. The same group also reported substantially greater perceived empowerment in using GenAI within real-world contexts, demonstrating strong increases in self-efficacy, impact, and meaningfulness compared to their counterparts in the CG. Implications suggest integrating GenAI into philology curricula foster engagement, autonomy, and confidence in creative tasks, provided educators mitigate risks to analytical rigor through intentional design.