Methodological Issues in the Evaluation of Intelligent Tutoring Systems

Abstract

Drill and practice, simulation, and tutorial are the main formats of computer-assisted instruction, yet tutorial format is particularly suitable to conceptual school subjects. When a knowledge module is added to the tutorial format, it is termed an intelligent tutoring system (ITS). Although ITS evaluation studies greatly influence interest in and support for the future ITS work, valid evaluation methodologies are lacking because ITS is fairly new. Consequently, this article examined the ITS evaluation studies, with the focus on the methodological issue. The critical component in ITS is the knowledge component that has not been evaluated adequately to verify the knowledge. Definitely summative evaluation is more difficult than formative evaluation since it involves the comparison of ITS with human tutors using traditional teaching methods across extensive subject domains.