Harnessing AI for Educational Measurement: Standards and Emerging Frontiers

Abstract

The surge of AI in education raises concerns about measurement downsides. Calls for clear standards are warranted. Fortunately, the psychometrics field has a long history of developing relevant standards—like sample invariance and item bias avoidance—crucial for reliable, valid, and interpretable assessments. This established body of knowledge, not unlike traffic laws for self-driving cars, should guide AI assessment development. Measuring new constructs necessitates stronger construct validity research. Instead of rewriting the rulebook, our focus should be on educating AI developers about these standards. This commentary specifically addresses the concern of empowering instructors not with high-stakes testing but with effective item writing through AI. We explore the potential of AI to transform item development, a key area highlighted by researchers. While AI tools offer exciting possibilities for tackling educational challenges, equipping instructors to leverage them effectively remains paramount.