Validity of a self-report measure of student learning in active learning statics courses

Abstract

Although faculty-centered pedagogies are endemic across undergraduate science, technology, engineering, and mathematics education, there is increasing interest in active learning approaches. As discipline-based educational research in mechanical engineering continues to assess strategies for improving student learning and development, researchers need data collection tools that ameliorate issues of bias, minimize costs (e.g. time and student attention), and provide reliable data that has been validated within the disciplinary context. This study analyzes the validity and reliability of a commonly used survey, the Students’ Assessment of their Learning Gains (SALG). Data from seven Introduction to Statics courses at two universities were used to identify and confirm the latent constructs of the measure and to assess their reliability and criterion validity. Results demonstrated that four scales—active learning, concept knowledge and skills, self-efficacy, and feedback mechanisms—explain the majority of variation in the SALG survey in relation to the teaching and learning of statics. These scales were statistically validated and shown to accurately capture the criterion they represent. The primary advantage of the SALG is that it is less burdensome to students, who are only required to spend 10 to 15 min once at the end of the course to complete the survey, rather than spending more time with longer surveys or with those that require completion at multiple points in time. The tool is therefore also less disruptive to the class, which may make it more likely that faculty will be willing to include data collection efforts in their courses.