Abstract
Scaffolding learning and incorporating hands-on activities are proven pedagogical techniques that improve student engagement. A growing movement within engineering education has been to educate beyond the technical skills and foster an improvement in students’ entrepreneurial mindset. A modified version of a validated self-efficacy tool is used here to assess the impact that scaffolding and hands-on activities within a Mechatronics course have on developing the three student learning outcomes associated with the Kern Entrepreneurial Engineering Network's entrepreneurial mindset: “curiosity,” “connections,” and “creating value” (3Cs). A total of 229 students were considered in this study. Presurveys and postsurveys were used to collect self-efficacy ratings for confidence, success, motivation, and anxiety with respect to each of the 3Cs. Additionally, for comparisons a baseline behavioral tool was assessed along with students self-reported prior experiences relevant to the course and overall time spent on course activities. The group was split into high-performers and low-performers for the purpose of analysis. For all 3Cs, aspects of student perceived confidence and success significantly increased between the presurveys and postsurveys for the entire group. However, student perceived motivation was not significantly improved for any of the 3Cs. Interestingly for “connections,” student perceived anxiety decreased only for the high-performing group. This study suggests that changes in these 3Cs can be identified within a single course when subaspects of the mindset are considered.
Introduction
There is a growing emphasis on imparting skills beyond technical knowledge in engineering education. Particularly, there is a focus on nurturing what is known as the entrepreneurial mindset (EM) within engineering students. EM is characterized by three key attributes: “curiosity,” “connections,” and “creating value” (3Cs). 1 The Kern Entrepreneurial Engineering Network (KEEN) is a driving force behind this movement and comprises of more than 5,200 faculty and staff from 380 institutions, all dedicated to the mission of instilling entrepreneurial mindset in undergraduate engineering students. The aim is to “equip the students with the capacity to generate personal, economic, and societal value throughout their careers.”
To accomplish this mission, KEEN offers various methods for educators to integrate EM into their curricula and provides a platform for members to share their findings through KEEN's online repository. While KEEN has played a pivotal role in advancing EM within engineering education, the development of validated assessment tools to measure progress in the 3Cs are still in its nascent stages, as evident by recent works.2,3
The 3Cs within the framework of EM are considered behaviors that students can learn and develop. Further, educators are encouraged to guide students in these areas and assess progress to inform EM training methods. In the context of the first learning outcome “curiosity,” KEEN states that “for engineers to succeed in a world with rapidly changing needs and tools, they need a sense of curiosity.” KEEN advocates for the use of techniques like brainstorming, question-formulation, and storytelling to foster curiosity in engineering students. Assessments in this domain include validated tools, based on interpreting motivation 4 and classifications of curiosity scales. 5
The second student learning outcome focuses on the necessity of interdisciplinary “connections.” KEEN states, “interdisciplinary connection-making is essential to the advancement of knowledge.” Here, KEEN promotes techniques such as concept mapping, problem-based learning, and innovation inspired by nature. A common assessment tool used in this context is one that interprets the connectivity within concept mapping. 6
The third student outcome underscores the importance of teaching students how to “create value.” KEEN states that “teaching your students the importance of creating value helps redirect their mindset and motivation—leading to more impactful engineering solutions.” Techniques endorsed by KEEN to develop “creating value” skills include exploring success through case studies, evaluating the impact of biases, and encouraging students to define their own projects in capstone courses. However, validated assessment tools for evaluating progress in this area appear to be less prevalent within the educational community.
Within the framework of entrepreneurial mindset, the 3C's can be considered as learning outcomes that can help characterize the effectiveness of a teaching method in developing the entrepreneurial mindset.
“Curiosity”—“Demonstrate curiosity about the changing world”
1
“Connections”—“Integrate information from many sources to gain insight”
1
“Creating value”—“Identify unexpected opportunity to create value”
1
While the 3Cs certainly have numerous educational techniques aimed at advancing each, additional assessments via validated tools would appear to be welcomed. Here we introduce the usage of an engineering design self-efficacy tool
7
as a means to assess self-efficacy attributes across all 3Cs. Via modifying the wording within this questionnaire, one can tailor to specific learning aspects within a course and establish a relation for each question to at least one of the 3Cs. Here we use this tool to assess a course that implements the combination of two best practices: (1) scaffold learning8–11 and (2) hands-on learning12,13 via applied assignments of increasing complexity. In this study scaffolding refers to a pedagogical approach in which students work on assignments of increasing difficulty or complexity while the level of guidance or support provided is gradually reduced. Learning activities that require physically building, integrating, and testing systems (mechanical-electrical-programming in this case) are referred to here as hands-on applied assignments or hands-on applied content. Other papers may refer to the same activities using language like: experimentation, practical, practicum, hands-on, and making. The aim was to extract the impact that the combination of these practices has on developing students’ EM.
Positionality statement
This study was partially funded by the KEEN network, and the effects of scaffolding and applied assignments on the entrepreneurial mindset were evaluated within a framework that was established by KEEN. This could introduce some bias. However, the authors have made every effort to connect the findings, discussion, and recommendations to the general literature on pedagogy.
Two of the authors, who are tenure-track professors, are actively involved in a curriculum reform committee within the university and are looking for ways to improve student engagement, learning, and learning experiences. The course was planned and conducted by the two tenure-track professors. The analysis was conducted by the other authors, who were not directly invested in the course. The other authors include a lecturer, a post doc, and a Ph.D. student. These unique identities help strengthen the experimental design and analysis. Two authors are women, two other authors are immigrants, and all authors acknowledge the importance of good-quality education for all.
Methods
Teaching methods
Here we provide an overview of the course selected for this study, Mechatronics. Mechatronics is a mandatory component of our mechanical engineering curriculum, designated as occurring in the second year of study. Within the course, each class is divided into sections of no more than 24 students each, so as to provide one instructor and 2 teaching assistants per class for an effective student to teacher ratio with respect to guiding learning activities. Each class meets for 1 hour and 15 minutes, twice every week and offers numerous additional office hours via combining the course resources across all the classes. The course is considered predominantly a flipped classroom. Instructor developed online course materials include: videos of lectures covering theoretical concepts and problem-solving examples, videos of demonstrations covering hands-on applied concepts and circuit building examples, and videos describing sample coding. Additional course materials also include instructor developed assignment guides, curated notes from other sources, news articles for relevant real-world applications, and an instructor curated kit of electronic components and tools that each student purchases. This student kit (Table 1) includes a commercial Arduino kit (Elegoo Super Starter Kit) and a customized set of electronics that is used throughout the course. The intention of the student kit is to allow instructor flexibility with respect to assignments for each iterations of the course, encourage students to engage beyond many traditional laboratory experiences by working at their own pace both inside and outside of the classroom, and practice scaffolding their knowledge with more hands-on applied and open-ended experiences. As such, the course's content naturally demands students to forge connections and integrate knowledge across mechanical, electrical, and computer science disciplines.
Content of the mechatronics student kit.
The course begins with the students learning fundamental engineering principles and applying them through their student kits with practical activities. They construct basic circuits, make measurements, and engage in troubleshooting exercises. As the course progresses, they move on to incorporating sensors and actuators into their microprocessors (Arduino Uno) to create programmable circuits, with the explicit aim of connecting these experiences to their mechanical engineering major. Utilizing a microprocessor with an open-source platform, both instructors and students have access to resources for hardware and programming instruction, enabling the acquisition of technical skills and inspiration from a wide variety of easily accessible online content.
One notable feature of the student kit is the inclusion of a laser-cut wooden chassis, a component typically designed by previous students and produced by our university's makerspace. The “making” of such a chassis demonstrates direct educational value of our makerspace within one of our required courses, reduces component kit cost over off-the-shelf options, and allows for future customization of hands-on applied assignments in the course. As such, the chassis becomes a physical platform for our mechanical engineering students to scaffold their knowledge to construct various custom, programmable robots. With the multitude of sensors and actuators in the kit, the robot's potential is virtually limitless, allowing the instructor to introduce diverse hands-on exercises and project objectives from 1 year to the next. For instance, in the context described, students developed an automated agricultural robot showcasing line following and depth detection. In previous iterations of the course, students have created sumo bots, interactive games, and drawing robots using the same kits.
For more insight to the course logistics, apart from the 2.5 class hours each week, the students had the option of receiving help from the course's overall two instructors and seven to eight teaching assistants during their office hours. In total, 12 hours of office hours were open to the students every week. These 12 hours of office hours were intended to account for variability in student schedules and encourage them to work on their assignments regularly while having the option to receive guidance. In the past, the course alternated between theory-based circuit analysis and hands-on applied assignments every few weeks. However, this approach seemed to result in uneven progress in students’ skills in both areas and fewer connections between theoretical knowledge and practical application. In the current implementation, we've evenly distributed class time between theory and hands-on applied assignments. This approach allows students to work on both theory-based assignments and practical assignments each week, with a reduction in the number of problems or requirements for each assignment to facilitate effective engagement in both aspects of the course.
Scaffolding of hands-on applied content
Each hands-on applied assignment introduced students to new concepts in coding and electrical engineering, as well as new mechanical equipment. See Table 2 for an outline of all the hands-on applied assignments. As noted in the first section, exploring success through cases studies is a KEEN endorsed technique to develop “creating value” skills, which are also closely tied to motivation. To provide motivation and establish connections to products, each hands-on applied assignment was associated with a relevant product example (Figure 1). These examples ranged from straightforward products like a “light switch” to more complex concepts such as a “self-parking car.” In the “light switch” hands-on applied assignment, the students used an Arduino board, an LED and a button switch to build a circuit that represents a light switch. The light (LED) turns on when the button is pressed. These product examples served a dual purpose: reinforce students’ comprehension of the required performance of each system and creating connections between their learning and real-world applications.
Example of a hands-on applied assignment (Table 2: see #5 Reflectance sensing, Robot Chassis, LED).
Outline of all the hands-on applied assignments.
The typical approach for students was to begin by acquiring background knowledge about new electrical engineering concepts and the corresponding mechanical devices. To help students relate different styles of diagrams, students were provided with both electrical circuit-style diagrams and the associated “Fritzing” style diagrams (Figure 2) for each circuit they needed to create. 14 Students utilized these diagrams to build their circuits using solderless breadboards. Each hands-on applied assignment included a code outline that usually required making 10–20 modifications to the code to achieve the desired system performance. The course's coding instruction was scaffolded, commencing with simple IF statements in assignment number 2 and progressing to IF…ELSE statements with up to 5 branches in assignment numbers 7 and 8. Logic conditions were also scaffolded, starting with equality assessments (==), and moving to the use of inequalities (>= and <=), culminating in conditions using Boolean logic (&&) to test multiple requirements. Instructors provided support at each stage through discussions with the students, diagrams, and tables. Each assignment mandated that the system demonstrates specific requirements.
Example of a “Fritzing,” pictorial, style diagram (Table 2: see #2 Buttons, LED, UNO).
The final project in the course focused on students’ ability to integrate the multiple systems they had created into a single mechatronic product (Figure 3). In this context, the final project centered on an autonomous agricultural robot capable of following designated paths in a simulated “field” and distinguishing between “plants” and “weeds” (represented by wooden blocks of varying dimensions), subsequently performing the correct actions of “watering” or “weeding” (indicated by the appropriate LED color and position of the mechanical servo motor arm). Students were tasked with reading news articles and watching videos to understand the potential applications of such robots, including weeding, plowing, and harvesting.15–17 This allowed students to identify and discuss the stakeholders involved and the potential value creation aspects of agricultural robots, including considerations related to human rights and environmental impact.
A representative final project done by a student (shown here following the line and correctly identifying with a blue LED attached to a mechanized boom to “water a plant”).
Scaffolding of theoretical content
Each of the theory assignments utilized scaffolding techniques (Table 3) but also made specific efforts to keep the layout of theory assignment sheets consistent. As a course designated for the second year, students in Mechatronics are still considered to be developing their system identification and problem-solving strategies, especially relevant with this often being one of their first engineering courses within their major. For all theory problems the layout always started with system identification questions that asked students to determine “What type of circuit am I solving?”, “What type of equation do I expect for that type?”, “What is the general form for that type of equation?”, et cetera. The goal was to give students a system for approaching circuit problems and confidence that they could solve a given type of circuit, even if it was slightly different from prior practice problems. New components, beyond resistors in resistor networks, such as capacitors and inductors were introduced over time and the instruction (along with the theory assignment sheet layout) emphasized that the same principles (e.g.Kirchhoff's Voltage Law and Ohm's Law) would still be applied the same as the approach for simpler circuits. For exam problems the layout was identical to theory assignment sheets so that students would feel as comfortable as possible to demonstrate their knowledge under exam conditions but using a framework they were used to. Students were very positive about these layouts and their consistency between homework and exams.
Outline of all the theory assignments.
As theoretical circuit concepts and components were scaffolded, a more subtle course development goal was to facilitate student ability to connect the mathematical foundations (e.g. first and second order differential equations, complex numbers) present in circuit analysis with future use of the same math in core Mechanical engineering courses such as heat transfer and system dynamics. As circuit complexity was scaffolded instructors explained the analogous mechanical components and systems to give students a basis for connecting their circuit behavior outcomes with physical systems that they would have more intuition or experience with. For example, a resistor-capacitor circuit behaves like a spring-damper system, going to some steady state when a fixed force is applied. For the most complex circuits addressed in the course (resistor-capacitor-inductor systems) students brainstormed physical mass-spring-damper systems and also categorized which behaved in underdamped (e.g. bungee jumping), overdamped (e.g. door/drawer soft closer), or critically damped (e.g. robotic arm) response patterns.
Beyond circuit analysis theory, the theoretical foundations of programming and logic were used to motivate and scaffold approaches to hands-on applied assignments (Table 2, Code Concept column). Early hands-on applied assignments required students to build and test the exact circuits that they were analyzing in theory. Comparing hands-on (multimeter) measurements of circuit voltages, resistances, and currents to their theoretical values clearly resulted in many “eureka” moments of students realizing that the circuit was the same, just investigated by two different methods. As the circuits in the hands-on applied assignments became too complex to analyze theoretically, the fact that many sensors are just variable resistors was emphasized (Table 2, Electrical Concept). In these ways and others, the pedagogical approach of scaffolding and the 3C's were integrated into the theory focused portions of the course.
Course assessment methods
The hands-on applied assignments were typically assessed on a binary scale, meaning they were evaluated as either meeting the requested performance criteria or not. Instructors used the phrase “Are you ready to check off?” to distinguish between receiving general support and a full demonstration. While the requirements for being “checked off” as complete were fixed, students were allowed as many attempts as needed to showcase their system's performance. Each attempt was accompanied by formative feedback, highlighting any unmet requirements. The approach aligns with EM goals: encouraging self-evaluation, learning from mistakes and taking initiative. In the initial stages of engineering training, students often accept the idea of partial credit or effort as satisfactory. However, in industry, engineers must ensure that all performance requirements are fully met before presenting a system to stakeholders, investors, or customers. During early training, supervisors or peers provide low-stakes feedback to ensure that partially completed systems are not presented outside of the working group. The binary scale assessment encourages students to swiftly learn from their mistakes in a low-risk environment and motivates the students to take action and fulfill all requirements. Over the course, students appeared to get better at “self-assessing” or correctly determining if they were ready to demonstrate or still needed support.
Another aspect of assessment designed to mimic industry experience was the late policy. Given the binary scoring system, a highly flexible late policy was implemented, offering 75% credit for assignments submitted up to one week late and 50% credit for submissions up to 2 weeks late. Instructors emphasized the importance of completing the assignment, even if it couldn’t be done on time. In industry, deliverable requirements don’t disappear when a deadline passes, but engineers receive diminishing recognition for achieving them as time goes on.
One aspect of the entrepreneurial mindset not addressed by a binary scoring system is the ability of engineers to identify opportunities and create value. For certain hands-on applied assignments, optional bonus performance challenges were introduced. In the final project, a points system with potential extra credit was implemented. Students had various ways to score points, with a maximum of about 150 points possible, but the final scores were capped at 110 out of 100. The majority of points, around 70, could be obtained by integrating prior work and demonstrating robot performance within a continuous 3-minute time frame. To score beyond this, students had to apply their prior knowledge to slightly different new equipment, such as using a positional servo instead of a continuous rotation servo or employ slightly more advanced coding techniques. Although the demonstration time frame remained fixed, students had the opportunity to demonstrate multiple times to enhance their scores.
With regard to theory assignments, they were provided as paper and digital copies. This typically includes 6 problems/circuits for the students to practice theoretical concepts. Theory assignments were accepted up to 24 hours late and assessed with a 15% penalty. After 24 hours solutions were posted, and assignments could only be submitted for redo credit. Redos allowed the students to correct any graded problems that they did not score full credit on. Any corrected points submitted for redo were awarded 25% credit and added to the Theory score. Redos were typically due 1 week after the grades were posted. As opposed to the hands-on applied assignments, this more traditional grading approach was used for theoretical assignments to externally motivate the students to learn the theoretical concepts. 18 The redos were allowed to provide parallel feedback and to emphasize learning over grades.
Three exams were conducted over the semester to assess student learning. The exams covered any prior material and included both theoretical and hands-on applied content. Typically, an exam was conducted after every third theory and hands-on applied assignment. Exams were structured such that the weighting was approximately 70% theoretical and 30% hands-on applied content. Given the inherent time constraints of an exam, problems with respect to hands-on applied were limited to predominantly coding questions and identifying or describing solutions based on pictures of physical setups. Redo policies existed for the exams as well and were the same as the theory assessments. Redos were available up to 1 week after the scored exam was returned; any corrected problems submitted as a redo was awarded 25% credit and added to the exam score.
The final grade was calculated as the weighted cumulative average of all the individual assessments. Theory (12%), Applied (13.5%), Exam 1 (16%), Exam 2 (16%), Exam 3 (18%), Final Project (18%), and Reflections (6.5%) add up to a 100% of the final grade. The reflections included written submissions about articles that showcase the broader societal implications of mechatronic systems. All assignments were completed individually by students, and all students were assessed individually.
In terms of final assessment, each student with a final percentage ≥93.00% was assigned the highest possible letter grade. This was noted as a letter grade of A and a GPA numerical score of 4.00 out of a 4.00 scale. Across all students included in this study, the average grade was 92.7%, and the median grade was 94.2%. This could be viewed as unexpectedly high as many courses might anticipate an average in the B range. This can be attributed to the course approach outlined above, which included opportunities for recovering missed points (redos), a moderate late acceptance policy, and the opportunity in the final project to score more than 100%. Additionally, it should be noted that 20 students were excluded from the study, due to noncompletion of both the presurveys and postsurveys, were also students that were generally lower performing overall and if included, would have yielded a class average of 91.3%.
3Cs assessment method
Across 4 semesters, 249 students took the course. A total of 229 students who completed both the presurveys and postsurveys were included for this study. The study protocol was approved by the University's Institutional Review Board.
The presurveys and postsurveys included a mechatronics-modified version of Carberry et al.'s engineering design self-efficacy tool, 7 which consisted of 4 similar sections of 12 questions each. The presurveys were conducted in the first week of class and after defining to the students what the field of Mechatronics is. The postsurveys were conducted during the last or second last week of class during the students’ final project efforts. Each question allowed a response rating of 1 to 10 using a Likert Scale. Each section represented one of four self-efficacy subsections: “confidence,” “motivation,” “success,” and “anxiety.” The 12 questions in these 4 subsections remained the same, except for the first word, which was changed to represent the section.
From the survey data, students self-perceived EM development for each of the 3Cs, as defined by KEEN, were measured. The first student learning outcome “curiosity” was determined as the average of the ratings on questions 1, 2, and 8. The second student learning outcome “connections” was determined as the average of the ratings on questions 3, 4, 6, 9, 10, and 11. And the third student learning outcome “creating value” was determined as the average of the ratings on questions 5, 7, and 12 in each category (Table 4). As such, each analyzed subattribute had three or more survey questions relate to the same attribute, or each of the 3Cs, and thereby allowed for statistical measures to be used to measure each.19–21
Example of survey questions in the confidence rating section, grouped into the learning outcomes of KEEN's entrepreneurial mindset: “curiosity,” “connections,” and “creating value” (3Cs).
KEEN: Kern Entrepreneurial Engineering Network.
To present a baseline for future comparisons of these results, the presurvey also included demographics information, questions on relevant prior experiences and a five-dimensional behavioral score, as described by Kashdan et al. The latter was performed with the original, or unmodified, tool. 5 It was used to obtain a general representation of the students’ view of themselves and provide a means for correlations between performance in this course and the overall behavioral tendencies of this analyzed group of students. Lastly, the postsurvey also included students self-reported information on time that they individually spent on assignments within the course.
Statistical analyses were performed using NCSS statistical software (NCSS, LCC, Kaysville, UT, USA). Plots were created in MATLAB (The MathWorks®, Natick, MA, USA) and edited in Inkscape (The Inkscape Project, Boston, MA, USA). The Wilcoxon Signed Rank test was used to compare the learning outcomes of KEEN's EM: “curiosity,” “connections,” and “creating value” (3Cs) ratings between the presurveys and postsurveys. Further, the differences between the presurveys and postsurvey results of the 3Cs were also analyzed separately within the four self-efficacy subsections: “confidence,” “motivation,” “success,” and “anxiety” using Wilcoxon Signed Rank test.
Pearson correlation was performed to determine whether time spent per week in completing theoretical and hands-on applied assignments were correlated to prior experiences with programming courses or electronics/mechatronics/robotics courses. Additionally, Pearson correlation was performed to analyze the relationship between behavioral tendencies of the students and performance in class. Reception of students to scaffolding was quantified by using a paired difference comparison of the student scores in the first portion of the class to the last portion of the class.
Significance was determined at p < 0.05. The changes in reported outcomes of EM were also separately quantified for high performers and low performers. The students who achieved the highest possible letter grade (>93% as defined in the Course assessment methods section) were grouped as high performers, and the remaining students were grouped as low performers. Knowing the cutoff for achieving the highest letter grade, and being able to constantly monitor grades online, students were aware if their performance in the course was falling below what was required to attain the maximum numerical (GPA) grade. This creates a logical argument for this cutoff between groups: those who knowingly were attaining the maximum possible grade and those who knew that their grade could be further improved (see also the Measuring developing self-efficacy for the 3Cs section). 22 The separate analysis of low-performing students is to quantify the effects of scaffolding and hands-on applied assignments on EM of the low performers. This could help infer whether scaffolded hands-on applied assignments could improve the performance and EM mindset of even low performers.
Results
Demographics
A total of 229 (181 males, 46 females, and 2 others) students who completed both the presurveys and postsurveys were included for this analysis. The demographics of the analyzed set of students are represented in Figure 4. The group consisted of students with college status of 153 sophomores, 70 juniors, 4 seniors, and 2 first-year students in the undergraduate mechanical engineering program. In the survey, 201 students identified as Caucasian, 7 as African American, 7 as Asian, 3 as Middle Eastern, 2 as Native American, 1 as Native Hawaiian, and 8 students did not disclose their race in the survey. Eight students in the group considered themselves as having Hispanic origins. This information is provided for population characterization purposes. Due to the small number of non-Caucasian and Hispanic students it is not possible to discern if race or ethnicity plays a significant role in student outcomes.
Demographic information.
Prior experiences and class performance
As shown in Table 5, the presurvey indicated that 137 (59.8%) of the students had never taken a course within electronics, mechatronics, or robotics before, and 142 (62%) of the students had never been in a robotics or STEM-related club. However, most students (96%) had taken at least one course in programming. This last result was expected though, as most students in our Mechanical Engineering curriculum would have taken a first-year programming class as a prerequisite to this course. Additionally, while not a prerequisite to this course, our Mechanical Engineering curriculum also includes a required second year programming course, which explains the large number of students indicating two or more courses in programming. However, it should be noted that four sophomores and five juniors indicated no prior experience with programming.
Prior experience in mechatronics and robotics.
Within this Mechatronics course, the final project score was considered as a cumulative measure of student learning. Analyzing student assessments with results from Table 5, there was no correlation between final project score or final grade and experience in electronics/mechatronics/robotics or prior involvement in robotics/STEM clubs. However, there was a positive correlation between performance in the final project and programming experience (Pearson correlation = 0.1304, p-value = 0.0487). Similarly, there was a positive correlation between the final grade and programming experience (Pearson correlation = 0.1550, p-value = 0.0189).
Recalling from the Course assessment methods and 3Cs assessment method sections, across all students included in this study, the average grade was 92.7%, and the median grade was 94.2%. Using the highest possible letter grade to differentiate two groups within the course, there were 148 students in the high performers group and 81 students in the low-performers group. The low performers had an average final grade of 86.1 ± 7.1%, and the high performers had an average final grade of 96.3 ± 2.1%. Even though using the median would have resulted in an even split of students in each group, we consider that the difference between a final grade of 93% and 94.2% is inconsequential, as both students end up with the same letter grade and students were able to constantly monitor grades online.
Details for the average scores of all the graded assessments are shown in Figure 5. There were no significant differences in the scores for theory assignments between the first portion of the course (Theory #1 to Theory #4 compared to Theory #6 to Theory #9) and the last portion. The scores in the hands-on applied assignments were significantly higher in the last portion of the course (Hands-On Applied #1 to Hands-On Applied #4 compared to Hands-On Applied #6 to Hands-On Applied #9), and the scores in the Exam #3 were significantly lower than the scores in Exam #1.
Average scores in all graded assignments.
Investigating student reported time spent on assignments, for the whole class there was no correlation between the time spent on hands-on applied assignments and the final grade (Pearson correlation = −0.1491, p-value = 0.241). However, there was a negative correlation between the time spent on theory assignments and the final grade. There were no correlations between the time spent on assignments and the final project score for the whole class. When evaluating based on performance, on average the low performers spent more time on hands-on applied (2.52 ± 1.33 compared to 2.31 ± 1.23 hours for the high performers) and theory assignments (2.46 ± 1.09 compared to 2.15 ± 0.85 hours for the high performers). When considered alone, the high performers showed a positive correlation between time spent on theory assignments and performance in the final project (Pearson correlation = 0.2027, p-value = 0.0135). The same trend did not exist for the low-performers group (Figure 6).
Average hours per week spent on theory and hands-on applied assignments.
Measuring developing self-efficacy for the 3Cs
Analyzing the presurveys and postsurveys of the self-efficacy surveys, the confidence rating and success rating significantly increased for the class as a whole for all 3Cs. The anxiety rating for the EM learning outcome of “connections” decreased significantly for just the high-performance students (Table 6). The 3Cs within the anxiety rating were not correlated to any other 3Cs within the considered self-efficacy metrics for the low-performance group. However, the 3Cs within the anxiety rating was inversely correlated to the 3Cs within the confidence rating and success rating for both the high-performers group and the class as a whole. This is true except for one comparison: the EM learning outcome of “curiosity” within the anxiety self-efficacy was not correlated to the EM learning outcome of “creating value” within the success self-efficacy for the high-performers group.
Presurvey and postsurvey results showing the self-reported rating in different self-efficacy categories, divided into the 3Cs.
The bolded results show significant differences between the presurvey and postsurvey ratings.
The 3Cs were strongly correlated (Pearson correlation > 0.8) to each other within each self-efficacy metrics. The 3Cs within the confidence rating was moderately correlated to the 3Cs within the success rating (Pearson correlation between 0.45 and 0.6, p-value < 0.001). The 3Cs within the motivation rating was weakly correlated to the 3Cs within the confidence rating (Pearson correlation between 0.17 and 0.27, p-value < 0.02). This is true except for one comparison: the EM learning outcome of “curiosity” within the confidence self-efficacy was not correlated to the EM learning outcome of “connections” within the motivation self-efficacy for the low performers group. Similarly, the 3Cs within the motivation rating was also weakly correlated to the 3Cs within the success rating (Pearson correlation between 0.27 and 0.44).
Comparison to behavioral scale
To translate these results with respect to potential future comparisons, analysis with respect to the unmodified 5-dimensional behavioral scale, 5 as conducted within the presurvey, was performed. Specifically, students’ view of themselves with respect to behavioral tendencies associated with the five-dimensions of “joyous exploration, deprivation sensitivity, stress tolerance, social curiosity, and thrill seeking” are presented for this group of students in this study, see Table 7. Across the whole class, the final grade (Pearson correlation = −0.2061, p-value = 0.0062), average theory grade (Pearson correlation = −0.1805, p-value = 0.0062), and average hands-on applied assignment grades (Pearson correlation = −0.1728, p-value = 0.0088) were negatively correlated to the thrill seeking behavioral dimension. The exam average was positively correlated with the stress tolerance behavioral dimension (Pearson correlation = 0.1477, p-value = 0.0254).
Self-reported scores on the five-dimensional behavioral scale within the high-performance and the low-performance groups.
There were no significant differences between the two groups within any of the five considered dimensions in this behavioral scale. With that said, within the low performers the exam average was positively correlated to the stress tolerance behavioral dimension (Pearson correlation = 0.1619, p-value = 0.049), and average hands-on applied assignment score was negatively correlated to the thrill seeking behavioral dimension (Pearson correlation = −0.1944, p-value = 0.0179).
Within the high performers, the average hands-on applied assignment score was negatively correlated to the thrill seeking behavioral dimension (Pearson correlation = −0.1944, p-value = 0.0179). The average exam score was positively correlated to the stress tolerance behavioral dimension (Pearson correlation = 0.1619, p-value = 0.049), and the final grade was positively correlated to the stress tolerance behavioral dimension (Pearson correlation = 0.1536, p-value = 0.0425).
Discussion and recommendations
The assignments and exams from the first portion of the course were compared to the last portion, to evaluate the progress in performance within the semester. Both groups showed a decrease in exam scores. The last exam scores were significantly lower than the first exam scores. As scaffolding was employed, the materials covered in the latter portion of the semester were more involved and could be a factor that contributed to the lower exam scores in the latter portion of the course. Interestingly, the low performers showed an improvement in the hands-on applied assignment scores. Indicating that hands-on applied assignments can foster interest in low performers. This improvement in performance is occurring despite the hands-on applied assignments being more involved in the latter portion, with multiple callbacks to previous topics. These results are similar to previous literature which has found that hands-on activities are more beneficial to helping disengaged students learn. 23
The significant positive correlation between experience in coding and final grade indicates that having some prior experience in coding helped the students perform better in class. As the course covers the cross-disciplinary topic of mechatronics, which blends elements of mechanical, electrical and computer engineering, any experience in coding would help the students be well situated for success in class. However, there was no correlation between performance in class and prior involvement in electronics work and robotics clubs. This could be because the prior experience in electronics would have been at high school level, whereas most students in mechatronics would have taken a collegiate level programming class as a prerequisite. Additionally, students who had prior experience in coding have been shown to have a higher self-efficacy in complex programming, 24 which is required in mechatronic systems that involves a physical system that responds to the program.
Interestingly, the high performers spent less time on theory and hands-on applied assignments as compared to the low performers. The low-performing students could have taken a longer time to understand the questions and iterate through to the final answer. Within the high performing group, the more time the students spent on the theory assignments, understanding the basic concepts, the better they performed in the final project.
The difference in mean ratings between the presurveys and postsurveys showed that the scaffolding hands-on applied approach improved the students’ self-perceived entrepreneurial mindset in both the high-performing and low-performing groups. Separately analyzing the 3Cs (“curiosity,” “connections,” and “creating value”) within the 4 self-efficacy sections revealed that the scaffolding hands-on applied approach was effective at increasing the confidence of both groups in problems related to Mechatronics. Similarly, both groups improved in self-reported success rating; the students felt that they had a greater chance of being successful in tackling mechatronics-related problems after the course. The increase in confidence and success rating across the whole class shows that the scaffolding hands-on applied approach was effective in improving self-efficacy. Our results also show that the two self-efficacy aspects of confidence and success are correlated. This finding is similar to previously reported findings. 4
Given some challenges in assessing student development of the 3Cs and the entrepreneurial mindset, it has been proposed that changes in the 3Cs likely develop over years and that time periods longer than a single semester may be needed to assess significant changes. 2 This study suggests that changes in curiosity, connections, and creating value may in fact be identified within a single course when subaspects of the mindset are considered. For example, student confidence in “creating value” may need to be meaningfully advanced before that student demonstrates (or reports) motivation in “creating value.” This interpretation is encouraging as it may allow educators to identify entrepreneurial mindset changes within a single course as well as giving further insight into how the 3Cs develop over longer time periods (i.e. are improvements in confidence prerequisites to changes in motivation).
Additionally, in this course all aspects of the 3Cs were addressed. Connections between mechanical and electrical elements were explained and supported with reference to how those elements appear in a final mathematical equation. In each hands-on applied exercise the systems created by the students were motivated and connected to some real-life consumer product (Table 2, right column). Students were at times asked to define the key features of the products and translate those to the requirements of their systems. Other times students were given abstract paragraphs explaining the customer desires for the product that required them to gather further information and determine a final system requirement. Particularly in the final project students were aware of a variety of ways in which points could be earned and had the freedom to decide what performance aspects to include in their system. Based on a course design encompassing all 3Cs rather than focused on just one, it appears reasonable that the study saw changes in confidence and success self-efficacy attributes across all 3Cs rather than a specific aspect of the 3Cs increasing for all self-efficacy attributes.
Motivation of students to learn is increased when they perceive that they are learning skills important to their future career. 25 To motivate and engage students in a meaningful way it is important to interweave theory and practice. 26 The current course structure of altering between theory and hands-on applied assignments has helped improve confidence and success self-efficacy ratings. Similar results in improvement of confidence has been seen in other studies that involved project-based learning experiences.25–27 Interestingly, multicultural project-based learning has previously been shown to increase all attributes of self-efficacy. 27 An improvement from our structure that could help improve self-efficacy ratings of motivation and anxiety could be through multicultural projects that involve international collaborations with different schools across the globe. There is an opportunity here, as more governments of developing nations are considering strategies to improve entrepreneurially minded education.28,29 A previous study considering uniqueness and usefulness as a measure of creativity has found that self-efficacy in creativity was at odds with engineering self-efficacy. 30 In other words, the confidence in engineering abilities was inversely proportional to the confidence in one's ability to be creative. This is concerning, but our breakdown shows that there are some attributes of usefulness that are improved; hands-on applied assignments helped the students improve their self-efficacy related to “creating value” within both the confidence and success ratings.
Project-based learning has been shown to improve self-efficacy and is correlated with high student achievement.31,32 However, it is difficult to assess the impact of project-based learning on self-efficacy due to its subjective nature. Measuring the impact on self-efficacy, by individually considering its different aspects, allows us to understand the impact of hands-on applied assignments on student self-efficacy in detail. Work sponsored by KEEN has previously created instruments, based on 9 or 10 factors, to assess the entrepreneurial mindset, 33 our work adds to this by evaluating different elements of self-efficacy which are tied to entrepreneurial mindset. 34 Only the high-performers group showed a significant decrease in anxiety about making “connections” related to mechatronics. This lack of reduction in anxiety in the low-performers group could be because of the perception caused by a lower course grade. Grades are meant as a tool to evaluate the students’ comprehension of the material and provide feedback to help improve student learning. However, as a consequence of contingent self-worth, lower grades can lead to a perception of lower ability and self-efficacy. 35 As self-worth is closely tied to stress, learning, and intrinsic motivation,36,37 it is important to note here that the low performers did not show a reduction in anxiety. Previous literature suggests that this is a complicated issue and does not have a one size fits all solution. Some students benefit from a strong connection between self-worth and academic performance, whereas for others it may impede psychological wellbeing. 38 Anxiety did not decrease for the low-performers group, and even within the high-performing group, a significant decrease in anxiety occurred only within the connections category. A previous study has shown that practicum is better than exams at improving self-efficacy. 31 Additionally, exam induced anxiety is a well-studied and well-known factor.39,40 Although the primary goal of exams was not to improve self-efficacy, inclusion of practicum instead of exams could provide a mastery experience that can help improve attributes of anxiety related to self-efficacy. Additionally, practicums have been shown to be effective at improving self-efficacy in women and increasing student retention in engineering across the board. 31
Parallel feedback has been shown to improve student learning and self-efficacy, when compared to serial feedback. 41 The allowance of resubmissions and interweaving of practice with theory in the course allowed students to learn about different aspects of a topic in parallel. The resubmissions allowed students to correct any theoretical misconceptions from previous weeks, while the current week's theory and practice allowed the students to learn and apply knowledge related to mechatronics. The parallel structure of the class could have been a factor that helped the students improve their confidence and success self-efficacy ratings. Other research says that people with high self-efficacy are more likely to take on challenges and persist to succeed. 41 The parallel structure could have been a reason for the improvement in attributes of success related to self-efficacy. This can also be explored as a factor to help reduce anxiety ratings. Parallel structure has been shown to improve general self-efficacy, and students with high self-efficacy are better at using honest criticism to improve, instead of conflating criticism with personal judgment.41,42
The scaffolding technique employed in the course, along with the use of hands-on applied assignments was able to successfully improve the entrepreneurial mindset within the self-efficacy categories of confidence and success. However, there was no significant difference in motivation. Motivation has previously been described as involving elements of self-belief, transactional engagement, institutional support, and active citizenship. 43 The presurvey and postsurvey has shown that it helps students be more confident about mechatronics and this in turn should help improve self-belief and help students autonomously enjoy learning. Quick feedback, assistance in class to help with hands-on applied assignments and allowance of resubmissions ensures transactional engagement and institutional support. Active citizenship is one aspect that can be improved to help students improve motivation. This could involve final projects that use knowledge learned in the class to help the community. Furthermore, a final project that helps students perceive that they are developing skills for their future career to become an active, involved, citizen can help students develop an engineering identity. 44 Self-efficacy is one antecedent to entrepreneurial intention, 45 and this study shows that some attributes of self-efficacy are significantly improved by using a scaffolding, hands-on approach to teaching Mechatronics. Some recommendations on improving the other attributes of self-efficacy are also provided. Overall, inclusion of hands-on applied assignments can help improve the EM of engineering students.
The behavioral traits of the students included in this study, as summarized using the five-dimensional behavioral scale, is provided in Table 7. This could help instructors decide whether the results of this study are transferable to their own student population. In the sample studied the students who rated themselves higher on the thrill seeker rating tended to perform worse in theory and hands-on applied assignments, whereas students who rated themselves higher on the stress tolerance scale tended to perform better in the exams. Thrill seeking behavior, described as a willingness to take risk to acquire intense experiences, has previously been shown to be inversely proportional to average grade. 46 The impulsivity and tendency for immediate gratification that comes with thrill seeking behavior might be preventing the students from spending time understanding different concepts and performing well in class.46–48 The students who rated themselves higher on the stress tolerance rating might be performing better on the exams because of a better capacity to handle exam anxiety. 39 The higher self-reported stress tolerance rating shows a confidence in one's ability to handle stress, and this has been shown to be correlated to a lack of avoidance in response to exam anxiety and better preparedness. 39 A limitation of the current study is that the groups were created based on the final grade, and there could be some score perception biases that are affecting the reported self-efficacy ratings. However, the grouping used represents a clear distinction between the performance of the highest achievers and the remaining students, which is helpful information for most classroom settings.
Conclusions
Here we present the assessment of the three student learning outcomes of KEEN's definition for entrepreneurial mindset using a modified version of a validated tool for engineering design self-efficacy. This study suggests that changes in these 3Cs, “curiosity,” “connections,” and “creating value,” may in fact be identified within a single tool and in a single course when subaspects of the mindset are considered. In particular, we show that a scaffold learning approach to teaching the course, Mechatronics, significantly increased student self-efficacy ratings of confidence and success across all 3Cs. The hands-on applied assignments were especially beneficial to the low performers. However, motivation was not significantly changed after the course suggesting that: (a) other strategies may be needed, (b) that changes in motivation are slower to develop, and/or (c) that changes in motivation may have perquisites such as first improving student confidence. Future work should focus on pedagogical strategies that target motivation as well as tracking the 3C's development over a longer time period.
Footnotes
Acknowledgement
Partial support for this work was provided by an ‘Engineering Unleashed’ faculty fellowship via the Kern Family Foundation's KEEN network.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Data availability statement
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.