Comparison of AR and physical experiential learning environment in supporting product innovation

Abstract

This article compares how virtual and physical learning aids enhance learners’ product innovation capability, that is, design experiences and domain knowledge. The virtual aid utilises augmented reality (AR) allowing learners to experience a range of animated mechanisms using smart devices. The legacy physical aid mechanisms were made using three-dimensional printers. We studied the effects of both manipulatives on learners’ understanding of mechanical movements, for example, rack and pinion, and Geneva mechanism. To investigate learning impact of each aid, we compared the experimental results derived from two learners groups (13 participants each). This study provides a case to support product innovation education under an experiential learning environment. The outcomes showed that both aids were useful in enhancing design experiences and domain knowledge. Pre-and-post attention, relevance, confidence and satisfaction motivation of both aids was found to be similar. However, distinctive differences were observed in terms of divergent search for ideation, suggesting for further research in combining both aids. We also found that learners’ learning motivation is lower in AR-based aid.

Background

In the domain of product design, learners were commonly striving for alternative design solutions to fulfil given requirements or resolve a problematic situation. A significant gap in the transformation from a field of inquiry (e.g. problem) into a proposition (e.g. alternative solution) was deem challengeable. ^1,2 Ahmed et al. ³ found that novice designers favour leaning aids that (a) enabling them to understand how the design worked or assembled, (b) interactive features to enable them to visualize the design or movement in three dimensions (3Ds) and (c) equipping with physical models to enable them to feel, touch and play with.

There are four learning styles which are accommodating, divergent, assimilating and convergent. ⁴ Different learning styles would impact learning outcomes. ⁵ Diverging learners have the ability to explore a range of observations and generate many ideas. Assimilating learners transform learning experiences into abstract concepts. Converging learners evaluate and refine ideas and theories for better practical uses. And, accommodating learners conduct hand-on activities. ⁶ In a study involving first-year architectural design students, Demirbas and Demirkan ⁷ point out the importance of providing learners with opportunity to exercise different learning styles during the designing activities. Dunlap et al. ⁸ proposed a process that incorporates experiential learning theory into a practical e-Learning workshop. Demirkan and Demirbas ⁹ further suggested the use of multiple divergence–convergence learning environment for design students. Many authors point out the need to match learning aids to learners’ learning style. Huang et al. ¹⁰ developed an eco-discovery augmented reality (AR)-based learning system. Nonetheless, research that incorporated learning aids (i.e. AR-based system) and learning processes to strengthen the effectiveness of experiential learning is limited.

In light of this, many researchers have advocated the use of physical or virtual manipulatives in improving learners’ understanding of learning contents. Both learning aids could provide a perceptual grounding for concepts that were potentially too abstract to easily grasped. ¹¹ Several research studies have attempted to investigate the respective value of each type of manipulatives. Gire et al. ¹² applied both virtual and physical manipulatives to enhance students’ understanding of different pulley concepts. They found that both types of manipulatives had its respective advantages. The other example was to investigate the effect of different types of manipulatives on the learning of light and colour concepts. ¹³ The results showed that the blended combination of the virtual and physical manipulatives was better in enhancing students’ conceptual understanding than the use of physical or virtual manipulatives alone. Brinson ¹⁴ reviewed post-2005 empirical studies in comparing learning achievement in traditional and non-traditional hands-on lab. His study found that 51% of existing studies demonstrated non-traditional manipulatives to be as good as or be superior to traditional hands-on manipulatives. He also found that quizzes and tests were the most commonly used instruments for measuring learning outcomes (about 53%, of 56 studies). As for learning contents, the main comparative studies were on the topics of pulleys, ¹² light and colour, ¹³ heat and temperature ¹⁵ and geometric relation of car engine. ¹⁶ Clearly, little is available in existing literature on how best to support the learning of product design/innovation. This article attempts to address this gap and contributes to current body of knowledge in experiential learning.

According to Demirbas and Demirkan ⁷ and Kolb, ¹⁷ learning is made through the process of experiencing in a cyclical and continuous manner. Through the process of experiencing, learners would gain deep knowledge and use that knowledge under a real-world setting. Learning is ‘the process whereby knowledge is created through the transformation of experience. Knowledge results from the combination of grasping and transforming experience’. ⁴ To consolidate Kolb’s learning framework and Demirkan and Demirbas’s ⁹ findings, an intelligent ‘learning for innovation’ laboratory was set up in Chung Gung University in 2009. The lab aims to provide a new and intelligent learning environment (ILE) to support students majoring design courses in the industrial design department.

The ILE laboratory was composed of advanced tools, legacy tools and synthesis methods that applied to the four experiential learning cycles of Kolb’s learning theory. This project emphasized the use of new and advanced information and communication technologies (ICTs) to build an ILE to enable learners to grasp design experiences and develop new knowledge for innovation. Learners were supported with learning aids for domain knowledge acquisition, that is, mechanical movements, divergent and convergent thinking activities and fast prototyping tools (three-dimensional (3D) printers and 3D metamodeling) to facilitate rapid prototyping. In short, the lab is equipped with a range of physical and virtual manipulatives to support a full design course learning stages, that is, from understanding, reflecting, innovating and prototyping. Thus, it distinguishes from existing experimental research which mainly focuses on specific aspects of design activities. The unique ILE laboratory provides an opportunity to better understand and compare the impact of two learning aids, that is, legacy physical aid and AR learning in enhancing students’ learning in design courses.

To investigate the impact of each aid in supporting product innovation, this article compares both approaches, legacy physical and AR aids, empirically in a design workshop of a product design introductory course. We designed and tested two different learning aids that were intended to deliver learning-related concepts and idea creation. Our aim was to compare both aids in the same ILE laboratory (study site) to determine their effectiveness in terms of attention, relevance, confidence and satisfaction (ARCS), knowledge acquisition, as well as idea creation capabilities. The rest of the article is structured as follows: second section presents the experimental design, setting and methodology; third section discusses the results of learners’ achievement and perception of both aids. The fourth section presents conclusions, implication of this research to academia and practitioners, research limitations and further research directions.

Materials and methods

Intelligent experiential learning environment for supporting product innovation education

The learning environment in the ILE laboratory composed of advanced tools, legacy tools and synthesis methods that applied to the four experiential learning cycles of Kolb’s learning theory (see Figure 1). Advance intelligent tools were implemented from ICT, for example, AR in smart devices to support students’ learning. Legacy tools include physical manipulatives that were used to enhance learning. These manipulatives were built using latest technologies such as 3D printers and 3D modelling software, for example, TinkerCAD. As for synthesis methods, there are divergent thinking such as brainstorming and the analogy thinking. Other ideation methods include convergent thinking, for example, abstraction and S-B-F modeller. ¹⁸

Figure 1.

The intelligent experiential learning environment based on Kolb’s learning cycle.

As such, the ILE laboratory enables learners to go through Kolb’s learning cycles. In the concrete experience stage, learners utilized the learning aid to see how things were moved through the operations. They could touch and manipulate the movement of the mechanism and observe the relation between each component of the mechanism. In the reflective observation stage, learners were guided to reflect their observation into what the input-output movement would be (e.g. rotatory and linear movement), understanding the formal terms and functions of the components, as well as the moving behaviour of each mechanism. In the abstract conception stage, we led learners to establish a metamodeling-like of the observed mechanism to better understand the physical constraints (e.g. size of the component). Learners were then asked to brainstorm the potential applications of the output movements, for example, for locking or for cutting. A new inspiration or utilization of the existing mechanism would be posed to students to trigger them to come up with ideas based on his or her experience. Learners were asked to sketch out their design concepts. In the active experimentation stage, learners were taught to plan the configuration and dimension of the component of their design concept. Metamodeling of each mechanism was provided for learners to transform (e.g. scaling, translating, rotating, etc.) the dimension into the 3D modelling software. 3D printers were provided for them to build design prototypes and to check whether the designed concept was fit for the intended purpose.

Legacy physical learning aid

The basic models used in this study were objects easily observed in everyday living environment. Thus, the models are suitable for novice designers to comprehend. The 12 mechanisms used were Crank, Worm Gear, Bevel Gear, Cam, Scotch Yoke, Slider Crank, Reciprocating Motion, Rack and Pinion, Lever, Geneva Gear, Bevel Gear and Fast Return. ² The selection of the mechanisms used in this experiment was based on the commonly seen mechanism. The physical objects based on these mechanisms were developed using the SpaceClaim software (http://www.spaceclaim.com) (3D geometric modelling tool). The developed 3D objects were then converted into STereoLithography (STL) files. In the experiment, all physical objects were made through the use of UP! 3D Printer Plus 2 and ZPrinter 150. Each component was designed into a simple and generic form. Significant dimensions for each component were defined, for example, gear diameter (40 mm), axis diameter (5 mm), the number of gear teeth (40 mm) and the thickness of each components (5 mm). The designed objects were developed with the consideration of physical tolerance to allow participants to assemble and operate all the components functionally. The participants were allowed to observe and operate each mechanism freely. The mechanism movements include rotate or translate continuously or intermittently.

AR learning aid

The AR learning aid shared the same mechanisms and the same 3D metamodeling objects with the legacy physical aid. The 3DMax created the AR mechanism from OBJ files modified from the same STL files used in the legacy physical models. The tool to implement AR was developed by Unity3D. When the markers mapped to each AR mechanisms were scanned by the AR App, the AR mechanism will appear on the AR App to demo the movement of the mechanism. The red component stands for the input, the yellow one stands for output and the blue one stands for relative components in the AR mechanism. To activate the movement, a rotational or translational AR markers were used. In the AR learning aid, the mechanism was represented in a movement status based on the assumption of a predefined continuous rotational or translational speed. Each AR mechanism moved repeatedly to show the mechanism motion. Six mechanisms were able to move from both ends. Each mechanism had two different AR markers, distinguishing the input and the output in different colours. In this aid, there were movement markers and mechanism markers. The learners were able to observe the still mechanism in various angles as well as the motion of the component and inter-connectivity of components through the AR mechanism to grasp the knowledge of the mechanism. Apart from experiencing the mechanisms, each participant was provided with an A3 size paper. Inside the paper, a table was presented with the photo (for physical-based group) and 3D image (for AR-based group) of each mechanism. The participant was asked to go through each row of the table and fill in or sketch their observation (see Figure 2, an illustration example of a crank mechanism).

Figure 2.

An example of crank mechanism comparison in both learning aids.

Participants’ team allocation

We allocated the 26 second-year, design-major university students into two groups (13 participants each), namely, AR-based and legacy physical-based groups. The AR-based group utilized the AR learning aid, while the physical-based group acquired the legacy physical learning aid. The selection criteria for the participants were based on the students’ enrolment in the course named basic industrial design (I). The enrolled students were all industrial design sophomores. In each group, three or four participants were assigned to share the learning aid. Four subgroups were set for each group. The assignment of each group was based on gender and related competence. Participants’ two course scores in fundamental design (I) and (II) were taken into consideration to ensure similarity among participants at each group. All the participants did not take any university-level mechanisms course.

Learning outcome evaluation

Four types of outcomes were used to compare the effect of AR and legacy physical learning aid in knowledge acquisition and ideation support. These outcome measures include learners’ motivation, achievement in testing, frequency in mechanisms selected for ideation and subjective preference. The first outcome measure was on learners’ motivation. Twenty-four questionnaire items in the categories of ARCS were adopted. These questionnaire items were modified from the 36 questionnaire items ¹⁹ of the instructional materials motivation survey (IMMS) motivation survey, ²⁰ developed from ARCS model of motivational design. The questionnaires were written in mandarins. For each questionnaire item, five-point Likert-type scale was used to assess participants’ opinion scale from 1 = Strongly disagree to 5 = Strongly agree. The questionnaire Cronbach’s α was 0.89 which indicated high internal consistency and reliability. ²¹ The second outcome measure was defined as the number of correct counts of the 10 multiple choice questions. Two classes of knowledge were tested, that is, kinematic movement of mechanisms and physical support of mechanisms. Each class was composed of five question items. Each question was shown with a mechanism in the perspective view, allowing participants to see the representation and then select one of the five choices. These questions were designed by the research assistant and reviewed by two domain experts. The third outcome was to count the frequency of each mechanism selected for ideation. Finally, learner’s reflective feedback based on three questions was applied. We collected their opinions in the aspects of knowledge acquisition and ideation support based on the use of the learning aid.

Assessment procedure

This assessment was performed by one instructor and one research assistant. The assessment was conducted in an ILE laboratory under the learning subject of ‘designing everyday stationery’. Each participant was required to perform individual design innovation, based on a chosen mechanism. Prior to that, participants used the AR/physical-based aid to gain knowledge into various mechanisms, that is, cam, rank, pinion and so on. Participants are required to develop their ideas or design concepts with the support of 3D printing technology in the prototype development stage. The setup required the participants to complete the design project in 5 days (over a period of 5 weeks). Figure 3 presents the experimental process. At each week, a daily activity was planned and conducted under a 6-h basis. In the first day, 1 h was allocated for each participant to complete the knowledge and motivation questionnaires. The details of the first day activity and lessons learned were shown by Liu et al. ²²

Figure 3.

The design studio process.

In the second day, the activities for the participants were to acquire related knowledge and to come up with ideas or design concepts. Due to the limited resources in making learning aids, each group was allocated into four subgroups for sharing the learning aid. Each subgroup had three to four participants. The allocation was based on participants’ preference. In the physical group, each subgroup was provided with a set of 3D printable mechanisms. Each participant in the group was encouraged to select each mechanism, enable the participant to manipulated, observe and play with the movement for each mechanism. In the AR-based group, each subgroup was allocated with one 7-in Google Nexus pad and all the AR markers. Each participant was taught how to use the AR application in combination with AR markers. The participants in the two group were encouraged to grasp the mechanism in the knowledgeable aspects of the input/output movement of each mechanism, each physical elements in the mechanism and the potential representative design concepts from using the mechanism. Having understood the use of the learning aid, the participants in each group were given 60 min to try each of the 12 mechanical movements. The participants were allowed to work alone.

In the third day, all participants were required to report their ideas or design concepts. Two instructors involved in reviewing and modifying each participant’s design concepts. Towards the end of the day, 1 h was allocated for post-test questionnaire survey. Each participant was required to complete the knowledge and motivation questionnaires, respectively.

In the fourth and fifth days, the participants will perform prototyping using the 3D printers available in the ILE lab and presented their designs to instructors. The activities in these two days were not part of this experimental design, thus will not be described in this article.

Analysis methods

An independent t-test for the two-group comparisons was adopted. All statistical tests were two-tailed, and a p value of less than 0.05 indicated statistical significance. Statistical analyses were performed with the SPSS v.20 software package (IBM Corp., Armonk, New York, USA).

We used a qualitative content analysis to analyse each participant’s designed ideas as well as reflective writing feedback. The results were collected by a research assistant and cross-checked by the instructor. The results on designed ideas were collected from each participant’s sketches and written notes. Each mechanism movement selected and converted into written ideas or sketches by the students was extracted and counted to obtain a cumulative tally.

As for reflective questions, the results were summarized in the pros and cons of each aid in dealing with the specific learning or innovating processes in (a) comprehending mechanism movements and (b) converting these mechanism movements into design concepts. The results were grouped in themes and clustered into specific categories.

Results and analysis

Participants

Twenty-six participants took part in this study. The participants were divided equally to AR-based and legacy physical group, that is, 13 participants each. The distribution of characteristics among the two groups (see Table 1) did not reveal significant differences in considerations of gender, course scores and pretest knowledge.

Table 1.

Distribution of characteristics among the two groups.

Variable		AR-based (n = 13)		Legacy physical (n = 13)		p Value
		N	%	N	%
Gender	Male	3	23.08	2	15.38	1.0000
	Female	10	76.92	11	84.62
Course scorea	Basic design (1)	80.77 ± 4.97		80.1 ± 9.53		0.8431
	Basic design (2)	86.62 ± 6.65		87.7 ± 5.87		0.6878
Pretest scoreb		5.08 ± 2.02		4.77 ± 1.42		0.657

AR: augmented reality.

^aBasic designs (1) and (2) are the capstone courses of the participants conducted in the first year.

^bPretest score was counted by number of items answered correctly from the 10 multiple choice questions.

Learning motivation

Table 2 shows the comparison of pretest and post-test of learning motivation. The four learning motivation categories in the AR-based group were slightly lower in the post-test but with no significant differences. Similarly, the attention and relevance categories of the legacy physical group were also found to be slightly lower in the post-test. However, the confidence category was better in the post-test. As for the mean score of the four categories, regardless of pretest or post-test, both groups have the same ranking, that is, with attention ranked highest, followed by satisfaction and relevance, and the lowest mean is in the confidence category. However, on the question ‘learning contents in designing based on mechanism movements were too much’, both groups have significant difference (p < 0.05). The post-test result of the physical group is better than pretest, but not in the AR group.

Table 2.

Comparison of pretest and post-test learning motivation.

Dimension			AR-based (n = 13)		p Value	Legacy physical (n = 13)		p Value
			Mean	SD		Mean	SD
Attention
	Overall	Pretest	4.10	0.80	0.1714	4.14	0.70	0.6885
		Post-test	3.99	0.81		4.10	0.75
Relevance
	Overall	Pretest	3.85	0.90	0.2678	3.90	0.85	0.8928
		Post-test	3.67	1.03		3.87	0.89
	Learning contents in designing based on mechanism movements were too much (reverse question)	Pretest	3.54	1.05	0.1199	3.46	1.05	0.2930
		Post-test	2.92	1.12		3.85	0.69
Confidence
	Overall	Pretest	3.44	0.78	0.7424	3.58	0.87	0.7759
		Post-test	3.40	0.75		3.62	0.93
Satisfaction
	Overall	Pretest	4.12	0.77	0.7480	4.19	0.69	1.0000
		Post-test	4.08	0.69		4.19	0.63

AR: augmented reality; SD: standard deviation.

Learning achievement in knowledge

The post-test results of knowledge achievement are better for both groups (see Table 3). The results showed that participants in both groups have improved their understanding of the kinematic motion of a given mechanism. Moreover, they were able to identify and match the appropriate geometrical support of the given mechanism. Specifically, in the geometrical support category, the legacy physical group performed (p < 0.05) better than the AR group.

Table 3.

Accuracy comparison of pretest and post-test knowledge achievement.

		AR-based (n = 13)	p Value	Legacy physical (n = 13)	p Value
		correct		correct
Overall	Pretest	5.08 ± 2.02	0.062	4.77 ± 1.42	0.019
	Post-test	6.38 ± 1.90		6.23 ± 2.17
Kinematic motion	Pretest	2.46 ± 1.76	0.106	2.69 ± 1.25	0.356
	Post-test	3.23 ± 1.54		3.08 ± 1.85
Geometrical support	Pretest	2.62 ± 1.12	0.170	2.08 ± 0.86	0.012
	Post-test	3.15 ± 0.90		3.15 ± 0.90

AR: augmented reality.

Mechanism taken for ideation

Table 4 presents the frequency of the mechanisms (12) chosen for ideation. For the AR group, cam mechanism was found to be most popular. Nine of the 13 participants selected cam for subsequent design ideation. The five less commonly used mechanisms were bevel gear, reciprocation motion, rack and pinion, Geneva stop and crank. However, in the legacy physical group, the three most popular mechanisms were fast return, rack and pinion and slider crank. Only one mechanism was less popular, that is, reciprocation motion.

Table 4.

The breakdown of mechanism selection in both groups.

Mechanisms	AR-based (n = 13)	Legacy physical (n = 13)
Often used	Cam (9)	Fast return (9)Rack and Pinion (8)Slider Crank (9)
Less used	Bevel gear (3)Reciprocation Motion (2)Rack and Pinion (4)Geneva stop (2)Crank (3)	Reciprocation Motion (4)

AR: augmented reality.

Subjective opinions

All subjects (26 participants) in both groups completed the subjective evaluation (i.e. open questions such as can AR/physical aid support understanding of mechanism movement?;can AR/physical aid support design ideation? etc.). Written feedback was analysed using the content analysis method. From the analysis, five specific themes were extracted (see Table 5). The AR-based group stated that the AR aid had the following four advantages: supporting comprehension (92%, 12 respondents), triggering idea inspiration and association (46%, 6 respondents), understanding detailed design in prototyping (15%, 2 respondents) and visualizing movements (8%, 1 respondent). However, the AR aid had one limitation, that is, its rotation angles (15%, 2 respondents). Similar results were reported in the physical group, with the additional advantage of ‘helpful in logical thinking’ (8%, 1 respondent). Nonetheless, three participants pointed out that the physical aid had one limitation in supporting ideation. They argued that their ideation/thinking was affected by the physical appearance of the observed mechanism. In other words, observing a limited number of concrete components might anchor their views and hinder them from free imagination.

Table 5.

The subjective opinions of the AR-based and legacy physical of the two groups.

	AR-based (n = 13)	Legacy physical (n = 13)
Pros	Support comprehending mechanism movements (92%, 12) Support idea inspiration and association (46%, 6) Support design details in making prototypes (15%, 2) Support timely observation of movements (8%, 1)	Support comprehending mechanism movements (92%, 12) Support idea inspiration and association (31%, 4) Support design details in making prototypes ((15%, 2) Helpful in logical thinking (8%, 1)
Cons	Observed angles were limited (15%, 2)	Ideas affected by the appearance of mechanism (23%, 3)

AR: augmented reality.

Discussion

This research investigates how the virtual and physical learning aids the ILE laboratory would support product innovation education for novice designers, in the aspects of learning-related concepts from the existing mechanisms and selecting to transform these mechanisms into designed concepts. To investigate the impact of each aid in supporting product innovation, we compare AR-based aid and legacy physical aid on learners’ knowledge acquisition and idea creation in a product design course. The results indicated that there is no overall preferred manipulative for product design learning; both aids were useful in enhancing design experiences and domain knowledge. However, the selection of mechanisms for ideation has shown distinctive differences, requiring further investigation. The contribution of this research includes a better understanding of how the proposed experiential learning environment would enhance learners’ product innovation capability. In the following sections, we discuss the results in terms of learning motivation, learning achievement, inspirations for ideation and user preference.

Learning motivation

In the learning motivation category, there was no significant difference for AR- and physical-based aid in both pretest and post-test comparison. The higher mean scores were found in attention and satisfaction categories. The results showed that students enjoyed and appreciated the supporting aids used and stayed motivated throughout the learning.

One thing worth mentioning was that in the post-test, the two groups showed significantly different perspectives (p < 0.05) towards the question ‘Learning contents in designing based on mechanism movements were too much’. Participants in the legacy physical group tended to consider the learning contents to be too much (between fair and agree), while not in the AR-based group (between disagree and fair). This would imply that further investigation is needed to reduce mental load in the physical group to see whether a format (either physical or virtual) would lead participants in providing different judgement towards the amount of learning contents. In the study, we also noticed that improving the introductory contents to make participants feel at ease and comfortable with both learning aids is crucial.

Learning achievement

In our study, both aids has shown positive results in overall knowledge testing, that is, virtual learning aid is as effective as physical learning aid. Although the pretest and post-test results for legacy physical aid were significantly improved (< 0.05), in comparison with marginally significantly (∼ 0.06) in the AR aid. In this aspect, legacy physical aids which enabled students to feel, touch and play with the mechanisms are better in specific domain knowledge, that is, geometrical support. This finding echoing results from previous studies. ^2,12 Both types of aids had some level of differences in enhancing learning. Gire et al. ¹² found that physical learning aid tempted to be superior in concrete knowledge introduction, for example, distance relation when pulling a physical pulley. However, the virtual learning aid was better in abstract concept illustration, for example, the concept of work. Although there are differences between our study and Gire’s study, that is, types of mechanisms (12 mechanisms vs. 1 mechanism), target groups (design-major students vs. university students in a conceptual physics lab) and technology (AR vs. computer). Nonetheless, both studies obtained similar findings.

Inspirations for ideation

In terms of the inspirations for ideation, the mechanisms chosen for ideation were largely different in both groups. In the AR-based group, the most often used mechanism was the cam, whereas fast return, rack and pinion and slider crank were more popular in the physical-based group. In short, five mechanisms were less used in the AR-based group as compared to only one mechanism in the physical group. This might imply that the format of learning objects either in virtual or physical format would affect the selection of mechanism for ideation. However, little is available in existing literature on the factors that would affect the selection of a certain mechanism for design ideation support. Further research should account for a wider range of design activities, target groups and decisions when selecting the mechanism for further ideation development.

Subjective evaluation

In summary, most participants in both groups believed that the AR- and physical-based aid was helpful in understanding mechanism movements. Some participants also pointed out that both aids were particularly useful for ideation and prototyping.

Specifically, in the AR-based group, participants pointed out that 3D virtual aid is very helpful in comprehending mechanism movements; 46% of the participants agreed that the aid would support idea inspiration and association, that is, observing the mechanical movement help to understand and better associate to abstract behaviour of certain design concepts. Two participants also pointed out that the aid enabled them to gain better insight into design details that were helpful in prototype making. They could learn by observing the support (e.g. joints and brackets), mechanical components and understand how different mechanisms would interact with each other. However, two participants also identified that the AR aid has limitation, that is, only limited to certain angles rotation. The participants could rotate, elevate or lift the smartphone to see the mechanical movements in different perspectives/angles. However, they were only limited to see from one perspective, for example, from the bottom of the mechanism.

In the legacy physical group, participants strongly agreed that the aid allowed them to better comprehend mechanism movements through direct observation and manual operation of the mechanism. This is a common advantage of using physical models to support the understanding of the learning contents. Four participants also pointed out that the aid could support idea inspiration and association. The mechanism movement would trigger or inspire them to come up with some ideas. Two participants also pointed out that the aid provided design details that were helpful in prototype making. Direct interaction (e.g. touching, rotating and observing) of the physical models was extremely helpful when making prototypes. With the introduction of new technology, for example, 3D printing technology, the manufacture of these physical objects has become relatively low cost, easy to make and available for wider application. Including the legacy physical aids into ILE is worthwhile and better than simply using one learning aid in the environment. However, two participants also identified that the physical aid has one limitation during the stage of ideation (e.g. brainstorming ideas). Their ideas could be fixed by the appearance of mechanisms. Observing a limited number of concrete components might anchor their views and hinder them from free imagination which is critical in product idea creation.

Limitations

The experiments described here were conducted under a specific course applying a predefined list of mechanisms for supporting learning and designing activities. Due to the small sample size of the students’ enrolment in the course, care should be taken in generalizing the results of using the proposed activity design to other learning groups. A wider range of target groups could be helpful for further explanation.

Conclusion

This study compares the impact of AR and legacy physical aids in the aspects of motivation, knowledge, mechanisms taken for ideation and subjective opinions. The comparison study of both aids was conducted in the same ILE laboratory (study site) in product innovation education under an experiential learning environment. Experimental results and participants’ feedback indicated that both experiential learning-based environment (integrated with technologies and intelligent systems) have potential to support their understanding of learned mechanism and convert their conceptual understanding into product design ideas. However, the generalizability of our findings might be limited due to the small sample size and heterogeneity of participants (i.e. our evaluation study only focused on design-major university students). Moreover, learners’ project results (designed artefacts) were not included in this study. This was because the progress in developing these artefacts involved experts’ suggestions and inputs, thereby, this could not be completely considered to be the contribution of the aids. Future works could explore the application of both aids to more diverse product innovation topics, adopt a different learning outcomes assessment and expand the use of both aids to wider learners from different background. ¹⁴ Furthermore, a study to combine both aids to act as a blended learning aid could be pursued (as seen in the works of Olympiou and Zachara ¹³ , Toth ²³ and Tan et al. ²⁴ ). In addition, further studies could be conducted to interpret the experimental results to gain further insights by using other theoretical models such as embedded and embodied perspectives on cognition. ²⁵