Abstract
The ability of research innovation is particularly important in the era of rapidly changing environment, challenges, and demands, where new knowledge, technologies, and unseen problems are emerging at an unprecedented speed. Engineering college students, especially postgraduate research students are well-trained with domain knowledge and technical skills to solve commonly known problems. However, the lack of a systematic critical thinking framework on research innovation prevents them from generating creative ideas, dealing with changes and new problems, and making step forward and groundbreaking innovations during their research program. This paper presents a seven-question based critical thinking framework, named “IDEA” catechism, which was proposed and implemented as a research training methodology in a research group at a Chinese higher education institution. Through the questions and answers (Q&A) process in the framework, engineering students can be better trained to deeply understand and critically scrutinize their research ideas and methods, and are eventually able to independently develop their own innovative ideas, especially leading them to do original study. The research group has been practicing this framework for several years and succeeded in developing a large number of distinctive innovation with national and international recognition, which were also collected as research cases.
Keywords
Introduction
Cultivation of talents is crucial for research and innovation in engineering area. In traditional engineering education, domain knowledge and technical skills are the main focuses in student cultivation, and there were many research attempts to innovation ability education.1–3 Needless to say, the acquisition of the domain knowledge and skills as the basis for accomplish engineering tasks is very important. However, the world is changing rapidly and engineers have to deal with unpredictable complex technical issues, making traditional training paradigm far from enough to cope with the challenges in today’s innovation projects. 4 In the current dynamic era with big challenges, disruptions and uncertainties, new requirements and technologies are emerging at an unprecedented speed.5,6 In engineering area, new manufacturing equipment and technologies, measurement methods and technologies as well as related upstream technologies are emerging day by day,7–9 only the researchers with innovation capability can keep up with the development pace of new technologies. The lack of originality will lead to weak competitiveness for a person, a company as well as for a country. So research innovation especially on fundamental study is very important and quite needs attention.
In this paper, a framework to facilitate innovation development in engineering research activities, named “IDEA” catechism is presented. The framework consists of seven questions which can be used to assess the level of innovation of a research work, and to gradually cultivate the critical thinking ability of students majored in engineering by guiding them through the questions and answers (Q&A) procedure. The authors have implemented the framework and practiced for many years to inspire engineering students to generate innovative ideas and achieved many significant breakthrough results with national and international recognition. Details and related case studies are described in the rest of this paper.
Overview of existing methodologies
As we all know, the basis of research and innovation is sufficient domain knowledge acquisition and critical thinking, and critical thinking is more important in many cases which should be emphasized in education, especially in engineering research subjects. In order to facilitate research and innovation, various methodologies and critical thinking frameworks have been proposed and practiced, which are briefly reviewed below.
Critical thinking in engineering education
There are various definitions and understandings on critical thinking. Critical thinking is deemed as reasonable and reflective thinking, whose purpose is to justify beliefs and actions. Critical thinking is also a kind of ability to evaluate, compare, analyze, criticize, and synthesize information. 10 In some sense, it is the ability to think outside yourself and reflect on your own thoughts. As a result, critical thinkers are willing to explore and investigate unsolved issues, as well as to scrutinize broadly accepted perceptions. Critical thinkers are able to analyze the quality of the evidence for their views and examine the flaws based on logic reasoning.
Critical thinking is so important because it inspires researchers to think independently toward conventional point of view instead of blind following. It is commonly believed that some personal qualities should be satisfied for critical thinking, including intellectual humility, intellectual courage, intellectual autonomy, intellectual empathy, intellectual integrity, intellectual perseverance, confidence in reason, and fair mindedness to name a few. 11
In the aspect of critical thinking in innovation education, there are mainly two approaches in practice, including (i) exclusive critical thinking courses in education curriculum, and (ii) incorporating critical thinking practice into other subjects.12,13 In the operational level, there are also many methods, of which two common methods for incorporating critical thinking in engineering and technology subjects are reflective writing and problem based learning. 14
There were a lot of related research on the above approaches and methods.15–18 For example, in the reflective writing aspect: Morse 19 described a strategy for promoting active learning and critical thinking through effective grading. Students were assigned written assignments on topics to be discussed in class during the period in which the written assignments are due. In the problem based learning aspect: Nasr and Ramadan 20 offered some suggestions for assessment in a problem based learning environment, for example, the ability to reason through given information and identify a solution to the problem. Shoop and Ressler 21 developed a course for critical thinking, creativity and innovation of undergraduate engineering students. The course was structured as a deliberate interactive engagement between students and faculty that combined the Socratic method 22 with the Thayer method 23 to develop an understanding of disruptive and innovative technologies and a historical context of how social, cultural, and religious factors impact the acceptance or rejection of technological innovation.
Adair and Jaeger 24 designed a critical thinking model by incorporating critical thinking into an engineering undergraduate learning environment. Assessment and feedback of critical thinking were also included in the model. Douglas 25 proposed two methods for measuring critical thinking in engineering, that is, missed methods study and qualitative study. Control experiments for undergraduate students and graduate students were tested. Project based teaching has been widely used for cultivating the ability of critical thinking and innovation, in which the teaching process is carried out on a detailed project, and students’ creativity is expected to be inspired. 26 It is quite a good model and a carrier for critical thinking and innovation cultivation.
At present, the education on critical thinking is focused on the principle and methodologies on critical thinking itself or some specific courses. It still lacks the systematic education of critical thinking on research and innovation in engineering curriculums both for undergraduate students and postgraduate students.
Critical thinking methodologies in research and innovation
The critical thinking ability plays a key role in innovation development, with many existing frameworks and methods evolved from previous efforts.
One of the mostly adopted methods is Brainstorm: everyone involved can express his or her idea freely under some pre-defined principles. Brainstorm is a very good way for idea generation, but it still needs some supervision so as to achieve better effectiveness.
Ling and Yang 27 tried to answer some important questions for graduate students and junior faculty members at universities such as: how can they identify good ideas for research? How do they conduct solid research to verify and realize their new ideas? How can they formulate their ideas and research results into high quality reports/papers, and publish them in highly competitive journals and conference proceedings? Although they gave some very good practical suggestions over the whole research stages, a systematic framework for critical thinking and a deep research work is still expected.
The director of DARPA, Heilmeier developed a list of questions for research proposals to answer in order to receive funding, that is, the Hilmeier Criteria or nine questions to a focused proposal, 28 such as: What are you trying to do? What is the problem? Why is it hard? How is it done today, and what are the limits of current practice? What is new in your approach, and why do you think it will be successful? The nine questions from DARPA set up good criteria for evaluating the worthiness of supporting a research proposal, but a systematic framework is still needed to educate engineering students so as to cultivate their critical and innovative thinking ability.
An editor from the Journal of Nature argued that “world-class original research” should satisfy three standards: significance, uniqueness, and rigor. 29 More specifically, some researchers 29 provided six different attributes on “world-class original research”: A major breakthrough in the field of classical science, new research methods and techniques that can be widely used, obvious practical applications, proposed a complete new concept, breaking the traditional theoretical system, and breaking new ground. These criterions are quite useful to guide high level research, and further detailed criteria are required for leading to original research.
Previous research in research methodologies attempted to evaluate innovation in engineering, idea generation, and innovation development, which are still difficult to replicate in training engineering graduate students, because what young researchers really need is how to inspire them to do fundamental research. Thus, a generalized and effective critical thinking framework is needed for educating engineering research students to make valuable innovation in a wider range of research projects. First, such framework should be utilized to train students to assess the level of innovation of existing research work and get inspired. Then, students can be assigned with some engineering problems. When new ideas are generated to solve these target problems, the framework should be further applied to cultivating students on their own innovation development.
The “IDEA” catechism framework
The authors are faculty members from a leading Chinese higher education institution, who have been observing and supervising engineering postgraduate students for many years on their research activities, including but not limited to, literature review, idea development, and paper writing. As far as the students involved, most of them, even with adequate English reading skills, still spent excessive time to truly understand a related research work. As a consequence, few of the students managed to impartially and consistently evaluate the novelty of a research work. It became particularly difficult for students to develop new ideas and critically evaluate their own research work due to the lack of a systematic thinking framework. As a team devoted to research and teaching engineering college students, we felt obliged to develop a generalized framework for cultivating engineering students to gain critical thinking ability and do high quality research. Motivated by the previously described “problem & method” criteria, we have develop the IDEA catechism as a more self-contained systematic framework to evaluate scientific research, especially in the scope of engineering research study.
The IDEA catechism was inspired by the Socratic questioning pedagogy, and contains seven questions as well as their further inquiries to be answered, as provided in Table 1, through which a research innovation, either engineering evolution or scientific breakthrough, can be better decomposed and digested. Unlike the nine questions in Hilmeier Criteria whose goal was to assess research proposals, the IDEA framework aimed to train postgraduate students, especially those majored in engineering research to practice critical thinking and judgment toward a completed/reported research work, as well as to generate novel ideas of their own. The analysis of research challenge is encouraged and included in this framework, by which students can understand why and how such ideas are made. Apart from the seven questions as the primary criteria, the IDEA catechism also encapsulates further inquiries after each of the seven primary questions. The answers to these further inquiries not only help reach a better judgment on the research work, but also inspire rational thinking to find and strengthen the students’ own weaknesses.
List of questions and further inquiries in the proposed IDEA catechism.
Background: Why are you doing this?
The first and foremost question should always be the one related to the motivation, as to explain why you are doing this research by introducing the background. Research background is a brief outline of related information and knowledge within a particular subject or field, through which the motivation of your research work can be possibly clarified. Engineering research should always explicitly involve background discussion in order to delineate its significance and potential contribution toward the development of human society. A better judgment can be made when the potential change to be made (either qualitative or quantitative, a step forward or breakthrough) after doing this research is pointed out to satisfy the motivation.
In addition, further inquiries under this question regarding the importance and necessity of such change help to unveil the exclusiveness of this research, which are also important criteria to assess the significance. For example, in the aircraft design research area, lightweight structure design and manufacture is always an essential approach to enhancing aircraft performance and range, which can never be omitted even with the availability of lightweight material such as carbon-fiber composite. Therefore, a convincing background argument should contain not only the research motivation, but also the potential change to be made and the explanation of whether the change is important and necessary. By critically investigating a research background using these questions, the motivation of doing this research can be impartially analyzed to identify whether further investigation is worth taking.
Problem: What are you trying to solve?
Defining a research problem and explaining the tasks you are taking to solve the problem can be quite challenging, especially when the problem underlies complex engineering issues. In fact, many scientific publications delineate their research problems with unnecessary jargons, making it difficult to truly understand the research objective, even for domain experts. Therefore, the research problem should be purified in an explicit and concise way, in order to articulate the objective to the researchers themselves and also to a wider range of stakeholders.
To better delineate the research problem, some further inquiries need to be addressed. In the scope of engineering research, most problems are from practical applications, either purely caused by technical issues or of fundamental scientific nature. Pertaining to the former, the solution to such technical issue may be directly obtained by integrating some off-the-shelf methods, while the research contribution will be limited to this specific technical problem. Otherwise if the problem is of a scientific issue, meaning that new theories/principles need to be developed to solve the problem, then such problem must be explained in a more scientific manner. Additionally, one is encouraged to identify the subject from a list of academic disciplines, or multiple subjects if the problem is interdisciplinary, so as to clarify the required background knowledge and potential solution.
Current status: How was it done by others?
A good interpretation of the research problem is not enough to justify the originality of the research. It is commonly recommended to conduct a thorough investigation from the literature to get an overview of state-of-the-art achievements done by previous researchers.
An effective and convincing literature investigation should contain three categories of related work. The first category includes those work that relates to the research subject and its relevant background, in order to solidify the necessity of investigating the raised problem. The second category involves previous achievements regarding the exact research problem to understand the state-of-the-art process in this niche area. The third category, if any, should contain those research achievements related to the proposed idea (solution). Most of the time, the proposed idea is inspired from a well-known theory or an existing method in a different research area. For example, in the area of intelligent manufacturing, researchers tend to adopt different machine learning algorithms to tackle the same manufacturing issue. In such case, the third category should contain publications regarding different machine learning techniques. In addition to impartial statement of related research from aforementioned three categories, one should also conduct rational argument to the related research because every published work has its own limitations and gaps. However, it is also recommended to make critical argument with solid justifications and from the aspect of the research problem.
Challenge: Why didn’t others do it well?
In addition to a comprehensive review and analysis of related research and achievements, it is reasonable to further justify the research challenge, by explaining why others didn’t do it well and you can solve it in a better way. Unfortunately, very few published research conducted in-depth analysis of this inquiry due to the lack of solid justifications. According to our practice, asking this question is particularly important in cultivating postgraduate researchers. If a student can objectively explain the reason of previous research gaps, the student will be able to build up his/her own competitiveness to solve this particular problem, and also a wider range of diversified problems in the future.
We try to explore the answer to this question from three aspects of reasoning. The first reasoning of the existence of research challenge is due to the lack of the content of knowledge of previous researchers. If a well-defined problem remains to be solved, it is most likely that certain knowledge regarding the solution is missing or undiscovered. In this case, someone with additional knowledge has greater chance to solve it. For instance, the distinguished group DeepMind was founded with scientists studying neuroscience and data science, which can be one important explanation of their success in the area of artificial intelligence. The second reasoning is due to the limitation of thinking. Innovation is usually achieved by thinking in a different or opposite direction. Even if someone with higher academic degree has sufficient knowledge base to comprehend the problem, he/she may still fail to solve it because of his/her fixed mindset. For example, when most researchers tried to eliminate the deformation of 3D printed parts to make it as accurate as modeled, someone started to take advantage of the deformation and proposed the concept of 4D printing, with the fourth dimension being time. The third reasoning is due to value conflict, which is rare but a possible and important explanation. People from different countries, religions, and with different experiences during their lifetime will gradually sculpt their own sense of values. This will render obscure impact to their mindset and thus influence their thinking and decisions when encountered with a research problem. Elon Musk is a renowned entrepreneur for establishing several world-class companies including Tesla and SpaceX. He is also the chief engineer of SpaceX responsible for all technical issues. When Elon Musk encountered three failed launches of SpaceX rocket, he was facing serious financial and trust pressures on failing another rocket. He would not have succeeded in the fourth launch without his obsession and strong commitment to fulfilling his intention for affordable space travel. The success of SpaceX and Elon Musk can be all attributed to value and which also explained why many other companies cannot become SpaceX.
Innovation: What is your new idea?
Innovation is a general term to describe successful implementation of creative ideas that include but not limited to: new goods, new methods, and new markets, which is related to but not the same as invention. 30 In the field of engineering research, we define two types of innovations, that is, technical innovation and scientific discovery. Technical innovation involves new ideas to solve engineering problems using new methods, tools and technologies, while scientific discovery generates new knowledge by observing and analyzing phenomenon to explore underlying principle. Scientific discovery can be either objective natural laws, for example, the earth orbiting around the sun, or explainable theory, for example, the law of universal gravitation. In this way, the significance of great innovation lies not only to its ability of solving new problems, but to its generated knowledge to better explain the mechanism behind.
Another important statement of innovation is to explain why it theoretically works, in order to convincingly justify its effectiveness. Sometimes people tend to misidentify technical skills into technical innovation as they both can solve a specific problem. However, it is necessary to distinguish these two types, because skills are sometimes unexplainable and irreproducible.
Method: How do you get it done?
The realization of innovation requires a suite of organized methods to exhibit step by step how you get it done. To elaborate the method, it would be better to explicitly prove that the proposed method is feasible and rigorous, because scientific research requires high level stringency. On a different note, reasonable amount of time and cost should be planned for the development of a method, because engineering research work should aim for eventually commercialization in the form of products, systems, or services (solutions). An innovative method that requires excessive financial and time support is unlikely to be finalized into applicable solution.
Verification: What criteria to be used for verifying results?
To quantitatively verify the effectiveness of the innovation, one need to provide some criteria, such as a benchmark test against existing methods. Therefore, once the criteria are chosen, whether the criteria are necessary and sufficient to evaluate the validity of the research results should be explained. As we go back to the background section where the potential change that the proposed innovation was expected to make, a corresponding response should be given to verify such changes. Additionally, if the solution consists of both new method and some existing ones, an ablation study is always expected to find out how much improvement gain is attributed to the new method, and how much is attributed to the combined existing methods.
Case studies
The “IDEA” catechism framework had been applied in research practice for a number of years, together with the a Collection of Cases established by the research team to train the critical thinking and innovation ability of postgraduate students, as shown in Figure 1. The team has archived a Collection of Cases from more than 90 research projects reported in top journals as well as from the research team. Each Case includes the presentation slides and the recorded discussions of the seven questions.
Research training framework based on the IDEA catechism.
In order to understand high quality papers from top journals, such as Nature or Science, a weekly seminar was held, where students and professors had brainstorm discussions to find appropriate answers to the seven questions regarding selected research work from the Collection of Cases. Generally, research work from cross-disciplinary fields requires longer time to understand due to the lack of domain knowledge, especially for young researchers. After applying the proposed framework, students were encouraged to disassemble a research paper into seven aspects. In this way, communication became quite effective to inspire students from both knowledge and thinking perspectives, and they could very rapidly learn how to do solid research and publish high quality papers. Once a new idea was proposed and/tested by students, it was discussed at the seminars and evaluated on whether it was qualified under the “IDEA” catechism. Completed research work was added to the archived Collection of Cases. Among the 90 research projects, examples include major technological breakthrough (14 research projects), such as “Flight of an aero plane with solid-state propulsion” (Nature 2018, 563, 532–535) and “Mastering the game of Go with deep neural networks and tree search” (Nature 2016, 529, 484–489); the discovery of new principles and phenomena (15 research projects), such as “Continuous directional water transport on the peristome surface of Nepenthes alata” (Nature 2016, 532, 85–89) and “Prefrontal cortex as a meta- reinforcement learning system” (Nature Neuroscience 2018, 21, 860–868); the cutting-edge investigations in computer science (28 research projects), such as “Characterizing and avoiding negative transfer” (CVPR. 2019); the latest research findings in manufacturing area (11 research projects), such as “Triode-mimicking Graphene pressure sensor with positive resistance variation for physiology and motion monitoring” (ACS Nano 2020), and the research to be published/presented by our own group (29 research projects).
To further explain the connotation of the seven questions, three typical cases established from the research team’s own practice are described next, including the answers to the seven questions. The three cases are very representative, one was published at a high profile international conference, one was published in a high impact international journal, and one was awarded a top prize as a competitive project. Each research project has its own characteristics on innovation, which can provide quite good reference.
Case 1: Research presented at a top conference
The first Case is the research orally presented at the 68th CIRP General Assembly in 2019 and also published in the CIRP Annals - Manufacturing Technology. CIRP is an international academic organization for manufacturing highly recognized by the manufacturing research community. The critical thinking framework based on the IDEA catechism was used in the whole process of the research as well as writing the paper. At the beginning of this research, we analyzed that traditional statistical methods cannot address the research problem to be solved, so we tried to find a new approach. When we reviewed a paper from a top journal, we learned a new method of meta-learning, and we re-defined the nature of the problem (from a statistical matter to a deep leaning matter), and used the meta-learning idea, finally good results were obtained. The answers to the seven questions and further inquiries are provided in Table 2.
A research presented at a top conference answering seven questions.
Case 2: Research published in a top journal
The second Case is the research published in IEEE Transactions on Cybernetics which is a top journal of artificial intelligence, with Impact Factor of 11.079 in 2020. The critical thinking framework based on the seven questions was used in the whole process of the research as well as writing the paper. The problem addressed in this research was raised from manufacturing issues. Firstly, we tried to use “transfer learning” to address the manufacturing issue. When we tried to use some existing “transfer learning” approaches, we found that they were not able to solve the problem. Then we tried to define the nature of problem from the perspective of data distribution, and proposed a new idea and developed a solution to the problem. The research also contributed to the artificial intelligence field. The answers to the seven questions and further inquiries are provided in Table 3.
A research published in a top journal answering the seven questions.
Case 3: Research awarded top prize
The third Case is an innovation competition project participating “Internet +,” the project was awarded gold prize. “Internet +” is the highest level of top innovation competition for college students in china, and it has wide and deep impact on college students from all over the world. 31 The critical thinking framework based on the seven questions were used in the whole process of the research as well as preparing and participating the competition process. The answers to the seven questions and further inquiries are provided in Table 4.
A research awarded top prize answering the seven questions.
By using the critical thinking framework, the significance, innovation, and effectiveness of research can be comprehensively evaluated, and the weakness of the research can also be improved accordingly. This method has been applied in our research team for many years, and the research ability of the students in the team have been significantly improved. To be specific, in the last 5 years of adopting the critical thinking framework, the average number of papers published by each researcher of the research team in SCI indexed journals has doubled, compared with the same period 5 years ago, and the impact factors of the journals that the papers were published are also much higher. In the aspect of innovation competition projects for college students, our team had outstanding achievements in the last 3 years, including three gold prize in “Internet +,” one grand prize in “Challenge Cup,” 32 both of the competitions are top Chinese innovation competitions for college students. The gold prize is very competitive, for example, there are only 50 gold prizes of “Internet +” each year for the whole country, and the gold prize rate is about one out of 10,000. All of the three projects from our team participated the competition obtained gold prize, where the seven questions based critical thinking framework played very important role in the process. The research team has cultivated many excellent talents who are currently working in industry using the framework for their innovative work, and playing important roles in the development of their enterprises. Therefore, the critical thinking framework based on the seven questions and further inquiries have played significant role in our research and innovation talents cultivation.
Conclusions
Critical thinking is essential for engineering research students to develop innovative ideas. This paper introduced a framework based on the IDEA catechism to practice critical thinking and cultivate innovative talents. This new framework consists of seven questions to critically evaluate research from seven aspects, that is, background, problem, current status, challenge, innovation, method, and verification. Each aspect also encapsulates further inquiries to make a better judgment of the research. In this way, a previous research work can be better understood, and new ideas can be better evaluated. Through continuous training, students gradually develop systematic critical thinking ability and are better inspired by those high quality research papers to conduct innovative research work of their own. The case studies from our own group proved with excellent results that this framework is effective in innovation development for engineering research, helping us to cultivate more innovative talents.
The proposed and implemented framework will be continuously refined in the future to adapt to students from different grades and majors. As for practice, high quality research work will continue to be selected to enrich the case collection. We also plan to construct a virtual platform to provide better accessibility of case studies and more effective online communications during pandemic, hopefully to inspire a much wider range of research talents in the near future.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
