Sage Journals: Discover world-class research

Abstract

The development of novel and useful ideas is a process that can be described in multiple steps, including information gathering, generating ideas and evaluating ideas. We evaluated a university course that was developed based on design thinking principles which employ similar steps. Our results show that the course was not effective in enhancing students' creative self-efficacy, the belief that they are able to generate novel and useful ideas. Comparison to previous training approaches lead to conclusions on how to improve the course, and to implications for future research.

Keywords

Creativity evaluation non-equivalent dependent variable design pre-test/post-test self-efficacy

Creativity, the development of novel and useful ideas, is important in today’s society. Employees in different occupations are required to solve problems creatively, and to contribute to their employer’s success by contributing suggestions on how to enhance efficiency and save costs. An individual characteristic that promotes creative outcomes is creative self-efficacy, the belief that one is able to develop novel and useful ideas (Tierney & Farmer, 2002). Creative self-efficacy has been linked to a range of favorable outcomes, including the development and implementation of ideas (Hammond, Neff, Farr, Schwall, & Zhao, 2011).

We evaluated the effect of a university course at a German mid-sized university developed based on design thinking principles (Brown, 2008). The course was structured according to the steps of the design thinking process: understand – observe – point of view – ideate – prototype – test (Plattner, Meinel, & Weinberg, 2009). We expected that creative self-efficacy in this course will increase because design thinking involves engagement in different phases, similar to the creative problem solving process (Maier, Streicher, Jonas, & Frey, 2005). This process starts when a problem is discovered that needs to be solved. In the next step, information is gathered to better understand the problem in all its details. After that, diverse solutions to problems are developed which are then judged for their appropriateness for the problem at hand. Finally, the solution is implemented to solve the problem at hand. The design thinking process follows the same logic: a problem first needs to be understood before ideas are developed. Compared to the creative problem solving process, the solution to the problem receives more attention as the development of a prototype is an integral part of the process (Brown, 2008). However, the basic approach of separating information gathering from generating ideas, and generating ideas from evaluating and testing ideas was kept.

Although creative self-efficacy has repeatedly been linked to creativity in previous studies (Hammond, et al., 2011), little research has examined the development of creative self-efficacy over time. Gist (1989) showed that a course oriented at behavior modelling was more effective in developing creative self-efficacy than a course consisting solely of a lecture and exercises. Mathisen and Bronnick (2009) showed that a course consisting of a lecture about creativity research and exercises using creativity techniques was effective in developing creative self-efficacy in students, teachers, and employees. An extended course including planning how to facilitate creativity in a group was slightly more effective. Finally, Tierney and Farmer (2011) demonstrate that the change in creative self-efficacy over time was predicted by enhanced supervisor' expectations concerning creativity and learner’s creative role identity. Although not specifically concerned with creative self-efficacy, meta-analytic evidence suggests that creativity training based on creative process models is successful in developing attitudes towards creativity and divergent thinking skills (Scott, Leritz, & Mumford, 2004).

The four sources of self-efficacy identified by Bandura (1997) are mastery experience, vicarious experience, verbal persuasion, and (low) physiological arousal. We expected the course to enhance students’ creative self-efficacy because three of the four types of experiences are likely to occur: Students experience that they are able to come up with novel solutions in class (mastery experience). Furthermore, the structure of the design thinking process helps to identify several tasks that make the achievement of creative outcomes more manageable (mastery experience). In addition, students can observe other team members during their regular work or other teams during their presentations in the test phase being successful (vicarious experience). Finally, they are supported throughout the process by the coaches (verbal persuasion). Concerning physiological arousal, it is hard to make any general assumptions, because coaches reported that some students felt anxious and nervous because of the final public presentation of their prototypes. Thus, based on the course design and previous research, we expected the course to enhance creative self-efficacy over time.

The aim of the course is to teach a structured way of developing new ideas and solutions to real problems, and it consists of a lecture about the design thinking process, a presentation on creativity techniques, and several hands-on experiences. The course was structured according to the steps of the design thinking process: understand – observe – point of view – ideate – prototype – test (Plattner et al., 2009). The design thinking steps were adapted for this course, and some were subdivided into several steps. For each of the steps, several activities are planned and experienced. For example, in the understanding step, which targets increasing awareness of real problems, detection of potential topics for idea generation, and team building, students discuss relevant problems in society to choose a topic for their team to work on; for example, in one group students discussed ways to improve the daily life of disadvantaged people like visually handicapped people. The students also get to know each other and the background and preferences of each member in class by using group-dynamic lineups. To get to know their respective knowledge and expertise, students discuss their mutual expectations concerning task-relevant knowledge. This is a pre-condition for later team building (e.g., for building an interdisciplinary team for the topic of visually handicapped people who use a white cane). In the observation step which is concerned with a deepened detection of problem areas, students discuss information collection, and first structuring of the information. Available knowledge is discussed, and students plan how to generate further knowledge relevant to the problem. For example, the team described above undertook first research on technical aids used by visually handicapped people in their daily life beside the white cane. The point of view phase consists of the selection of a problem and meeting of potential users. Thereby, students learn to take the perspective of a potential user of the problem solution by developing a concrete image of this person including her/his name, age range, and previous experiences, and by interviewing potential users. For example, the white cane team learned that visually handicapped people of all ages still use the white cane to cope with known and unknown distances. They face a lot of problems like chewing gum machines which are situated on pavements, and which are not covered by the range of the white cane. The team member noticed that the interviewed visually handicapped people all used smart phones. In the ideation phase which includes idea generation and selection, students learn to apply creativity techniques, including brainstorming and analogy finding, and to select the most appropriate solution by using specific techniques. For example, the white cane team generated plenty of ideas based on research on nocturnal animals, and observation of car technologies like automatically parking by using sensor systems based on ultrasonic sound. In the prototype step, a preliminary model is built to be presented to the other teams, to potential users, and to experts. Furthermore, a business model canvas is developed that contains information on the key aspects of the selected idea. The white cane team developed an accessory unit for iPhones which is equipped with ultrasonic sound, and a distance meter. Based on feedback (test phase) the idea is refined and a new prototype is built which is finally presented to an external audience. All in all, the course takes about 36 hours of coursework plus a minimum of 30 hours of extra activities outside class. This course is an elective in about half of the university’s programs and is taught for students from different fields of study (Martin & Schäfer, 2016).

Using a non-equivalent dependent variable design (Frese, Beimel, & Schoenborn, 2003), we also checked the effect on attitudes towards teamwork as an additional outcome. Although teams were supported by coaches in their creative problem solving activities, no special emphasis was laid on developing effective teamwork. Thus we do not expect attitudes towards teamwork to change as a result of the course. Comparison between changes in intended (creative self-efficacy) and in non-intended outcomes (attitude towards teamwork) allows the ruling out of some potential alternative explanations of significant results (repeated responding, demand characteristics). We also evaluated the course by assessing participants’ reactions (e. g., satisfaction with materials, and satisfaction with course content).

Method

Sample and procedure. As part of the evaluation, 69 participants of 5 different classes in 2012 and 2013 provided ratings. They were on average 25.8 years old (standard deviation = 5.3). Among the participants, 39.1% were female. The majority of participants were business and economics students (30.4 %), mechanical engineering students (27.5 %) or students from product design (10.1 %). They received a questionnaire containing the measures at the beginning of the course (t1), and in the end (t2). Participants were reassured that their answers would be treated confidentially and would not have any impact on their course work. Responses were matched using a code generated by the participants. Matching t1 and t2 questionnaires resulted in 65 usable cases.

Measures. All items were rated on a six point scale ranging from 1 – not at all true to 6 – totally true. We measured creative self-efficacy with three items based on Tierney and Farmer (2002). This scale has been used extensively in previous research, and has been linked to various indicators of creative performance in employees (cf. Hammond et al., 2011). A sample item is “I have confidence in my ability to solve problems creatively.” Cronbach’s alpha was 0.78 at t1 and 0.87 at t2. These values are comparable in size to those reported by Tierney and Farmer (2011; α_t1 = 0.74; α_t2 = 0.81) but somewhat lower than in Tierney and Farmer (2002; α = 0.83 in manufacturing, α = 0.87 in operations).

Attitudes towards team work were assessed using three items taken from the psychological collectivism questionnaire developed by Jackson, Colquitt, Wesson and Zapata-Phelan (2006). Specifically, the three items target the preference for teamwork which predicts supervisor ratings of team member job performance (Jackson et al., 2006). A sample item was “I prefer to work in groups rather than working alone.” Cronbach’s alpha was 0.87 at t1 and 0.92 at t2.

Reactions to the course were assessed using 10 items targeting the degree to which participants were satisfied with or liked the course content, material, and external conditions taken from an established instrument (Staufenbiel, 2000). Staufenbiel (2000) provides evidence of validity. A sample item is “The seminar had a clear structure”. The Cronbach’s alpha was 0.89. Participants also rated their satisfaction with their team using three items based on validated job satisfaction measures (Neuberger & Allerbeck, 1978; Warr, Cook, & Wall, 1979). A sample item is “Overall, I am satisfied with my team.” Cronbach’s alpha was 0.89.

Participants were also asked to note down what they specifically liked or disliked about the course using an open-ended question.

Results

On average, participants were satisfied with the content, delivery and material of the course. Their mean rating reached a 4.43 on a six-point scale, indicating that there was room for improvement (see Table 1).

In response to the open question about what they liked or disliked, they frequently mentioned the good atmosphere. Furthermore, 9 of the 38 participants who provided answers to an open question about what they particularly liked about the seminar mentioned creativity in one way or the other: “Freedom and the opportunity for creative development”; “To develop creative ideas!”; “We were inspired to be creative, but in the end most ideas were not really innovative”; and “The idea of this course, the complete freedom in the development of ideas, independent of the area of studies is rare in my study program”. 6 of the 38 participants mentioned lack of time or the amount of work as something they disliked. This might be one reason why students were not completely satisfied with the course.

The comparison of t1 and t2 creative self-efficacy revealed that participants’ creative self-efficacy did not increase significantly over time (t = 1.53; p = 0.13, d = 0.23). Furthermore, attitudes towards group work remained stable (t < 1.0, p = 0.76, d = 0.02) over time.

To explore who of the participants benefitted most from the course, we calculated the increase in creative self-efficacy as the difference of the t2 and t1 measures, and correlated this increase in creative self-efficacy with all the other outcomes (see Table 2). Results indicate that the increase in creative self-efficacy was correlated with satisfaction with the course (r = 0.33, p < 0.01) and with the team (r = 0.32, p < 0.05) but not with attitude towards team work (r = −0.05, p > 0.05).

Table 1

Means and Standard Deviations of Main Study Constructs

	T1 Mean	T1 SD	T2 Mean	T2 SD	t	p	d
Satisfaction	–	–	4.43	0.78	–		–
Attitude towards teamwork	4.22	1.04	4.24	1.06	<1.0	0.76	0.02
Creative self-efficacy	4.53	0.71	4.71	0.86	1.53	0.13	0.23

N = 63 for satisfaction; N = 65 for matching ratings of creative self-efficacy and attitudes towards teamwork. SD: standard deviation

Table 2

Intercorrelations of Main Study Variables with Increase in Creative Self-Efficacy

		1	2	3
1	Satisfaction with course
2	Attitude towards group work	0.07
3	Satisfaction with team	0.14	–0.09
4	Increase in creative self-efficacy	0.33**	–0.05	0.32**

N = 62–65 due to missing values; ** p < 0.01.

Discussion

These results suggest that the course was not effective in fostering attitudes that have been linked to successful idea generation in previous research, namely creative self-efficacy. However, the effect size is similar in size to the meta-analytic results derived effect size for attitudes/behavior (Scott, Leritz, & Mumford, 2004). The effect size is somewhat smaller than that reported by Mathisen and Brannick (2009). In their course, the participants’ creative self-efficacy increased by d = 0.40 on average whereas our evaluation yielded an effect size of d = 0.23. Two notable differences emerge when comparing their training to our course. One difference is the extent to which the courses rely on team work. Their training consisted of three components, each comprising approximately a third of the course: (1) lecture, discussion and demonstrations; (2) training in creative processes; and (3) planning to facilitate a creative process outside the course (in the extended version only). Only the second component which demonstrated the use of creativity techniques was administered to groups of participants. In contrast, in our course, the majority of time was spent using group work. Furthermore, the lecture on creativity in this class was considerably shorter than a third of the course.

Cognitive load theory (Sweller & Chandler, 1991) suggests that learning is hindered by extraneous load during the learning process. It might be that the group format posed an extraneous load on the students which hindered the development of creative self-efficacy as the necessity of teamwork might have diverted some of the attention away from the key learnings of this course. Furthermore, it might be that problems with teamwork have occurred that prevented participants from having the kind of experiences necessary to enhance self-efficacy (Bandura, 1997). For example, when problems within the team occur, mastery experiences and vicarious learning might be limited, and arousal might have been elevated. The fact that satisfaction with the team was correlated with the increase in creative self-efficacy provides some support for this view. Based on these arguments, the course content and delivery could be improved by a stronger focus on creativity, and less involvement in teamwork. Based on the answers to the open question, one could conclude that the course could also be improved by allowing students more time for working on their idea.

Previous research also shows that the degree to which creative self-efficacy develops over time is dependent on the learner. Tierney and Farmer (2011) argued and empirically showed that creative role identity, the identification with the role of being creative at work, predicts changes in creative self-efficacy. It might be that our course participants differed in their creative role identities. Whereas some might have had a strong creative role identity and showed greater task engagement as a result, others might have focused on different aspects of the course instead. Because a greater task focus leads to more experiences necessary to enhance creative self-efficacy, the difference in creative role identity might have affected the degree to which individuals benefit from the class. Furthermore, different roles emerge in team work. Whereas some individuals might be the drivers of creative ideas, others might be responsible for the supportive functions (Furnham, Steele, & Pendleton, 1993). In future research, the creative role identity and team role of students as a predictor of their creative self-efficacy increase needs to be examined.

The generalizability of our findings is limited by the non-random assignment because the student self-enrolled in the course. It might be that students self-selected into the course because of their interest in creativity and thus the results might not be generalizable to all students. However, in the course announcement only limited reference was made to creativity. In contrast, the announcement stressed problem solutions and actions. Thus, a self-selection of students interested in creativity seems unlikely. The validity of our findings might be threatened by a lack of a control group, but the use of a non-equivalent dependent variable design, and the results concerning this variable mitigate this concern. Nevertheless, in future evaluation studies, a comparison to alternative elective courses would shed light on the question how to further improve the course design. A larger sample would be needed to test possible boundary conditions, for example, group processes or the degree to which individuals made critical experiences during the course. Furthermore, long-term effects of the training need to be assessed. It is an open question if creative self-efficacy will be enhanced over the long run and lead to creative outcomes inside and outside training. Taken together, more research is needed on the effect of training of creative self-efficacy.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biographies

Sandra Ohly is professor in business psychology at the University of Kassel, Germany since 2010. She received her PhD from the Technical University of Braunschweig, Germany in 2005, and her habilitation from the Goethe University Frankfurt, Germany in 2010. Her research focuses on creativity and proactive behavior at work, and in affective and motivational processes leading to well-being and performance. Correspondence: ohly@uni-kassel.de

Laura Plückthun is employed at a psychosomatic and orthopedic rehabilitation clinic since 2014. After graduating from the University of Gießen, Germany as a psychologist in 2010, she worked at the University of Kassel in the department of business psychology. Her research interests are in the areas of teamwork, diversity in work teams, creativity and innovation.

Dorothea Kissel is a lecturer in the Business Psychology Department at FOM University of Applied Sciences for Economics and Management, Frankfurt, Germany. She is committed to teaching and to the development of skills and knowledge of her students. Her teaching interests include psychological diagnostics, social psychology, and entrepreneurship. Her research interests include psychological diagnostics of entrepreneurial behavior and team work. Correspondence: dorothea.kissel@googlemail.com

References

Bandura

(1997) Self-efficacy: The exercise of control, New York, NY: Freeman.

Brown

(2008) Design thinking. Harvard Business Review 86(6): 84–92.

Frese

Beimel

Schoenborn

(2003) Action training for charismatic leadership: Two evaluations of studies of a commercial training module on inspirational communication of a vision. Personnel Psychology 56: 671–698.

Furnham

Steele

Pendleton

(1993) A psychometric assessment of the Belbin Team-Role Self-Perception Inventory. Journal of Occupational and Organizational Psychology 66(3): 245–257.

Gist, M. E. (1989). The influence of training method on self–efficacy and idea generation among managers. Personnel Psychology, 42, 787–805.

Hammond

M. M.

Neff

N. L.

Farr

J. L.

Schwall

A. R.

Zhao

(2011) Predictors of individual-level innovation at work: A meta-analysis. Psychology of Aesthetics, Creativity, and the Arts 5: 90–105.

Jackson

C. L.

Colquitt

J. A.

Wesson

M. J.

Zapata-Phelan

C. P.

(2006) Psychological collectivism: A measurement validation and linkage to group member performance. Journal of Applied Psychology 91: 884–899.

Maier

G. W.

Streicher

Jonas

Frey

(2005) Innovation und Kreativität. In: Frey

von Rosenstiel

(eds) Enzypklopädie der Psychologie, Themenbereich D, Serie III Band 6 Wirtschaftspsychologie, Göttingen, Germany: Hogrefe, pp. 809–855.

Martin, C., & Schäfer, E. (2016). Ideenwerkstatt MACHEN! - Entwicklung von Ideen und unternehmerischer Kompetenz im Ideenmanagement [Development of innovative ideas and entrepreneurial competence by design thinking]. In A. Jäckel, C. Kersting, & O. Sträter (Eds.), Zukunftsorientiertes Ideenmanagement - Anforderungen und Entwicklungstendenzen (pp. 173–192). Frankfurt am Main, Germany: Deutsches Institut für Ideen- und Innovationsmanagement.

10.

Mathisen

G. E.

Bronnick

K. S.

(2009) Creative self-efficacy: An intervention study. International Journal of Educational Research 48(1): 21–29.

11.

Neuberger, O., & Allerbeck, M. (1978). Messung und Analyze von Arbeitszufriedenheit: Ersahrungen mit dem Arbeitsbeschreibungsbogen (ABB) [Measurement and analysis of job satisfaction]. Bern, Switzerland: Rubber.

12.

Plattner, H., Meinel, C., & Weinberg, U. (Eds.) (2009). Design-thinking. Heidelberg, Germany: Springer.

13.

Scott

G. S.

Leritz

L. E.

Mumford

M. D.

(2004) The effectiveness of creativity training: A quantitative review. Creativity Research Journal 16: 361–388.

14.

Staufenbiel, T. (2000). Fragebogen zur Evaluation von universitären Lehrveranstaltungen durch Studierende und Lehrende [Questionnaire for the evaluation of university lectures by students and teachers]. Diagnostica, 46, 169–181.

15.

Sweller

Chandler

(1991) Evidence for cognitive load theory. Cognition & Instruction 8: 351–362.

16.

Tierney

Farmer

S. M.

(2002) Creative self-efficacy: Its potential antecedents and relationship to creative performance. Academy of Management Journal 45: 1137–1148.

17.

Tierney, P., & Farmer, S. M. (2011). Creative self-efficacy development and creative performance over time. Journal of Applied Psychology, 96, 277–293.

18.

Warr

Cook

J. C.

Wall

T. D.

(1979) Scales for the measurement of some attitudes and aspects of psychological well-being. Journal of Occupational Psychology 52: 129–148.

Developing Students’ Creative Self-Efficacy Based on Design-Thinking: Evaluation of an Elective University Course