Teaching university students co-creation and living lab methodologies through experiential learning activities and preparing them for RRI

Abstract

During the last decade, the living lab and co-creation concepts have started being blended with the Responsible Research and Innovation approach, aiming to evaluate potential societal anticipations toward fostering an inclusive RRI behavior. Teaching co-creation concept and living lab methodologies to university students has started been considered as valuable for future researchers along with the demand of companies and public sectors which turn toward user-center techniques for inspiration to develop innovative and services. To this end, the scientific publications presenting work on teaching co-creation and living lab methodologies are not so many while there are no published research studies on experiential learning activities for teaching co-creation and living lab approaches to university students. This study presents a course based on living labs and co-creation methodologies through experiential learning activities, consisted of four different lectures and an open event. The study involves stakeholders from the academia, the citizens, and the public sector. The results show that lectures with the participation of end-users were the most enjoyable. Furthermore, students thought that they learned the most when they first met the end-users. This lecture was perceived as a successful way to gain methodical knowledge for user-centered design and software development.

Introduction

A living lab is a virtual or physical place where various stakeholders collaborate together to address real-life issue and co-create solutions for societal challenges, technologies, services, and products. ¹ Ensuring the Quadruple helix stakeholder engagement, they bring together stakeholders from public institutions at the level of cities or regions and local, regional and national, private organizations, as well as academia and citizens. The European Network of Living Labs (ENoLL), an umbrella organization for living labs, gives the definition as “user-centred open innovation ecosystems based on a systematic user co-creation approach, integrating research and innovation processes in real-life communities and settings” (openlivinglabs.eu/aboutus). Openness (gathering many stakeholders from various domains with various expertise and competence), continuity (establishing trustful long lasting relations between stakeholders), empowerment (enabling users to actively be engaged in the innovation process), realism (involved with real-users in real-life settings during the development of the innovation), and spontaneity (detecting and analyzing emerging need and ideas of stakeholder) are the five Living Labs’ principles based on description.^2,3

During the last decade, the living lab and co-creation concept have started being blended with the Responsible Research and Innovation (RRI) approach, aiming to evaluate and detect potential societal implications and anticipations toward fostering an inclusive RRI behavior. Responsible Research and Innovation has been introduced as a new way of research implementation, providing tailored answers to social and environmental challenges. ⁴ RRI is meant to support the researchers, citizens, business, third sector organization, policy makers, and other stakeholder to collaborate throughout the whole research and innovation process toward aligning the outcomes with the values, needs, and expectations of society.

The last years, a variety of funding schemes on RRI Action calls, highlighted the importance of co-creation and RRI approaches in the implementation of the projects. More and more calls on research have as objective the participatory design, including phrases such as “It is crucial that users are involved and drive the innovation at all stages of design and development, including user acceptability, satisfaction and impact in realistic settings.” or “Evidence of user-centred design and innovation” (SC1-PM-15-2017 H2020 call). In this context, research projects exploring the intersection of RRI and Co-Creation through living lab experiential activities, have already been funded (e.g. SISCODE H2020 project).

Over the past 10 years, Living Labs have come to be considered as key and effective research infrastructures, where humans (young people, older adults, patients, etc.) are at the center of the study, ¹ as essential for any ecologically valid research. ⁵ Despite the growing research interest to living labs over the last years, ⁵ there are many aspects that remain underexplored while their essence could be considered as unclear to many.

The competencies provided by Higher Education curriculums are becoming more complex and demanding as the professional and personal requirements of academic and scientific staff are changing. Higher education institutions tend to focus more and more on providing effective learning, ⁶ and the importance of including co-creation activities has become clear. ⁷ There is a demand for this as companies and public sectors ⁸ increasingly turn toward user-center techniques for inspiration to develop innovative products and services.

While there are scientific publications exhibiting the results of students being involved and engaged in the design of the curriculum of the universities, ⁹ the scientific publications presenting work on teaching co-creation and living lab methodologies are not so many. ¹⁰ In one of the cases, the co-creation course was based on three sessions focusing on ethnographically inspired methods and six meetings dealing with particular methods from the field of participatory design, all of them conducted among the students. Experiential learning activities and learner-centered approaches have shown strong evidence of generating better learning outcomes. ¹¹ To the best of our knowledge, there are no published research studies on experiential learning activities for teaching co-creation and living lab approaches to university students.

Our study presents a course based on living labs and co-creation principles and methodologies through experiential learning activities, consisted of four different lectures and an open event. The evaluation results on students’ feedback are also presented. The study involves stakeholders from the academia, the citizens and the public sector as well as the private sector. This paper is structured as follows. Following this brief introduction, we describe the methodology followed in the second section, including the course schedule and material, as well as the evaluation tools. Next, we present the results of the quantitative and qualitative analysis. The final section concludes this study by pointing out its implications and limitations.

A course on co-creation

The course was designed by the Thessaloniki Active and Healthy Ageing Living Lab (Thess-AHALL) ¹² which is established and governed by the Lab of Medical Physics in the Medical School of the Aristotle University of Thessaloniki in Greece. The goal was to create a first prototype course on teaching university students the co-creation and living lab methodologies through experiential learning activities, based on the Kolb’s Experiential Learning Model. ¹³ This model defines four stages of the Experiential Learning as a cyclical process (cf. Figure 1) including the Concrete experience, Reflective observation, Abstract conceptualization, and Active experimentation. Concrete experience learning relies on open-mindedness and adaptability of the learner and is acquired through encountering of new or existing experiences, in Reflective observation the person who learns is observing the environment and trying to understand different perspectives, in Abstract conceptualization logical analysis of ideas and impartial understanding of a situation drive the learner, and lastly in Active experimentation the person tests the theories and learns by doing and applying acquired skills.

Figure 1.

Course methodology and its relation to the Kolb’s Experiential Learning Model.

Course overview

In this section, an Overview of the designed course methodology is provided (cf. Figure 2). It was designed as a short course for engineering students and was composed of four lectures span over four consecutive days, taking place for approximately 2 h each. The course was compulsory, and all students received a final grade. The course output as defined by the facilitators was to design and implement serious games web applications for Parkinson patients that would help them improve some personal capacities (e.g. memory, attention) that will assist the on facing the disease’s symptoms. The envisaged learning outcomes was to experience and understand the value of co-creation and user-centered design. To increase the students’ engagement level, the incentive of having their work exposed to the city citizens through a public event was announced at the beginning of the course.

Figure 2.

Overview of the four course lectures.

The Concrete experience stage of the Kolb’s Experiential Learning Model was the main part of the first lecture which was an introductory lecture on the living lab and the co-creation concepts through concrete examples of Thess-AHALL experience and activities using presentations, and traditional lecture methods (theory). The living lab experts (course facilitators) shared practical tips and techniques and allow the students to be introduced in a new concept that requires the ability to adapt to change. The Reflection observation stage took part in the same lecture in which the students have an active discussion based on the experience the living lab experts shared with them. The students tried to learn the experts perception, understand their point of view, and conceive their own ideas by maintaining a critical thinking. The Abstract conceptualization stage followed in during the second lecture, the students had the time to understand the situation, put their ideas into paper, and create their course of action. They created a plan on the possible interaction that they are going to have, before meeting some Parkinson disease patients’ hereafter mentioned as end-users. The meeting lecture was the beginning of the Active experimentation, during which the students discussed with the end-users on what serious games they would like to play (requirements elicitation methods). At the third lecture (hands-on), the students were divided into teams in order to start developing the games based on the requirements that they have gathered. The programming tools were also presented to them (implementation). After 1 day of designing and developing their solutions, the students presented the serious games they have developed to the end-users that evaluated the outcome and provided feedback from their perspective (review). Finally, the course was concluded with the evaluation day where the students went through the final evaluation procedure as well as an open discussion about the presented course. Considering that students have achieved some learning experience about the co-creation methodology, the final step lead the back to the Concrete experience stage with the reinterpretation of the acquired experience which could allow a new learning cycle to being.

Course schedule

Lecture 1: Introduction on the co-creation and living lab methodologies

The first lecture introduced the theory, the concepts, and the definitions of co-creation, participatory design, living labs, and RRI. A 2-h lecture through presentations of theory and use cases relevant to the topic following the traditional lecture methods. Given that the target of the course was the development of an application, this lecture went through the history of the mobile apps from the very early stages until today as well as the expected emerged technologies in the future. Emphasis was given to co-creation and the need of involving end-users in all stages of exploration, experimentation, and evaluation of a solution based on the reasoning made by Schuurman et al. ¹⁴ As both of the teacher have applied co-creation in the research domain, examples and experiences on applying co-creation in real life case where presented to the students. In order to increase students engagement, the traditional lecture was enhanced with interactive lecture techniques such as small breaks for open discussion and brainstorming. ¹⁵ At the end, the students were introduced with the next lecture’s content where they would meet end-users in order to understand their needs. A small reflection session on how they would interact with them, based on the experience the teachers shared with them, concluded Lecture 1.

Lecture 2: Meeting the end-users and getting requirements by interviewing them

The second lecture was designed as a requirements elicitation tasks. Three end-users, members of living lab’s Collaboration and Research community for the independent Living (composed mainly of older adults and patients), were invited to join an open discussion with the students. ¹⁶ The aforementioned Community is meeting on a frequent basis and performs co-creation activities, so the invited end-users were familiar with the methodologies and procedures. They were not given any instructions before the lecture in order to elicit a spontaneous response and discussion on the students ideas. The students were informed of any symptoms the Parkinson patients might experience during the discussion. They were encouraged to ask any questions they think would be useful for designing their solutions as well as any other question that would help them understand the end-users’ real-life needs. After a 30-min discussion and introduction of the patients and the students, the discussion was around serious games for Parkinson’s patients, what symptoms could be alleviated in their daily life if specific tasks were introduced through the serious games as well as what the reward for winning such a game could be. At then end of the discussion it was announced to the students the reward of the best selected serious game (see Incentive sub-section).

Lecture 3: Hands-on workshop and implementation

The third lecture was organized as a hands-on workshop on designing and developing web applications. Initially, 14 short videos with serious games storyboards were presented as an example to the students. These storyboards were designed for Parkinson disease patients during the i-Prognosis H2020 project ¹⁷ and presented as a means of enhancing their understanding and boost their ideas. The webFitForAll ¹⁸ platform was used as the tool for developing the online serious games. webFitForAll supports simple software development requiring low programming experience level and easy integration in the final web platform. The development was based on a presented template and some provided examples. At the end of the lecture the students were divided into five teams. Each team had to work on a serious game design and implementation focusing on the discussion they previously had with the end-users. The programming experience level of the majority of the students did not allow them to progress beyond simple button functionality, play sound, and hide/show elements.

Lecture 4: Delivering the serious game to the end-users

It the last lecture, the students presented the serious games they designed and developed to the invited end-users for evaluation. Each team was given 10 min to present and explain the serious game and to answers to any potential questions. The end-users graded the presented games with a score 1–10. This score contributed by 60% in the final grade of the course while the other 40% was assigned by the course facilitators for the quality of presentation, quality of design and development, innovation level of the idea, and team spirit.

Evaluation

For the evaluation of the proposed course, a five-item questionnaire was designed. The five items were (i) how much did you enjoy the lecture?, (ii) how much did you learn from the lecture?, (iii) how difficult did you find the lecture?, (iv) how well was the lecture prepared?, and (v) how well did the lecture introduced the co-creation concept? All the questions were scored from 1 to 5. The questionnaire included a question on how the students would rate their programming skills. Finally, the evaluation was concluded with an open discussion on the overall course.

Pilot

The first prototype course on teaching university students the co-creation and living lab knowledge through experiential learning activities was piloted during the “Course on Technology” organized by the Board of European Students of Technology (BEST) (https://www.best.eu.org/event/details.jsp?activity=k3vj7ra). BEST is a non-profit organization, organizing activities where students of technology from member universities can join to complement their fields of study in order to learn and take their first steps into an international career. A BEST “Course on Technology” targets on certain field of technology about modern technical applications of technology and the latest research achievements. Twenty-two students coming from 14 countries participated in the course and went through all the four lectures and the evaluation. The Parkinson’s patients are member of the Parkinson’s Association of Northern Greece, part of the Thess-AHALL stakeholders community. ¹⁶ The first and the third lecture took place at the Engineering School of the Aristotle University of Thessaloniki while the other two at the premises of the Lab of Medical Physics, Aristotle University of Thessaloniki.

The students filled in the questionnaire the day after the last lecture. During the evaluation day, an open discussion with the students and the teachers for each of the lectures took place. This open discussion gave the opportunity for collecting valuable feedback. In addition, an open discussion with the real users, who participated in the course, took place. The real users were asked about their impression, any comments as well as if they were willing to participated in such a course again in the future.

Increasing the public awareness

In order to increase the engagement level of the students to all the activities, apart from the final grade, they were offered with the reward of having the best game presented to the public through a social event on the World Parkinson’s Day that was already planned. The intention was also to show them how their work (developed games) can reach a wider audience beyond the primary stakeholders. Furthermore, the event was aiming to increase awareness of the general public for the Parkinson’s disease and reduce the cultural discrimination of PD patients. Thess-AHALL, continuing the “Play for . . . serious games campaigns in the social innovation perspective, organized a 2-day campaign called ‘Celebration on the streets’ for the World Parkinson’s Day.”

A play kiosk was placed in the most crowed place of Thessaloniki, the Aristotelous square. Parkinson patients, friends of the association as well as all people passing by were informed for the disease through leaflets and invited to “Play for the Parkinson’s Association.” A large TV set, a Kinect and the webFitForAll (part of the LLMCare service^19,20) were placed inside the kiosk and volunteers were asked to interact with the winning game and some pre-existing games of the platform. Each volunteer interacting with the games, was donating 10 points to the Parkinson’s Association, aiming to reach the target of 1000 points that will reward the Parkinson’s Association with prizes from the campaign’s sponsors. The target of the 1000 points was reached in the evening of the second day. The use of interactive games specially designed for PD patients, increase the public interest for the event and the kiosk. In that way, people passing by stopped to donate some of their time to the Parkinson’s Association and contribute to the collection of the points. That in terms made the interaction with the patients and the Association more active and meaningful.

Results

The evaluation questionnaire was used to measure the perceived levels of enjoyment, learning effect, difficulty, lecture preparation level, and lecture co-creation introduction for each of the four lectures. The matrix table in Figure 3 presents the histogram of the scores along with the average value for each question for each lecture. The lecture where the students met the real users to get requirements was scored as the one that they enjoyed more, they learned more, the less difficult, the best prepared lecture as well as the one introduced the co-creation concept better than the others. The fourth lecture, where the students delivered the games to the real users, was rated second both in levels of enjoyment and in introducing the co-creation concept.

Figure 3.

Questionnaire evaluation results. Each lecture was evaluated against five questions.

When the students were asked to rate themselves based on their programming skills, half of them rated themselves as beginners while the other almost half rated themselves as intermediates.

During the open discussion with the students, many of them shared their reflections on the lectures’ structure and content as well as on the understanding of the co-creation and RRI concepts. Most of the students were excited for meeting real users and working on solutions for them. For instance, one of the students stated:

I hadn’t realized the power I have in my hands, due to the university knowledge, to help people in need, like the Parkinson’s patients.

Similarly, another student said:

I enjoyed discussing with them in order to understand what they think, what they like and what they don’t. It is my first time meeting and discussing with a patient.

As came out from the discussion there were some prejudice about patients with Parkinson’s disease that could lead to misbeliefs about their needs. A student mentioned:

When we were told that we would meet some Parkinson’s patients, we didn’t know what we would face. We had in mind that they would be sad people with adverse symptoms. This is why we were very quiet when the patients entered the room. As soon as we realized that the patients are normal people, with sense of humor, we enjoyed it. They allowed us to ask any kind of question for their daily life, beyond the serious games we had to work on.

During the open discussion we introduced some questions to elicit also information about what they understood about the co-creation and RRI and their value on designing solutions for people. The majority of the students expressed that they understood the value of involving real users early in a project as well as discussing with them frequently. A representative answer of a student was:

I would never have thought that simple games, such as the ones we designed, could be of value for Parkinson’s patient, if we hadn’t discussed with them before designing. Even the feedback we received during the competition (lecture 4) could be used to improve more the games.

Finally, the students said that it was very important for the design of the game to knew what the end-users would like to be includes as reward in the game. One request was to include content that has to do with traditional music and local celebrities. This worked as a motivation for course students to discuss with local students for information on local culture about traditional music, dance, and celebrities.

The team discussed also the invited end-users’ perception which was of equal importance. The Parkinson’s patients enjoyed the interaction with the students a lot and they expressed their will to be invited again in similar activities. They were excited to see that students and future researchers worked on solutions that could alleviate some of their symptoms. One of them said:

We know that it is not feasible that effective solutions for our problems could be introduced and used in our generation. But we are very positive to participate in activities like this as we might help the researchers and the future researchers find solutions that could be effective for the next generation patients.

When they were asked about the “Play for the Parkinson’s Association” campaign they told us that this year, where the serious games for the citizens were introduced, the pavilion was full of people. The previous years the pavilion was not attractive for people passing by and did not attract many visitors. They were glad that such a campaign achieved to attract people at the pavilion because they had also the opportunity to be informed about the disease and the association.

Discussion

The objective of this work is to present a course methodology and overview along with a case study example for introducing university students to co-creation and living lab methodologies through experiential learning activities. ¹³ Elsharnouby ²¹ argues about the value of student participation in their overall satisfaction with university experience and Tassone et al. ²² points out the need for Higher Education to prepare students to engage in RRI activities. The results from evaluation of presented course show that lectures with the participation of end-users were the most enjoyable. Furthermore, students thought that they learned the most from this course in Requirement elicitation lecture when they first meet the end-users. This lecture was perceived as a successful way to gain methodical knowledge for user-centered design and software development.

Qualitative results show that it was a newly introduced methodology to them which they found important for the personal and professional development. Greater collaboration between academic and research community and society can lead to the development of new personal capacities of operating, researching, and innovating. ²³ The implementation lecture was perceived as the least enjoyable and the most difficult part of the course and this can be attributed to the students lack of higher level programming skills. Despite the restricted programming knowledge, the serious games application delivered from the students received very positive comments from the end-users. Given the programming knowledge level of the students, the facilitators team provided easy to use tools for development of the applications and due to the restricted period for implementation the students were asked to focus on functional mock-ups and first prototypes that would satisfy the collected user requirements, rather than fully working solutions. They were allowed to use any tool beyond the tolls presented in lecture 3. That is why besides the limited programming skills, the students manage to deliver meaningful applications.

Contrary to similar studies where “real-users” were not involved in the course activities due to practical and conceptual reasons, ¹⁰ we manage to bring together members of a patients association with real needs, students, and citizens. This model fosters social innovation and RRI which are key factors in the research domain. The results of the work presented in this paper exhibit the promising value of the close collaboration among the universities and the living labs in fostering the future researchers.

The structure of this course was also helpful in order to reflect and experience the four RRI process dimensions. Although it was the first encounter with the concept of co-creation and RRI, students seem to gain meaningful understanding RRI process dimensions.

1. Diverse and inclusive: The students experienced the involvement of stakeholders in early stage of the deliberation in order to get higher quality knowledge of their needs. The further broadening of the range of actors could yield even better results in terms of democracy in expertise and perspectives.

2. Anticipative and reflective: It is important that students understand the impact as well as the assumptions they do during the research and innovation processes. Planning their strategy in the Abstract conceptualization phase and then reflect on their initial assumptions make them experiment actively with the anticipative and reflective dimension that RRI should have.

3. Open and transparent: For the first time in their academic journey, the students had to communicate their results with the public, in two stages, first the evaluation and feedback phase from the end-user’s and second in the public demonstration event in World Parkinson’s day. They experienced the significance of communicating research and innovation in a meaningful way and enabling public discussion.

4. Responsive and adaptive to change: Due to the short duration of this course, this RRI dimension was not covered. However, future versions of this course will include more iterations enabling students to modify aspects of their results to align with changing knowledge and the needs expressed by the end-users.

Limitations and future work

The time constraints of the course that was organized by the BEST community entailed some limitation in the course methodology. It could be further enriched by structuring the schedule including distinct steps of design thinking, ²⁴ namely: empathize, define, ideate, prototype, and testing. The empathize and define session coincides with the requirement elicitation session where the students get to know the end-users and identify the basic problem, they are called upon to solve. Ideate and prototype should take place in the third Implementation lecture, while the active involvement of the end users in the design phase. Based on Kolb’s Experiential Learning Model, ¹³ all the lectures were part of the four stages model while the whole course could be considered as the students experience that would begin a new Experiential Learning cycle.

Furthermore, current implementation of the course involved only the end-users as key stakeholders in the requirement elicitation and feedback gathering. This is justified by the fact that the applications produced by the students were meant to be used only by patients. For more complex targeted applications, the involvement of other stakeholders should be investigated. For example, another useful approach would be to target the family and informal caregivers of patients, who are usually undertaking the burden of care, and ask the students to design directly an application for them. Moreover, family and other informal caregivers can be engaged in later steps of the course in order to get additional feedback and insights on the patients’ everyday lives and routines.

The involvement of main actors of the Quadruple Helix model will also be considered in future implementations. This could be attribute to the power of the living labs, which adopt the Quadruple Helix model where the “the General public” is the fourth helix of the “classical” three actors, Academia, Industries, and Government. ²⁵ Academia and General public are the two main actors involved in our approach. Industries and SMEs could be involved in the implementation step to support service and product development but also in later steps to connect students with business ideas. The Government actor, including different government levels and agencies should further assist the open innovation with financing, advisory, and public administration service. This generalized process will not only introduce co-creation but also make students experience the open innovation process which is key target of Living Labs.

The continuation of the involvement of four actors of the Quadruple Helix model is of primary importance for the sustainability of the Living Lab and the effective education of students regarding RRI. Thess-AHALL envisages a strategic pathway for continuous inclusion of stakeholders through a series of events. Such event will take place in various places, including the University campus, museums and cultural buildings of the city in order to engage the local community, the Academia, and the public authorities. Thess-AHALL also envisages to further involve the General public in the Academia and academic education by adopting the concept of patient as educators which exists for several years in health professions as medical education was always patient centric. A structured methodology for implementation of curriculum courses in Academia, not only in medical education, could improve the acquired skills for RRI. This is in-line with the sixth key policy of European Commission for RRI, the science education. ²⁶ In that ways, Thess-AHALL builds a local ecosystem that can be actively engaged in science education providing various benefits.