Educating Boundary Crossing Planners: Evidence for Student Learning in the Multistakeholder Regional Learning Environment

Abstract

The regional learning environment (RLE) embeds planning student learning in a real-world participatory planning process. This study investigates the effectiveness of the RLE and its “boundary crossing” design characteristics with respect to student learning. The quasi-experimental study combines a quantitative competence test (N = 225) with qualitative student and teacher learning reports. The RLE, in general, stimulates planning students’ learning. “Working in multidisciplinary student groups” and “high coaching intensity” have specific added value for learning. “Intense student–stakeholder collaboration” does not significantly improve learning; however, qualitative data indicate this to be a powerful boundary crossing design principle of the RLE.

Keywords

authentic learning environment boundary crossing competence development multidisciplinary multistakeholder planning education

The Need for Effective Learning Environments in Planning Education

Professional planners need to cooperate and facilitate cocreation with a variety of stakeholders from different disciplinary fields and with different perspectives. Planners must be able to operate across different practices and, therefore, need “boundary crossing” competence (Akkerman and Bakker 2011; Walker and Nocon 2007). Consequently, planning education needs learning environments that effectively stimulate boundary crossing. Up to now, the effectiveness of learning environments in planning education from a boundary crossing perspective has not been demonstrated. This study investigates the effectiveness of the real-life multistakeholder regional learning environment (RLE), designed from a boundary crossing perspective, for planning students’ learning.

Scholars are well aware of the need to incorporate real-world planning practices in planning curricula to prepare future students for their profession (e.g., Angotti, Doble, and Horrigan 2011; Dalton 2007; Edwards and Bates 2011). Indeed, different authentic learning environments have been developed, that is, learning environments that represent the contemporary planning practice and that provide students with the opportunity to develop their professional competence in a real-life context (Newmann and Wehlage 1993). Well-known examples are the planning studio and service learning environments (Angotti, Doble, and Horrigan 2011; Long 2012).

Several studies have portrayed learning experiences in studios, service learning, and in mixed variants of these two, resulting in suggested guidelines for the educational design of these learning environments (e.g., Balassiano 2011; G. Harris 2004; Higgins, Aitken-Rose, and Dixon 2009; Sletto 2010; Winkler 2013). However, these studies provide only limited, anecdotal evidence for the effectiveness of these authentic learning environments in terms of student learning. More specifically, evidence lacks for the effect of typical learning environment design characteristics (e.g., working with multiple stakeholders) that represent the current planning profession, on student learning. Such evidence is needed to develop education programs that meet the demands of the current profession (Young 2009), as well as to support evidence-based design and pedagogy of authentic learning environments in planning education (Németh and Long 2012; Roakes and Norris-Tirrell 2000).

The multistakeholder RLE is a new authentic learning environment in Dutch planning education programs (Foorthuis, Lutz, and Rippen 2012; Meijles and Van Hoven 2010). A unique feature of the RLE is that student learning is embedded in a real multistakeholder participatory planning process, aimed at stimulating both student learning and “regional learning.” As such, the RLE represents the current planning profession (Albrechts 2013; Boelens and De Roo 2014). It requires students to “cross boundaries” between multiple disciplines and multiple perspectives.

This quasi-experimental study explores, in a mixed method pre- and posttest design, the effectiveness of the RLE for planning students’ learning. Moreover, the study analyzes whether “working in multidisciplinary student groups” and “working in close collaboration with multiple external stakeholders” enhance this learning. These two characteristics of the RLE are specifically meant to stimulate students to cross boundaries, and are therefore referred to as the “boundary crossing learning environment characteristics.”

The results of this study highlight the effectiveness of the new RLE for planning students’ learning in view of current professional requirements for planners (Dalton 2007; Frank et al. 2014). These results may guide the future design and pedagogy of RLEs and of various other authentic learning environments in planning education, considering their parallels. On the curriculum level, insights obtained in the study are relevant to develop planning curricula that show an increasing outbound focus, establishing long-term partnerships between universities and the outside world to build on social relevance and civil engagement (e.g., Balassiano and West 2012; Winkler 2013).

We will first outline the theoretical considerations that guided this study and explain the importance of boundary crossing in the professional practice of planners. Next, the educational design of the RLE will be described, including how boundary crossing is explicitly stimulated in the RLE.

Coevolutionary Planning and the Rising Need for Boundary Crossing Planners

The context in which planning operates has changed considerably over the past three decades. Society has become increasingly globalized and networked, and faces complex issues with unpredictable changes in land use systems through multiple scales. Planning processes worldwide have transitioned from an exclusive governmental affair to complex, coevolutionary multistakeholder processes (Albrechts 2013; De Roo and Silva 2010). Planning as a profession responded, and still responds, to these changes by moving beyond more structuralist planning approaches, toward developing and applying approaches that try to address the “wicked” character of current issues and processes (Albrechts 2013; Boelens and De Roo 2014; Booher and Innes 2002; Boonstra and Boelens 2011; Forester 1999; Healey 1997; Rittel and Webber 1973). All these approaches share the idea that tackling planning issues requires taking into consideration the perspectives of multiple actors with diverse backgrounds, values, and interests and who have multiple understandings and interpretations of reality (Albrechts 2013; Domingo and Beunen 2012; Healey 1997, 2003). Traditional planning and decision-making approaches that were commonly focused on achieving explicit goals gradually move toward working on “communicative ideals where planning actions enhance the achievement of consensus on perceived situations” (Boelens and De Roo 2014, 43).

Planners increasingly become participating mediators (De Roo and Silva 2010), also called “social entrepreneurs” (Boelens and De Roo 2014, 19), instead of the more technical facilitators from the times of governmental led blueprint and strategic planning practices (Healey 2003; Nienhuis, Van Dijk, and De Roo 2012). Nowadays, planners are expected to mediate the coevolutionary processes by sharing perspectives of multiple stakeholders and situations and supporting the cocreation of new communicative and situational knowledge. This mediating role requires new competencies of planners, and the development of those in their education. Alongside long-established technical, communicative, and facilitating competencies (Dalton 2007; Healey 1997), more mediating competencies become relevant for planners. Examples of these new competencies, recently identified by planning scholars, are being able (1) to quickly identify and understand multiple disciplines, cultural traditions, interests, values, and perspectives; and (2) to admit to and intertwine these; (3) to cultivate a safe, respectful and stimulating collaborative climate; and (4) to support the reflection on process, products, and performance and mutually learn from each other to achieve shared understanding and reasoning (e.g., Balassiano 2011; Domingo and Beunen 2012; Thomas 2012; Umemoto 2001).

The concept of “boundary crossing competence” encapsulates these new competencies and as such is a key competence of current planners. Boundary crossing competence is defined here as the ability to manage and integrate multiple discourses and practices across different sociocultural boundaries (Akkerman and Bakker 2011; Umemoto 2001; Walker and Nocon 2007).

The RLE is designed to explicitly stimulate boundary crossing in planning education. The next section describes the RLE in more detail and shows how its design reflects the boundary crossing planning practice. To position the RLE in the planning education landscape and explicate its expected added value, the RLE is compared to planning studios and service learning environments.

The Authentic Multistakeholder Regional Learning Environment

Regional Planning and the RLE

In the Netherlands as well as in other European countries, regions have become a subnational, supralocal focus point for spatial development (Haughton and Counsell 2004; Lagendijk 2001). In 2005, a regional partnership between nine municipalities, two provinces, two water boards, and several educational institutes in the Dutch Peat District (a region in the northern part of the Netherlands), developed and experimented with a regional learning arrangement to collaboratively face regional developmental issues. This experiment was the starting point for the further development of the RLE, supported by the Dutch Ministry of Economic Affairs (Foorthuis, Lutz, and Rippen 2012).

The RLE is meant to be a catalyst for “regional learning”—a learning and working community in which students, teachers, researchers, policy makers, members of NGOs, entrepreneurs, and/or citizens cooperatively face complex issues of regional spatial development while mutually learning (Foorthuis, Lutz, and Rippen 2012; Meijles and Van Hoven 2010; Scholz and Steiner 2015). The RLE has been established so far in thirteen Dutch, mostly rural, regions that are characterized by having high landscape and biodiversity values, recreational pressure, and economic and demographic decline. The RLE aims to facilitate the collaborative creation of new knowledge toward sustainable regional development. Educational institutions are always one of the partners in the RLE. Appendix A (Supplementary Material) provides a comprehensive illustration of the state of affairs in one of the studied RLEs.

Educational Design of the RLE

From an educational perspective, the RLE is an authentic, multistakeholder learning environment. Here, authentic refers to an instructional approach in which learning takes place in a real-life context to allow students to meaningfully construct knowledge and develop skills for real life (e.g., Herrington and Oliver 2000; Newmann and Wehlage 1993). Multiple stakeholders in this study are defined as a collection of persons or parties, from (semi)government, the business sector, NGOs, research bodies, and/or society, who either have a stake in the issue(s) at hand, and/or participate in the cocreation of knowledge (Freeman 1984).

The RLE is always typified by the following learning environment design characteristics:

The RLE’s general aim is twofold, namely (1) to support students’ and other parties’ learning in terms of both domain-specific expertise and professional skills development, and (2) to contribute to sustainable regional development.

Students are exposed to real trans-disciplinary problems in a real-world situation (Scholz and Steiner 2015); i.e., regional planning problems identified and commissioned by an external client. Working on the assignment engages students in authentic, complex tasks. Appendixes A and B (Supplementary Material) show examples of assignments that have been carried out in the RLEs included in this study.

Knowledge is collaboratively constructed between students, between students and their teachers, and preferably between students, teachers, and multiple stakeholders.

Students work in student groups.

Working in the RLE results in a realistic product that has value for the external client(s) and contributes to regional development. In practice, deliverables vary depending on agreements between the students, their teachers, and the external client(s). Next to the product for the client, process reflection reports are required as a deliverable. Product and process deliverables are both part of the assessment, but assessment criteria and procedures vary between RLEs.

The teacher’s role is to facilitate and/or coach the learning process rather than to transfer knowledge as an expert. Additionally, the teacher is a “learner” himself, working in an almost equal relationship with the students to collaboratively tackle complex regional problems.

The RLE preferably has two additional design characteristics that are expected to explicitly stimulate students to work and learn across boundaries between different disciplines and different perspectives (Foorthuis, Lutz, and Rippen 2012; Meijles and Van Hoven 2010):

Students work in multidisciplinary student groups, which means that the groups consist of students from different study programs, that is, disciplines.

Students collaborate intensively with multiple stakeholders to enable the inclusion of diverse perspectives and interests when solving complex, transdisciplinary regional problems.

These characteristics are referred to as the boundary crossing learning environment characteristics, upon which this study specifically sheds a light.

The RLE Compared to the Planning Studio and Service Learning

The RLE shares educational characteristics with the commonly used authentic planning studio, service learning environment, and mixed variants of these. However, the RLE characteristics purposely differ from and add to these existing authentic learning environments when it comes to explicitly addressing boundary crossing. Providing insights in similarities and differences between the characteristics of the RLE, studio, and service learning enables readers to use the results of this study for the further educational development of both the RLE and the planning studio (Higgins, Aitken-Rose, and Dixon 2009; Long 2012; Németh and Long 2012) and of service learning (Angotti, Doble, and Horrigan 2011; Roakes and Norris-Tirrell 2000; Sletto 2010).

For the RLE, planning studio, or service learning, no internationally standardized pedagogy has been described. However, all three learning environments contain elements of social constructivist learning theories (Vygotsky 1978), authentic learning (Herrington and Oliver 2000), and experiential learning (Dewey 1938). Students collaboratively construct knowledge together with other students, teachers, and society; they work on real-world assignments and learn through experience (Watson 2001).

The planning studio aims to expose students to a professional experience by introducing them to real-world problems in a quasi-real–world situation. Learning outcomes, although often not clearly described (Long 2012; Németh and Long 2012), focus on student learning in reference to the integration and application of theory into practice, and development of various practical skills (Frank et al. 2014; Németh and Long 2012). The planning studio mostly starts with an open-ended complex problem. The problem takes account of current issues in the real world, but is mostly described by the teacher, and not always offered by real external clients (Balassiano and West 2012). Students work individually or in groups. The planning studio is finalized with a final presentation to faculty and/or the client, sometimes but not necessarily with an authentic product that offers added value for the client. As a remnant from design studios, assessment often takes place through several rounds of formative assessments (jury-crits) involving presentations of students and feedback from diverse assessors (peers, tutor, and experts). Although the intention is to base assessment on absolute, criterion-referenced quality standards, standards are still lacking (Németh and Long 2012). The teacher fulfills the role of expert, providing a flexible though strong instructional frame. Recently, scholars have advocated a broader role of stakeholders and to run studio courses with mixed groups of students from various disciplines (Balassiano 2011; Long 2012).

We believe that the RLE adds three typical design characteristics to the current mainstream characteristics of the planning studio; that is, (1) students always work on real-world transdisciplinary assignments identified by and relevant for one or more external stakeholders; (2) students work in groups, preferably in multidisciplinary groups; and (3) students, their teachers, and multiple external stakeholders collaboratively construct knowledge and mutually learn.

Service learning is defined as a pedagogy that aims to integrate meaningful community service with formal education (Giles and Eyler 1994; Jacoby 2014). Next to the objective of strengthening communities, student learning outcomes focus on civic responsibility, critical problem solving skills, adaption to challenging and unexpected situations, and critical reflexivity (Roakes and Norris-Tirrell 2000; Sletto 2010). In service learning, students learn how to use the knowledge and skills from a specific course to provide service to society. Learning outcomes are in most cases closely connected to those of a related theoretical course. Students work individually or in groups on real-world, societal assignments that ideally are relevant for the community at the local or sublocal level (G. Harris 2004; Sletto 2010). The assignments are often, but not necessarily, multidisciplinary. Service learning allows students to work closely with community organizations on applied field projects. Knowledge construction explicitly takes place through structured reflective thinking, mainly between the students and their teacher. The final product provides added value to the community and is presented to the client. Assessment criteria and procedures are not commonly defined (S. C. Harris and Irazábal 2011). Teachers and students work in a master–apprenticeship model in which the teacher fulfills the role of facilitator of the process and reflection (G. Harris 2004; Roakes and Norris-Tirrell 2000; Schön 1987; Sletto 2010). In planning education practice, many different service learning courses have been developed and evaluated, for example, the service learning studio (Forsyth, Lu, and Mc Girr 1999; Winkler 2013) and the research seminar (G. Harris 2004). In these service learning variants, both faculty members and students maintain a close relationship with external organizations and deliver a formal community service report.

Service learning environments and the RLE share the characteristic of students working on a real-world assignment with actual relevance for one or more external parties. In both service learning and the RLE, contact with external parties is a prerequisite for a sufficient project result, and an oral presentation of the final product to client and/or other external parties is part of the assessment. However, RLE assignments always have a transdisciplinary character and are always demand driven, while in service learning they can also be launched for the purpose of a (service learning) course. In addition, RLE students preferably work in multidisciplinary student groups. So far, working in multidisciplinary student groups is not regarded as a prerequisite for service learning. The process of reflection to transform an experience into learning, and thus stimulate reflective practices, is, to date, less pronounced in the RLE, while it is a core element of service learning.

To sum up, the RLE adds to the planning studio and to service learning, a set of typical boundary crossing characteristics namely that students in the RLE (1) always work on a demand-driven, transdisciplinary assignment identified by one or more external actors, and (2) students always work in groups, preferably multidisciplinary student groups. Alternatively, the element of systematic, critical reflection to explicate and learn from experiences across the boundaries, an explicit characteristic of service learning, could add to the boundary crossing learning potential of the RLE.

Studying the Learning Potential of Regional Learning Environments

The new RLE is increasingly used in planning education in the Netherlands. However, still undiscovered is the effectiveness of the RLE in terms of students’ learning and/or competence development, and the actual added value of learning in multidisciplinary student groups and with multiple external stakeholders, that is, the two typical boundary crossing characteristics. Investigating the educational effectiveness of the RLE is important to confirm the hypothesized added value of this new learning environment, and add to the small record of systematic, large-scale, quantitative studies on the effectiveness of studios and service learning (Angotti, Doble, and Horrigan 2011; Long 2012).

This study addresses four research questions:

To what extent do RLEs stimulate planning students to develop competencies identified as relevant for working in an RLE setting?

Do the two boundary crossing characteristics of RLEs (multidisciplinary student groups and intensive student–stakeholder collaboration) enhance student learning?

What learning outcomes do teachers perceive as a result of working in multidisciplinary student groups and with multiple external stakeholders?

What preconditions do teachers perceive as necessary for exploiting the learning potential of RLEs?

Method

This study has been carried out in a quasi-experimental mixed method pre- and posttest design combining quantitative data on students’ competence development with qualitative student and teacher reports on learning outcomes of the RLE.

Participants

Five RLEs implemented in different planning education programs were monitored: three in academic study programs (n = 81, 64, 52) and two in professional higher education programs (n = 15, 13). Students in all the RLEs worked in student groups of mostly five or six students. Each student group worked on a different project assignment of which the results were meant to contribute to the development of the respective region (see Appendix B, Supplementary Material). Table 1 shows the general characteristics of the five RLEs and mean age and gender of the participating students.

Table 1.

Characteristics of and Developed Competencies in the Studied Regional Learning Environments.

	RLE 1	RLE 2	RLE 3	RLE 4	RLE 5
General characteristics
Number of students (n)	81	15	52	64	13
Educational level	Academic year 4 or 5	Professional higher education year 3	Academic year 1	Academic year 2	Professional higher education year 3
Study load	8 weeks full-time	16 weeks 2 days/week	2 weeks full-time	8 weeks 1 day/week	20 weeks 2 days/week
Student group size	5–7, mostly 6	6 or 9	6	5–6	3–4
Students’ mean age	25	22	20	21	20
Students’ gender, n (%)
Male	31 (38)	10 (67)	30 (58)	51 (80)	9 (69)
Female	50 (62)	5 (33)	22 (42)	13 (20)	4 (31)
Classification
Mono-/multidisciplinary groups	Multi	Multi	Mono	Mono	Mono
Active stakeholder collaboration	Yes	Yes	Yes	No	No
Coaching intensity	High	High	Low	Low	Low
Results
Amount and type of developed competencies as a result of paired sample t tests^a	All 9 competencies	5 of 9 competenciesDomain-specific professional expertiseDeciding and initiating activitiesCollaborating and discussingPlanning and organizingActing customer oriented	4 of 9 competenciesDomain-specific professional expertiseCreating and innovatingCollaborating and discussingInvestigating	3 of 9 competenciesDomain-specific professional expertiseCollaborating and discussingActing commercially	None
Effect size of competence development (in Cohen’s d^b)	0.3–0.67medium	0.69–1.29medium to large	0.32–0.97medium to large	0.39–0.64medium	Not applicable

p < 0.05.

d < 0.2 = small, d ≈ 0.5 = medium, d > 0.8 = large.

To answer research questions 3 and 4, twenty-five teachers participated in a semistructured workshop. All teachers were experienced in the development and implementation of RLEs, including the five monitored RLEs.

Classification of Students on Boundary Crossing Learning Environment Characteristics

All studied RLEs met the general educational characteristics. The RLEs differed regarding educational level, study load, size of the student groups, number of students involved (Table 1) and focus in content (Appendix B, Supplementary Material). To answer research question 2, individual students in every RLE were classified for the two studied boundary crossing design characteristics. A student was classified as “working in a mono-disciplinary student group” when he or she worked only with other planning students. A student was classified as “working in a multidisciplinary student group” if he or she worked with students from other study programs (e.g., landscape architecture, environmental sciences, and management and economics). A student was classified as “working on a low level of multistakeholder collaboration” if he or she only read information about the stakeholders and their opinions without contacting them personally or only asked the stakeholders informative questions that were answered without any discussion. A student was classified as “working on a high level of multistakeholder collaboration” if he or she discussed project-related issues with one or more stakeholders or worked together in collaborative working sessions with one or more stakeholders. The attribution of the classification for multistakeholder collaboration was based on three weighed ratings: (1) teachers’ observations of the level of stakeholder collaboration during the projects (rating low or high), (2) researchers’ observations of the level of stakeholder collaboration during the projects (rating low or high), and (3) students’ reported level of stakeholder collaboration in an evaluation questionnaire. In this questionnaire, the students ticked off how many stakeholders they collaborated with and how the collaboration took place (ranging from “finding information on the Internet about this stakeholder” to “collaborating with the stakeholder via personal contact during the whole project”). A scoring scheme translated these answers into six possible relationships for which scores 1–3 were classified as “low level of collaboration” and scores 4–6 as “high level of collaboration.”

Classification of Students on Coaching Intensity

Besides classifying for the above two typical learning environment characteristics, a classification for the independent variable “coaching intensity” was added. This was done as observations of the RLEs illuminated coaching intensity as an important varying variable between the RLEs. Students in RLEs who were classified as obtaining “low coaching intensity” only met their teacher coach once a week for a state-of-the-art group discussion mainly focused on the product and the process toward a final result. Students in RLEs who were classified as obtaining “high coaching intensity” followed an intense parallel coaching trajectory in which their learning experiences with collaboration across boundaries were explicitly addressed and utilized to optimize the RLE product and process. Both group and individual meetings with a teacher coach were organized purposely. Students’ classification for coaching intensity was based on the descriptions of coaching in the study manuals.

Measuring Competence Development

To answer research questions 1 and 2, a validated pre- and posttest questionnaire assessed the perceived level of domain-specific professional expertise and eight generic competencies (Table 2).

Table 2.

Competencies as Assessed in Pre- and Posttest.^a

No.	Competency
1	Domain-specific professional expertise
2	Deciding and initiating activities
3	Showing attention and understanding
4	Collaborating and discussing
5	Investigating
6	Acting commercially
7	Creating and innovating
8	Planning and organizing
9	Customer-oriented acting

For a description per competency, see Appendix C (supporting information).

This total of nine competencies was identified as relevant for working in RLEs by (1) a group of fifty-six academic planning students working in an RLE project that was used as a pilot project for the setup of this study (June 2011), and (2) ten planning teachers experienced in working in RLEs, including the five monitored RLEs. Their selection was based on a list of twenty-five generic competencies for Dutch vocational education (COLO 2006) as developed on the basis of the SHL Universal Competency Framework (Bartram 2011; www.cebglobal.com). Students and teachers were asked to rate each of the twenty-five competencies on relevance for working in the RLE on a four-point scale (1 = certainly not relevant to 4 = certainly relevant). Nine competencies, including “domain-specific professional expertise” were convincingly rated as certainly relevant by the respondents. For this reason, these nine were selected for the purpose of this current study. The questionnaire, part of the validated competency measurement instrument COM (Khaled et al. 2014), consisted of a description of each of the nine competencies (supporting information Appendix C) and four to six performance indicators per competency derived from this description. In both pre- and posttests, students awarded themselves a score for each performance indicator on a ten-point scale. A competence mean score was based on students’ rating of the four to six performance indicators per competency. At the start of the project, directly after being informed about their project assignments, students filled out the pretest. At the end of the project, right after the final presentation of the project result, they filled out the posttest. The scales were reliable (α > 0.80). RLEs were compared based on their development scores between pre- and posttest (dependent variables).

Measuring Students’ Other Learning Outcomes

To answer research question 2 in terms of “other learning outcomes,” the posttest asked students the open question “What did you learn more from your RLE project? Please write down as many of your ideas as possible regarding your learning in this project.”

Measuring Teacher Perceptions on Student Learning in the RLE

During a semistructured workshop, teachers first individually and then in five groups of four to six participants wrote down experienced learning outcomes typically resulting from (1) working in multidisciplinary student groups and (2) multistakeholder collaboration in RLEs. Every statement was individually written on a Post-It to allow coding and counting. Additionally, teachers wrote down statements regarding what they experienced as preconditions for optimal learning in the RLE.

Analysis

Paired sample t tests were used to calculate development of the students per RLE on the nine competencies (research question 1). Effect size for the paired sample t tests was measured in Cohen’s d with d <0.2 showing a small effect, d around 0.5 showing a medium effect, and d >0.8 showing a large effect (Cohen 1988). Three multivariate general linear models (GLMs) compared competence development across RLEs using mono-/multidisciplinary student groups, low/high multistakeholder collaboration, and low/high coaching as independent variables (research question 2, quantitative part). Effect size for the GLMs was measured in partial eta-squared (partial η²), with partial η² ≈ 0.01 showing a small effect, partial η² ≈ 0.06 showing a medium effect, and partial η² ≈ 0.14 showing a large effect (Cohen 1988).

The reported learning outcomes from the students (research question 2, qualitative part) were deductively coded on “referring to working in multidisciplinary student groups,” “referring to collaborating with multiple stakeholders,” or “referring to other learning environment characteristics” (Miles, Huberman, and Saldana 2014). The interrater reliability kappa (κ) of this coding step was 0.85 (two independent raters), which represents an almost perfect strength of agreement (Landis and Koch 1977). After coding, the percentages per code were calculated for students that worked in either mono- or multidisciplinary student groups or with a low or a high level of stakeholder collaboration (i.e., the two typical boundary crossing learning environment characteristics). Finally, two illustrative examples of learning outcomes per category were chosen.

The statements resulting from the teacher workshop were first coded by the same two independent researchers that coded the student learning outcomes. They used deductive coding (Miles, Huberman, and Saldana 2014) on learning outcomes resulting from either multidisciplinary group work or multistakeholder collaboration (research question 3), and open, inductive coding (Glaser and Strauss 1967; Miles, Huberman, and Saldana 2014) on preconditions for utilizing the learning potential of RLEs (research question 4). They clustered the codes into meaningful learning outcome and precondition categories, after which axial coding was used (Strauss and Corbin 1998) with an interrater reliability к of 0.88. Additionally, learning outcome statements were subcoded (Miles, Huberman, and Saldana 2014) as representing one of the nine competencies as measured in the pre- and posttest or “another learning outcome.” The interrater reliability к of this coding step was 0.84, which also shows an almost perfect strength of agreement (Landis and Koch 1977).

Results

In response to research question 1, results showed differences in competence development between the five RLEs. Competence development ranged from no significant development in RLE 5, to significant development of three (RLE 4), four (RLE 3) and five (RLE 2) competencies, to significant development of all competencies in RLE 1 (p < 0.05, see Table 1 for developed competencies and effect sizes). Analyses did not show statistical differences in competence development per RLE for gender and age. Although not statistically analyzed, the data in Table 1 suggest the absence of systematic differences in competence development for study load and educational level. In the four RLEs that showed competence development, “domain-specific professional expertise” always developed and always developed the most. The competency “collaborating and discussing” also developed in all four RLEs that showed competence development.

In response to research question 2, regarding students’ competence development resulting from three typical learning environment characteristics, GLM analyses showed a large (partial η² ≥ 0.14) positive multivariate effect of the learning environment characteristics:

Working in multidisciplinary student groups (F[9, 113] = 2.432, p < 0.05, partial η² = 0.162): Competence development scores (posttest minus pretest) were significantly higher for students working in multidisciplinary student groups for the two competencies “deciding and initiating activities” and “collaborating and discussing.” The trend showed a higher, but nonsignificant, development score for all competencies of students working in multidisciplinary student groups, except for “domain-specific professional expertise.”

A high coaching intensity (F[9, 113] = 2.373, p < 0.05, partial η² = 0.159): Development scores (posttest minus pretest) were significantly higher for students working with a high coaching intensity for four competencies. These four competencies were “deciding and initiating activities,” “showing attention and understanding,” “planning and organizing,” and “collaborating and discussing.” In addition, the trend showed a higher, but nonsignificant, development score for all competencies of students who worked with a high coaching intensity.

Multistakeholder collaboration showed no significant multivariate effect, but it did show a trend of higher, nonsignificant, competence development scores for all competencies of students working on a high level of stakeholder collaboration.

In response to research question 2 regarding students’ other learning outcomes related to the typical learning environment characteristics, we found that students working in multidisciplinary student groups referred in 14 percent of the reported learning outcomes to learning from multidisciplinary group work. Students working in mono-disciplinary student groups did so in 7 percent of the reported learning outcomes. Students working on a high level of multistakeholder collaboration referred in 26 percent of the reported learning outcomes to learning from multistakeholder collaboration. Students working on a low level of multistakeholder collaboration did so in 27 percent of the reported other learning outcomes (see Table 3 for percentages and examples of reported learning outcomes).

Table 3.

Percentages of Students’ Reported “Other Learning Outcomes” Referring to Multidisciplinary Group Work and Multistakeholder Collaboration per Category of Learning Environment Characteristics.

	No. of Students per Category	No. of Excerpts per Category of Students	No. (%) of Excerpts that Refer to Multidisciplinary Group Work	Illustrative Example
Students working in multidisciplinary student groups	134	277	38 (14%)	I learned to collect and analyze information outside my own field of study.I learned how I am perceived by others.
Students working in monodisciplinary student groups	87	81	6 (7%)	I learned to reformulate the unclear clients’ assignment to make sure that we could work within our own professional domain.If you make plans, it is sometimes difficult to find the right necessary information in time.
Total no. of students	221	358	–

	No. of Students per Category	No. of Excerpts per Category of Students	No. (%) of Excerpts that Refer to Multistakeholder Collaboration
Students working on a high level of multistakeholder collaboration	131	253	67 (26%)	I learned to better accept other people’s views, but also to better defend my own perspective.All stakeholders have their own vision. You should talk with all of them and take all these visions into account.
Students working on a low level of multistakeholder collaboration	57	99	27 (27%)	I learned how to adapt a project to changing clients’ desires.Working for a client helps you to get a much broader perspective on what’s going on in the area.
Students working on an unknown level of multistakeholder collaboration	33	6	0 (0%)	–
Total no. of students	221	358	–

In response to research question 3 on teachers’ perceived learning outcomes, we found six typical learning outcome categories resulting from working in multidisciplinary groups (Table 4). Eighty-five percent of the teacher statements in these categories related to one of the measured competencies. Fifty-three percent of these 85 percent related to “domain-specific professional expertise,” including statements like “development of domain-specific professional expertise through explaining your own expertise knowledge to others.” Next, we found eight typical learning outcome categories resulting from multistakeholder collaboration (Table 4). Fifty-nine percent of the teacher statements in these categories were related to the measured competencies (e.g., “learning to acknowledge diverse, often conflicting interests” related to “showing attention and understanding”). Other statements referred to additional learning outcomes such as “developing insight into the profession” and “finding ways to effectively deal with diverse interests and perspectives during the whole process.”

Table 4.

Learning Outcome Categories Typically Resulting from Multidisciplinary Group Work and Multistakeholder Collaboration.

Learning Outcome Categories Typically Resulting from Multidisciplinary Group Work		Illustrative Example of a Teacher Statement for Each Learning Outcome Category
1	Developing knowledge within one’s own knowledge domain as a result of collaboration with students from other disciplines	Students learn to see the relevance of their own discipline.
2	Crossing the borders of your own discipline and learning from other disciplines	Students learn to speak each other’s professional languages.
3	Enriching the process by combining process approaches from diverse disciplines (including negotiation skills and assertiveness)	Learning to advocate for the importance of your own disciplinary knowledge.
4	Creating a better product by combining knowledge from diverse disciplines	Better solutions for a problem because the solution also has a multidisciplinary nature.
5	Increasing the performance level of collaboration and negotiation skills	Learning to record agreements.
6	Increasing insight into own abilities as a consequence of reflection with group members from other disciplines	Students get more insight in the strengths and weaknesses of all group members.
Learning Outcome Categories Typically Resulting from Multistakeholder Collaboration
1	Realizing the importance of clear articulation of the project assignment in consultation with all actors involved	Students learn to investigate the interest of the client.
2	Increasing sensitivity and openness to other interests and perspectives	Students learn to think from another person’s interest.
3	Finding ways to effectively deal with diverse interests and perspectives during the whole process	Students learn to think strategically, be politically correct and tactful.
4	Increasing the performance level of collaboration skills	Dealing with a limited (less than expected) contribution of stakeholders.
5	Increasing the performance level of presentation skills	Presenting for a “real” audience is more challenging than presenting at school.
6	Developing insight into the profession	Learning to know which issues are currently relevant in the professional practice
7	Creating understanding of the applicability and value of domain-specific expertise in practice	Learning to see the societal relevance of your discipline.
8	Providing enriched motivation as a result of working on a realistic project relevant to other stakeholders	Students are motivated since this is the real world.

In response to research question 4, we deduced six categories of preconditions for exploiting the learning potential of RLEs (Table 5). The largest numbers of statements were attributed to the precondition categories “intensive process coaching” (ten of thirty-two) and “real collaboration with the stakeholder” (seven of thirty-two).

Table 5.

Categories of Preconditions for Optimal Learning in the RLE.

Number	Precondition for Optimal Learning in the RLE	Illustrative Example of a Teacher Statement for Each Precondition Category
1	Intensive process coaching	Show students what they learn from the stakeholders.
2	Reflection to illuminate the learning potential before, during, and after the project	Students only learn if we organize process reflection.
3	Clarity about the project assignment for all parties involved	The assignment should be clear for the students, teachers, and clients; otherwise, students get easily demotivated.
4	Real collaboration with the stakeholders	Students should really be in contact with the stakeholders. Then they feel the relevance of solving the issue.
5	Students have basic time management and communication skills	Students should be able to well plan their work.
6	Practical preconditions such as flexible education schedules and willingness of the management to support the RLE	We need a more flexible school schedule when working in the RLE.

Discussion

Results for research question 1 show the RLE to be an effective learning environment for developing domain-specific professional expertise, as well as various generic competencies, all identified as relevant for working in RLEs by participating planning students and teachers. Four of five studied RLEs show significant competence development for three to nine competencies. Domain-specific professional expertise significantly developed and was the most developed competency in all four RLEs that showed development. This is a relevant finding with regard to concerns about students’ development of professional knowledge and expertise in innovative and authentic learning environments for higher education such as competence-based education (Biemans et al. 2004) and problem-based education (Schmidt, Vermeulen, and Van der Molen 2006). The competency “collaborating and discussing” also developed in all four RLEs that show development. This confirms the potential of the RLE for developing collaborative capacities of planners (Healey 2003; Seltzer and Ozawa 2002).

Answering the second research question, the study found that the typical RLE “boundary crossing” learning environment characteristic of working in multidisciplinary student groups enhanced students’ competence development. Working in multidisciplinary compared with monodisciplinary student groups led to higher competence development scores, and to more student-reported learning outcomes referring to learning from working in a multidisciplinary setting. In response to research question 3, the study found that RLE teachers also identify student learning outcomes resulting from multidisciplinary group work, in most cases related to one of the measured competencies, which underwrites the quantitative findings. This combination of quantitative and qualitative exposure of learning from working in multidisciplinary groups supports the added value of this boundary crossing characteristic of the RLE (Akkerman and Bakker 2011).

The second typical boundary crossing learning environment characteristic, working on a high level of multistakeholder collaboration, did not significantly enhance students’ competence development more than working on a low level of collaboration. Also, the number of reported student learning outcomes referring to learning from multistakeholder collaboration did not differ for a high or a low level of multistakeholder collaboration. The lack of an enhancing effect of intense multistakeholder collaboration for planning student learning is disappointing because this disputes the RLE to be an effective boundary crossing learning environment for upcoming planners. Remarkably, the percentages of the total amount of student learning outcomes that referred to learning from multistakeholder collaboration (26 and 27 percent of the total number of excerpts) were much higher than the percentages of student learning outcomes that referred to learning from working in a multidisciplinary setting (14 and 7 percent of the total number of excerpts). This could be caused by the fact that students are excited by what they learn from “real” external partners (in case of low intense stakeholder collaboration from the external client), more than what they learn from their peers. This suggests at least some kind of impact of working with external stakeholders on learning. Also, the participating teachers identified student learning outcomes resulting from multistakeholder collaboration. Different from multidisciplinary group work, these learning outcomes were in only half of the cases related to one of the measured competencies. The COM, as used in this study, may not have measured the things that students actually learn from working with external stakeholders. This could have contributed to the lack of a significant quantitative effect of multistakeholder collaboration on student learning. Probably, students learned other things than, or next to, the competencies measured in the COM. A second reason for the lack of a significant effect of intense collaboration with multiple stakeholders on competence development could be that learning with and from stakeholders was not explicated as a learning objective and typical learning activity in the studied RLEs. We were supported in this idea by the fact that the students who participated in the selection of competencies being relevant for working in the RLE before the start of this study (see Method section and Table 2) did not select the competency “building relationships and networking” as being relevant for working in the RLE. Apparently, they seemed not to be aware, at least at the start of their project, of the relevance of this competence for working in the RLE. A third reason for the lack of effect could be that stakeholder collaboration was not explicitly addressed in instructions or object of coaching during the RLE process. Just being embedded in a multistakeholder environment seems to not guarantee explicit learning thereof. Not explicitly aiming for and coaching student–stakeholder collaboration in the studied RLEs contrasts with the results from research question 4 where participating teachers identified collaboration with stakeholders to be an important precondition for optimal learning in the RLE. Overlooking these considerations, and regarding the many promising reported student learning outcomes on learning with and from multiple stakeholders, we argue that the RLE contains an unemployed learning potential of multistakeholder collaboration caused by the lack of explicating and supporting learning with and from the stakeholders in the RLE.

Results for the added independent variable coaching intensity showed that working in RLEs with a high coaching intensity compared to a low coaching intensity led to higher competence development scores. This result adds significance to the statements of teachers—from the results of research question 4—that process coaching and reflection are important preconditions for exploiting the learning potential of RLEs (Table 5).

Looking into how the majority of the students per RLE were classified (Table 1) might partly explain the differences in competence development per RLE. The two RLEs that showed most development (RLE 1 and RLE 2) both included all three typical characteristics. The two RLEs that showed the least development did not include the typical characteristics. This observed relation supports our statistical findings for the enhancing effect of the three typical characteristics and may explain differences in competence development between the studied RLEs.

Two limitations of the study may have influenced the results. First, competence development in this study was mainly measured by students’ own perception thereof. This was a consequence of differences in assessment strategies per RLE, and as such the nonavailability of comparable assessment data. It has recently been advocated that self-reports provide results that are as reliable as third-party data (Braun et al. 2012; Chan 2009). However, to be cautious, we decided to combine the quantitative measurements with qualitative student data on learning outcomes of the RLE, and with teacher data on student learning outcomes and preconditions for learning in the RLE. Next, all measurements took place before the students were graded on their RLE work, and both oral and written instructions stressed that the results of the measurements did not influence students’ RLE grading. The second limitation of the study relates to its quasi-experimental design. Since the study was conducted in educational practice, the RLEs differed in more than the studied variables (e.g., characteristics of the assignment [perceived authenticity, urgency, complexity etc.] or willingness of the stakeholders to really collaborate with the students and show an explanatory attitude). Although we did not see systematic variations or influences of these variables in our qualitative data, observations and discussions with students and teachers, it is appropriate to state that we did not study five equal RLEs and that the results for the value of typical design characteristics cannot completely be attributed to the studied variables. However, the study obtained evidence for student learning in a quasi-experimental design, and as such meets the need for studies in planning education that go beyond the description of an educational innovation (Wu and Brooks 2011). Implications of the results for planning education and future research are discussed in the concluding section.

Conclusions and Implications

This study confirms the effectiveness of the new, authentic multistakeholder Regional Learning Environment for planning students’ learning. The study specifically shows the added value of the learning environment characteristic of “working in multidisciplinary student groups” and identifies the learning potential of “intense collaboration between students and multiple stakeholders.” These learning environment characteristics prompt boundary crossing, inherent to current planning practice. Additionally, this study found a high coaching intensity to be a preconditional learning environment characteristic for boundary crossing learning in the RLE, and to enhance students’ competence development in the RLE.

Having identified similarities between RLEs, planning studios, and service learning, this study adds to the scientific debate and its practical implications on the optimization of studio and service learning pedagogy in planning education (Angotti, Doble, and Horrigan 2011; Long 2012). First, the confirmation that working in multidisciplinary instead of monodisciplinary student groups more strongly stimulates various competencies regarded as important for professional planners provides an argument for more explicitly facilitating multidisciplinary group work in planning studio courses (Long 2012) and in service learning (G. Harris 2004; Sletto 2010). Second, the revealed importance of intense process coaching and reflection in order to facilitate and explicate learning experiences in the RLE evidentially supports the important role that reflection has been given in service learning (Roakes and Norris-Tirrell 2000; Sletto 2010). For the future improvement of coaching and reflection in the RLE, especially with regard to students’ collaboration with stakeholders, we suggest RLE teachers make use of insights in systematic reflection strategies as extensively utilized in service learning environments. Third, although this study showed insignificant effects of student–stakeholder collaboration for students’ competence development, teachers supported the hypothesized learning potential of intense collaboration with multiple stakeholders. Planning studios and service learning are recommended, just like RLEs, to further experiment with using real-world, transdisciplinary assignments and collaborative knowledge construction between students, teachers, and multiple stakeholders. This will allow us to examine when and how multistakeholder collaboration fosters student learning. We also recommend addressing student–stakeholder collaboration more explicitly in learning objectives and coaching. Part of the learning outcomes of multistakeholder processes cannot be predicted beforehand because of unexpected learning opportunities during authentic collaborative trajectories. Therefore, we suggest authentic learning environments in planning education to experiment with the allowance of “learning surprises” (Scardamalia et al. 2012) to gain insight into what students really learn from working and learning with other stakeholders. Allowing such surprises and trying to address these in learning objectives and assessment strategies could support students in their learning to plan for the “undefined becoming” (Boelens and De Roo 2014).

Looking ahead to future research and reviewing some limitations of this study, we suggest carrying out follow-up studies to strengthen the effects of this study and deepen understanding of the RLEs’ learning potential, especially of its boundary crossing characteristics. These studies are specifically required for further investigation of the unexpected, insignificant findings of “intense collaboration between students and multiple stakeholders.” Adding the competency “building relationships and networking” to the competence test would be interesting since this competency is likely to be directly influenced by working in the multistakeholder RLE. In addition, adding teacher ratings to the student reported competence levels could increase the reliability of the quantitative data. Next to the more large-scale and quasi-experimental studies as described in this article, more qualitative, in-depth studies could also shed interesting light on the actual learning processes taking place or hampering learning in the RLE. Next, intervention studies, both quantitative and qualitative, could investigate whether active support of stakeholder collaboration effects student learning in the RLE. Both follow-up studies and intervention studies can reveal our identified “hidden” learning potential of stakeholder collaboration.

Beyond an effective innovation in planning education, the RLE can be positioned as a new model of knowledge production in planning practice. This supports the idea of different planning scholars to expand the boundaries of authentic learning environments in planning education outside the educational context (Angotti, Doble, and Horrigan 2011; Balassiano and West 2012; Long 2012; Sletto 2010; Winkler 2013). The potential of this idea could be investigated by broadening the scope of effectivity studies from student learning outcomes to learning outcomes for all parties involved, and to results for spatial development.

In conclusion, RLE insights and experiences provide a range of opportunities for the future development of planning education on our way to educate collaborative, “boundary crossing” planners.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material for this article is available with the manuscript on the JPER website.

Author Biographies

Carla Oonk is a researcher, teacher, and education coordinator at the Education and Competence Studies Group of Wageningen University and Research, the Netherlands. In her work she combines her backgrounds in Environmental Sciences (BSc) and Land Use Planning (MSc) with her experience in Educational Sciences (PhD). Her research focuses on student and teacher learning in hybrid learning configurations.

Judith Gulikers is a researcher and teacher at the Education and Competence Studies Group of Wageningen University and Research, the Netherlands. She has a background in Educational Psychology with a focus on authentic assessment. Her current research focuses on authentic assessment in cross-boundary learning settings.

Martin Mulder is a Professor Emeritus at the Education and Competence Studies Group of Wageningen University and Research, the Netherlands. His research mainly focuses on competence theory and development in both initial and professional, and formal and informal education.

References

Akkerman

Bakker

2011. “Boundary Crossing and Boundary Objects.” Review of Educational Research 81:132.

Albrechts

2013. “Reframing Strategic Spatial Planning by Using a Coproduction Perspective.” Planning Theory 12 (1): 46–63.

Angotti

Doble

Horrigan

2011. “At the Boundaries: The Shifting Sites of Service-Learning in Design and Planning.” In Service-Learning in Design and Planning: Educating at the Boundaries, edited by Angotti

Doble

Horrigan

, 1–16. Oakland, CA: New Village Press.

Balassiano

2011. “Tackling ‘Wicked Problems’ in Planning Studio Courses.” Journal of Planning Education and Research 31 (4): 449–60.

Balassiano

West

2012. “Seeking the Studio Experience Outside of the Studio Course.” Journal of Planning Education and Research 32:465.

Bartram

2011. The SHL Universal Competency Framework. Surrey, UK: SHL White Paper. http://www.cebglobal.com.

Biemans

Nieuwenhuis

Poell

Mulder

Wesselink

2004. “Competence-Based VET in the Netherlands: Background and Pitfalls.” Journal of Vocational Education and Training 56 (4): 523–38.

Boelens

De Roo

2014. “Planning of Undefined Becoming: First Encounters of Planners Beyond the Plan.” Planning Theory 12:46–63.

Booher

Innes

2002. “Network Power in Collaborative Planning.” Journal of Planning Education and Research 21:221.

10.

Boonstra

Boelens

2011. “Self-Organization in Urban Development: Towards a New Perspective on Spatial Planning.” Urban Research and Practice 4 (2): 99–122.

11.

Braun

Woodley

Richardson

J. T.

Leidner

2012. “Self-Rated Competences Questionnaires from a Design Perspective.” Educational Research Review 7 (1): 1–18.

12.

Chan

2009. “So Why Ask Me? Are Self-Report Data Really That Bad.” In Statistical and Methodological Myths and Urban Legends: Doctrine, Verity and Fable in the Organizational and Social Sciences, edited by Vandenberg

R. J.

, 309–36. New York: Taylor & Francis.

13.

Cohen

1988. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum.

14.

COLO. 2006. “Competentiemodel Kenniscentra Beroepsonderwijs Bedrijfsleven.” http://www.kwalificatiesmbo.nl.

15.

Dalton

L. C.

2007. “Preparing Planners for the Breadth of Practice: What We Need to Know Depends on Whom We Ask.” Journal of the American Planning Association 73 (1): 35–48.

16.

De Roo

Silva

2010. A Planner’s Encounter with Complexity. Burlington: Ashgate.

17.

Dewey

1938. Experience and Education. New York: Macmillan.

18.

Domingo

Beunen

2012. “Regional Planning in the Catalan Pyrenees: Strategies to Deal with Actors’ Expectations, Perceived Uncertainties and Conflicts.” European Planning Studies 21 (2): 187–203.

19.

Edwards

M. M.

Bates

L. K.

2011. “Planning’s Core Curriculum: Knowledge, Practice, and Implementation.” Journal of Planning Education and Research 31 (2): 172–83.

20.

Foorthuis

W. R.

Lutz

Rippen

2012. Knowledge Arrangement for the Learning Region: Kenniswerkplaats as a Method for Regional Learning and Lifelong Learning. The Hague: Ministry of Economic Affairs.

21.

Forester

1999. The Deliberative Practitioner: Encouraging Participatory Planning Processes. Cambridge, MA: MIT Press.

22.

Forsyth

Mc Girr

1999. “Inside the Service Learning Studio in Urban Design.” Landscape Journal 18 (2): 166–78.

23.

Frank

A. I.

Mironowicz

Lourenço

Franchini

Ache

Finka

Scholl

Grams

2014. “Educating Planners in Europe: A Review of 21st Century Study Programmes.” Progress in Planning 91:30–94.

24.

Freeman

R. E.

1984. Stakeholder Management: A Stakeholder Approach. Boston, MA: Pitman.

25.

Giles

D. E.

Jr. Eyler

1994. “The Theoretical Roots of Service-Learning in John Dewey: Toward a Theory of Service-Learning.” Michigan Journal of Community Service Learning 1 (1): 77–85.

26.

Glaser

B. G.

Strauss

A. L.

1967. The Discovery of Grounded Theory: Strategies for Qualitative Research. New York: Aldine Transaction.

27.

Harris

2004. “Lessons for Service Learning in Rural Areas.” Journal of Planning Education and Research 24 (1): 41–50.

28.

Harris

S. C.

Irazábal

2011. “Transforming Subjectivities: Service That Expands Learning in Urban Planning.” In Service-Learning in Design and Planning: Educating at the Boundaries, edited by Angotti

Doble

Horrigan

, 107–24. Oakland, CA: New Village Press.

29.

Haughton

Counsell

2004. “Regions and Sustainable Development: Regional Planning Matters.” Geographical Journal 170 (2): 135–45.

30.

Healey

1997. Collaborative Planning: Shaping Places in Fragmented Societies. Basingstoke, UK: Macmillan.

31.

Healey

2003. “Collaborative Planning in Perspective.” Planning Theory 2:101–23.

32.

Herrington

Oliver

2000. “An Instructional Design Framework for Authentic Learning Environments.” Educational Technology Research and Development 48 (3): 23–48.

33.

Higgins

Aitken-Rose

Dixon

2009. “The Pedagogy of the Planning Studio: A View from Down Under.” Journal for Education in the Built Environment 4 (1): 8–30.

34.

Jacoby

2014. Service-Learning Essentials: Questions, Answers, and Lessons Learned. New York: John Wiley.

35.

Khaled

A. E.

Gulikers

J. T. M.

Tobi

Biemans

H. J. A.

Oonk

Mulder

2014. “Exploring the Validity and Robustness of a Competency Self-Report Instrument for Vocational and Higher Competence-Based Education.” Journal of Psychoeducational Assessment 32 (5): 429–40.

36.

Lagendijk

2001. “Regional Learning between Variation and Convergence: The Concept of ‘Mixed Land-Use’ in Regional Spatial Planning in the Netherlands.” Canadian Journal of Regional Science 24 (1): 135–54.

37.

Landis

J. R.

Koch.

G. G.

1977. “The Measurement of Observer Agreement for Categorical Data.” Biometrics 1977;33(1):159–74.

38.

Long

J. G.

2012. “State of the Studio: Revisiting the Potential of Studio Pedagogy in U.S.-Based Planning Programs.” Journal of Planning Education and Research 32:431.

39.

Meijles

Van Hoven

2010. “Using the Rural Atelier as an Educational Method in Landscape studies.” Journal of Geography in Higher Education 34 (4): 541–60.

40.

Miles

M. B.

Huberman

A. M.

Saldana

2014. Qualitative Data Analysis: A Methods Sourcebook. Thousand Oaks, CA: Sage.

41.

Németh

Long

2012. “Assessing Learning Outcomes in U.S. Planning Studio Courses.” Journal of Planning Education and Research 32 (4): 476–90.

42.

Newmann

Wehlage

1993. “Five Standards for Authentic Instruction.” Education Leadership 50 (7): 8–12.

43.

Nienhuis

Van Dijk

De Roo

2012. “Let’s Collaborate! But Who’s Really Collaborating? Individual Interests as a Leitmotiv for Urban Renewal and Regeneration Strategies.” Planning Theory and Practice 12 (1): 95–109.

44.

Rittel

Webber

1973. “Dilemmas in a General Theory of Planning.” Policy Sciences 4:155–69.

45.

Roakes

Norris-Tirrell

2000. “Community Service Learning in Planning Education: A Framework for Course Development.” Journal of Planning Education and Research 20 (1): 100–10.

46.

Scardamalia

Bransford

Kozma

Quellmalz

2012. “New Assessments and Environments for Knowledge Building.” In Assessment and Teaching of 21st Century Skills, edited by Care

Griffin

Wilson

, 231–300. Dordrecht, the Netherlands: Springer.

47.

Schmidt

Vermeulen

Van der Molen

2006. “Long-Term Effects of Problem-Based Learning.” Medical Education 40 (6): 562–67.

48.

Scholz

R. W.

Steiner

2015. “The Real Type and Ideal Type of Transdisciplinary Processes: Part I—Theoretical Foundations.” Sustainability Science 10 (4): 527–44.

49.

Schön

D. A.

1987. Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions. San Francisco, CA: Jossey-Bass.

50.

Seltzer

Ozawa

C. P.

2002. “Clear Signals Moving on to Planning’s Promise.” Journal of Planning Education and Research 22 (1): 77–86.

51.

Sletto

2010. “Educating Reflective Practitioners: Learning to Embrace the Unexpected through Service Learning.” Journal of Planning Education and Research 29 (4): 403–15.

52.

Strauss

Corbin

1998. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Thousand Oaks, CA: Sage.

53.

Thomas

2012. “Values and the Planning School.” Planning Theory 11 (4): 400–17.

54.

Umemoto

2001. “Walking in Another’s Shoes: Epistemological Challenges in Participatory Planning.” Journal of Planning Education and Research 21 (1): 17–31.

55.

Vygotsky

L. S.

1978. Mind in Society: The Development of Higher Mental Processes. Cambridge, MA: Harvard University Press.

56.

Walker

Nocon

2007. “Boundary-Crossing Competence: Theoretical Considerations and Educational Design.” Mind, Culture, and Activity 14 (3): 178–95.

57.

Watson

2001. “Social Constructivism in the Classroom.” Support for Learning 16 (3): 140–47.

58.

Winkler

2013. “At the Coalface: Community–University Engagements and Planning Education.” Journal of Planning Education and Research 33 (2): 215–27.

59.

Brooks

2011. “Report from the Editors: Writing Pedagogy Articles.” Journal of Planning Education and Research 29 (4): 401–02.

60.

Young

2009. “Implementing National Qualifications Frameworks: Problems and Possibilities.” In International Handbook of Education for the Changing World of Work, edited by Maclean

Wilson

Young

, 2917–33. Dordrecht, the Netherlands: Springer.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.22 MB