Abstract
Design-Based Research (DBR) is being implemented more broadly in educational research in Germany as a way of finding solutions for complex situations in education and ensuring their applicability for practitioners. The objective of DBR is to achieve a product that can be used in the field and a contribution to scientific theory through the process, including a broader generalization to other contexts. DBR has overlaps with Action Research (AR), including participatory aspects and emphasis on the voices and reality of people in the field. The process is open-ended and can involve several iterations until a solution is found that is convincing from the different participating perspectives. In this article, we discuss the relationship between DBR and AR and the potential of DBR as a way to advance participative qualitative and mixed-methods research in educational science. We present a project with two schools in Hamburg, Germany, as an example of DBR in education. It is too early for concrete results, but we will give an insight into the expectations, hopes, fears and first experiences of different participating parties at the onset of the project.
Introduction
Design-Based Research (DBR) is being implemented increasingly in educational research in Germany. The approach is aimed at finding solutions for complex situations in education and ensuring their applicability for practitioners. In 2023, we (the authors) started a DBR project with two schools in Hamburg, Germany, to support student transition from primary to secondary education (Grade 5–6) and to systematically introduce socio-emotional learning contents in heterogeneous classrooms in secondary education in Hamburg. In both schools, teachers and students from three and five classrooms respectively are implementing (it is an ongoing project) recently published didactic materials from the SeELe program (Sozial-emotionale Entwicklung mit Lernleitern/Socio-emotional development with ladders of learning). Through an ethnographic approach, we observe and support their implementation. In collaboration with the teachers and students, we work on redesigning the materials to make them more accessible for a diverse group of students. As researchers, we aim to map out conditions for a successful implementation of self-regulated socio-emotional learning in diverse classrooms. Christine is the lead researcher and Citka is a doctoral student who conducts the ethnography in the schools. Because we bring different experiences and perspectives to the project, we write our names at the beginning of the paragraphs that explicitly reflect our voice.
Design-Based Research in Education
Some authors have argued adamantly that Design Science is not Action Research (Baskerville, 2008). We will present Design-Based Research (DBR) as a framework for educational research and discuss its relationship with Action Research (AR)—which has only been thematized marginally in the literature (e.g., McKenney & Reeves, 2020; Moser, 2018; Silverman, 2015).
Design-Based Research (DBR) in education was developed out of a frustration with experimental or quasi-experimental research in education with its research-practice gap and its failure to take into account “that classrooms are complex and dynamic systems that should be examined holistically” (Rubin et al., 2019, p. X). For example, educational researchers often see practitioners as “a group to be studied” (Reinking, 2021, p. xiii) without a real commitment to inform practice. DBR’s emergence is also informed by the idea that much qualitative research also fails to have an impact on practice (Oh et al., 2016). Different names are used for the DBR approach, for example, “design experiment,” “design study,” “educational design research” or “design research” (Rubin et al., 2019, p. XI). McKenney and Reeves (2020) use Educational Design Research as the overarching term for a “family of approaches” (p. 85) including design experiment and design-based research. In this article, we use the term “Design-Based Research” unless we quote authors who specifically differentiate the term they use from this one.
DBR began developing explicitly as an approach with relevant projects and landmark papers in the 1990s (McKenney & Reeves, 2019; Reinking, 2021), although its roots can be traced back to Vygotsky and his followers (Campanella & Penuel, 2021) and Dewey (Reinking, 2021), among others. Moser (2018) interprets its development as rooted in “a revival of the need” (p. 494) for practice research (such as Action Research).
DBR explicitly combines design and research and aims at two forms of outcomes: an improved practical intervention and a new theoretical understanding of a phenomenon. To achieve this dual aim, researchers interact with practitioners, implement the developing intervention in a real setting and make sure their research and design are responsive to the context (McKenney & Reeves, 2019; Rubin et al., 2019).
McKenney and Reeves (2019) identify three core phases of Educational Design Research that can be repeated in multiple sub-cycles to form a whole design research process: (1) Analysis and exploration during which the problem is identified and analyzed through literature research and collaboration with practitioners, (2) Design and construction, a “deliberative-generative cycle” (p. 85) in which a (tentative) solution for the problem is developed that takes into account the available knowledge and conceptual relationships. (3) Evaluation and reflection, in which the solution is implemented, its qualities are studied and, through reflection, the results of research and development are considered and brought together. This may lead to ideas for a redesign of the intervention.
Studies may comprise only one cycle of each phase and research proposals and reports might only encompass a cluster of some cycles. However, the whole process usually involves several iterations that at least include the second and third phase several times. Because the whole DBR process might take several years and even be comprised of different projects with distinct funding sources, DBR doctoral theses might cover only a part of the process (Oh et al., 2016).
DBR is not a method, and some authors even come to the conclusion that “Educational design research (EDR) is not so much a specific research methodology, as it is an idealistic genre of systematic inquiry into significant educational problems and challenges” (Oh et al., 2016, p. 1). As such, DBR can integrate different research methodologies from different traditions, for example ethnography, interviews, and forms of quantitative inquiry (Campanella & Penuel, 2021; Rubin et al., 2019). When analyzing qualitative data, researchers in DBR often code them inductively, to allow for the discovery of unexpected patterns and topics, which results in a closeness to grounded theory (Freedman & Kim, 2019; Rubin et al., 2019). Paradigmatically, DBR has “pragmatic commitments to changing educational practice” (Campanella & Penuel, 2021, p. 4) and it “can and does take critical perspectives” (ibid.).
Doing DBR brings with it several challenges and ambivalences. Researchers and practitioners come from divergent environments (Emmler & Frehe-Halliwell, 2020), have a different way of speaking about educational topics, and a distinct rhythm and dissimilar structures that frame their work. One of the main features of DBR is its responsiveness to local context. However, in the long run, it aims at generalizability and spread to other contexts. Balancing those two aims can be a challenge (McKenney & Reeves, 2019; Reinmann, 2022a). Researchers doing DBR need to stay open to unexpected results and developments in their projects that might require changes in their process. At the same time, they need to implement a systematicity that differentiates DBR from pure practice (Reinmann, 2022a). Since DBR is a framework for educational research that does not have a long track record, it does not yet have commonly accepted standards. This makes keeping the aspects that make DBR special a challenge while working towards it becoming established (McKenney & Reeves, 2019; Reinmann, 2022a, 2022b).
DBR and AR
Christine. When I first read about DBR, I thought: “Isn’t this a form of Action Research?” I am a teacher myself (special education with an emphasis on socio-emotional development and learning) and had done research with participatory elements before (Schmalenbach, 2018; Schmalenbach et al., 2023). In fact, I had accompanied the development of the participatory Action Research Special Interest Group ((p)AR-SIG) of the International Congress for Qualitative Inquiry (ICQI) out of an interest in AR and its possibilities for enhancing participation in research. A deeper analysis of the differences and commonalities between AR and DBR was both fascinating and frustrating to me. Every time I thought I had identified a distinguishing element, I found examples of projects that limited these findings. In the end, I can say that there are big overlaps between the two research genres, but also tendencies that are stronger in one or the other:
The most salient distinguishing feature of DBR is
The second aspect in which projects within the framework of DBR might differ from projects within the framework of AR, is
There are also examples from the world of Higher Education where researchers apply DBR to their own teaching (Reinmann & Brase, 2022). And within AR, there are different degrees of involvement of practitioners in research. Lewin’s research in the 1930s and 1940s, which some roots of AR are traced back to, has been criticized as lacking participation (Hendricks, 2019). When this is the case, the differences between DBR and AR are diminished.
This leads us to the third (potential) distinguishing feature, to what Noffke (2009) calls “
When participating researchers and practitioners recognize equity as an important aspect of an intervention, and include the voices of marginalized groups in the development of the intervention, the project clearly contains the political dimension. Such an approach to DBR has been published under terms such as “participatory design research, critical design ethnography, and socially transformative design” (Rubin, 2019, p. 249). Campanella and Penuel (2021) state that “explicit commitments to equity and justice, to remediating and repairing relationships in human communities and with the Earth, as well as to cultural sustainability and resurgence, are increasingly goals of DBR” (p. 6). They propose a critical pragmatism for DBR, noting that Dewey’s pragmatism always included the idea of “prepar[ing] young people for participation in a democratic society” (p. 17). This distinguishing feature is not as clear as it would seem at first glance. Wallerstein and Duran (2008) postulate that CBPR (Community-based Participatory Research), which can be seen as one version of AR, has two different traditions, on the one hand the more pragmatic “utilization-focused research with practical goals” (p. 27), on the other hand, openly emancipatory research. In short, more and less politically aware tendencies also exist in AR.
The fourth feature that could distinguish AR from DBR is their
When considering these four potentially distinguishing features, the overlaps of the two approaches become more traceable. For example, McKenney and Reeves (2020) classify AR and DBR both as members of the “family of approaches seeking practical and scientific synergies” (p. 85) whereas Silverman (2015) writes of “designerly ways as practical skills for action research” (p. 717). Kim (2019) integrates the two approaches in a project of Design-Based Action Research (DBAR), a term proposed by Nijhawan (2017). Terms like DBAR or “participatory design research” (Rubin, 2019, p. 249) show how researchers find ways to have the two approaches enhance each other.
AR and DBR are ultimately two different genres of research that have developed out of different roots but in the confrontation of similar challenges (e.g., lack of relevance of traditional research for complex problems in real contexts) and that partially deal with similar questions (e.g., how to identify problems and solutions and how to bridge the gap between theory and practice). In Figure 1, we illustrate some of the relationships I have presented thus far. There are slightly different priorities when it comes to the specifics—relevance of design of a concrete product vs. social action, collaboration between practitioners and researchers, focus on the personal and political dimension, generalizability and theory generation. However, the overlap between them is such that one AR project (AR1) and one DBR project (DBR1) might be closer to each other than to another project of the same approach (AR2 and DBR2). When this is the case, both sides might benefit from a thorough exchange of ideas and experiences to enhance the projects. The relationship between DBR and AR as a Venn diagram (features only as tendencies).
In the following paragraphs we present an ongoing project in Germany that can be described as DBR at the interface with AR. We describe how we are implementing the design of a product, developing our research-practice partnership, experiencing the personal and political dimension and aiming at scaling up and theory development. The project is still ongoing and we wrote this article when it had just started, so this is only an introduction into the process and our first impressions.
Self-Regulated Socio-Emotional Learning at Secondary Schools in Hamburg—DBR Project in Germany
Christine
Our design process started in 2017. I am part of a group of four researchers and educators who were interested in pedagogical approaches to support the socio-emotional development of students. I am an expert on Cooperative Learning as a pedagogical and didactic approach (e.g., Schmalenbach, 2018). Thomas Müller had worked with ladders of learning as an approach to structured self-regulated learning both in practice and in research (e.g., Müller et al., 2015). Stefanie Roos had done extensive research on socio-emotional learning (SEL; e.g., Roos & Petermann, 2005) and Anja Grieser is an educator specialized on socio-emotional development who had designed a part of a ladder of learning on that topic as part of her graduate degree. Anja had also studied art, which proved to be helpful for the aesthetic aspect of the design of our materials. The four of us noticed that socio-emotional learning is often implemented in schools as a training, separate from other classes, and saw a need for materials that facilitate socio-emotional learning in a self-regulated manner within the classroom. We wanted to offer a possibility for students from diverse backgrounds to bring their previous experiences and strategies into an open-ended learning process and to engage socio-emotional topics (e.g., emotions, communication, conflicts) both autonomously and in interaction with others. We aimed at designing the materials in a way that could be used by students with diagnosed emotional and behavioral difficulties, and by other students as well so that they are fit for inclusive settings (Schmalenbach et al., 2020). The developed materials included a poster of a ladder of learning with 16 topics, each of them forming a so-called milestone, a learning unit, with 16–19 tasks (see Figure 2). Through symbols and numbers, the ladder guides the students to the materials, where they can read how to proceed in each activity and whether they should do it alone or with peers or with teacher support (Müller et al., 2022; see Figure 2). Examples of SeELe materials, taken from Müller et al. (2022), courtesy of Ernst Reinhardt Verlag.
We worked on the curriculum and the materials for five years. During that time, besides conducting literature research on socio-emotional learning and generating knowledge from the combination of our expertise in different learning methodologies, we engaged in an intensive exchange with practitioners. Some practitioners (including Anja Grieser) tried out parts of the materials we had created and gave us feedback on their experiences with them. A more systematic implementation and evaluation we had intended was truncated by the Covid pandemic. In 2022, the SeELe-materials (SeELe - Sozial-emotionale Entwicklung mit Lernleitern/Socio-emotional development with ladders of learning) were finally published (Müller et al., 2022). Since then, we have received feedback from teachers who are using them—some find them very helpful and are excited about them, others tell us of challenges they face when using them. We are also hearing of very different ways that they are implemented—not all of them in the manner we intended, but not necessarily in a less valuable way.
When I started my position as assistant professor in education with a focus on learning difficulties and socio-emotional development at the University of Hamburg, teachers in leading positions at two secondary schools in Hamburg approached me with an interest in doing research about different aspects of socio-emotional learning. They found our approach intriguing. That seemed to be a good opportunity to accompany the implementation of the SeELe program and materials more closely, systematically collect feedback and find out if there are ways to make the materials more accessible to a broader range of students, while also figuring out teacher support needs. Around the same time, Citka approached me expressing interest on writing a doctorate dissertation on self-regulated learning. We went to meetings with the schools together and in an exchange of experiences, needs and ideas, the project took shape.
Citka and Christine
The schools we work with are dealing with different challenges with regard to socio-emotional learning in their school development. School A wants to find additional ways of supporting the transition of students from primary to secondary school (forth to fifth grade in Hamburg). This school already systematically implements SEL in fifth grade but in smaller groups outside the usual classroom. The SeELe materials seemed to school leaders to be a good option to have the whole classroom engage in socio-emotional learning together and improve classroom cohesion. The SeELe materials were already known in School B and the coordinator for special needs, who is also a sixth-grade teacher, proposed to her colleagues to start a collaborative implementation to support each other and to find out whether these materials could be used to include SEL in the school more systematically. At the same time, because of our expertise with the topics and materials, they hoped that collaboration with us as researchers would help them in their first implementation (Schmalenbach, Brand, & Steinvoord, 2023). After meeting with them to present the materials and our own research interests in more detail, they decided to embark on the project with us.
Areas of Focus on Two Projects.
Permission from the educational authority and funding were obtained for the overarching project on self-regulated socio-emotional learning at integrated comprehensive schools in Hamburg. The product we (Christine and Citka) are working on collaboratively is a revised and enhanced version of the materials that is adapted for more diverse classrooms and, if needed, an expanded manual for professional development for teachers. We are also aiming to map out conditions for a successful implementation of self-regulated socio-emotional learning in diverse classrooms.
Citka
At the start of the school year, Christine and I introduced the materials to the teachers at both schools. In school A, three classes participated in the project. The classes are mentored by a team consisting of a teacher and an educator. In school B, five classes are part of the project. Each class is guided by a team of two teachers, one of which prepares and leads the lessons while the other provides support during the lessons. I collect data through ethnographic observations in classes and an analysis of students’ written work. We conduct focus group interviews with students and teachers at three points during the project.
At School A, due to a favorable timetable, I am able to attend all classes in which students work with the SeELe materials. Because the classes in School B have their sessions with the SeELe materials at the same time, I can only be in one class each week, so I decided to focus on three of the five classrooms. Still, I make sure to meet with the teachers during breaks before and after lessons and in regular meetings, so that they can contact me at any time (Figure 3). The students work with the SeELe materials for around 90 minutes a week. We expect the classes to complete all topics by the end of the 2024/2025 school year. My approach to data analysis is Reflexive Grounded Theory (Breuer et al., 2019) and I plan to have the report on our project finished by summer 2026. After the materials have been redesigned, there is a plan to implement them at a larger scale within a quasi-experimental design. This will have to be another research project. Research methods within the DBR process.
Christine and Citka
Our current project can be clearly allotted to the genre of DBR because the (re)design of the SeELe materials to make them more inclusive is one central aspect of our work. Concerning the core phases of DBR (McKenney & Reeves, 2019), this particular project starts with a phase 3 and continues with a repetition of phase 2. The participating teachers were not included in the original design (although others were) but they are now influencing the redesign—as are the students. This points to the second aspect that makes our project DBR, the research-practice partnership. As elaborated above, the different stakeholders had different hopes and expectations from the program that complemented each other rather well. However, finding a way of working together did take some time and communication and is an ongoing process.
At the beginning, teachers were afraid of not implementing the program correctly, or even harming the project, and being seen critically by Citka during her participant observation. We, however are interested not only in learning about teacher needs when implementing the program as it was conceived, but also in documenting how teachers implement the program in real settings and adapt it to their teaching styles, and the consequences of adaptation for the students. So, when difficulties in the implementation arise, Citka sometimes looks for a solution together with the teachers (and occasionally brings the topic to Christine so she can support the search for a solution) and at other times reassures them that we are interested in seeing their approach to the program—not only implementation “by the book.” One teacher expressed: “I always think you have very clear ideas about how a lesson should run and I don’t want to do anything wrong. But if you say that’s okay, then that’s what I’ll do.”
Many of the teachers had not used much self-regulated learning in their classrooms before, so the use of the materials required a change in their approach to teaching. They had to find out how far they were willing to go and how much they would adapt the use of the materials and the method to their own style and context. Part of Citka’s role as participant observer became to encourage the teachers and educators if things did not go as planned. All interactions and decisions were carefully documented. With time, teachers became less worried about making mistakes.
How the project started also had an influence on how it was perceived by teachers. In one school, classroom coordination teams were put together over the summer and some educators only started working at this school at the beginning of the school year. Although they had expressed their desire to participate in the project as it was presented to them by their colleagues, these teams did not participate in a systematic introduction from us. Since everything was new for the teams at the beginning of the school year, the time we were allotted for an introduction was only 1.5 hours for the presentation of the program and the distribution of the materials. One educator expressed unease at this after several weeks: “I felt like I was thrown in at the deep end at the beginning. I didn’t and still don’t really know what we should do and how. We need a proper introduction at the beginning that lasts longer than just 1.5 hours. Like a further training course.” In the other school, we had met with all the participating teachers during the school year before our project started. They had discussed together whether they wanted to be a part of the project and had decided that they did. Nelly Brandt, who had initiated the process, was worried that their sentiment might have changed over the summer, but it had not. She shared her experience in a podcast in which Christine and she were interviewed together about the project: After the summer vacation, Christine and Citka came to our school on the very first day that the students were back to do this teacher training, the first of the new school year. And of course, I thought, oh God, now my colleagues had six weeks [of vacation] and now they’re going to have a training course in the afternoon on the very first day / and maybe they're all thinking: “What has she brought here?” And I was totally relieved that everyone was somehow still really enthusiastic about it. And that they were still so happy to be on board. And that was the first time for me that I thought, yeah, that's really cool. (Schmalenbach, Brand, & Steinvoord, 2023, 23:30–24:40)
When teachers at that school reflected on the collaboration between researchers and practitioners after four months of working together, they found it very positive. In one of our focus groups, a teacher described her view of working with Citka: “It’s also nice when you’re […] in the class. I also feel like you’re part of the team, so I don’t have the feeling that this is the university and this is the school. Rather, we are somehow doing it together at the moment.” Citka sees the project as part of her professional development as a pre-service teacher: As an aspiring special education teacher in the area of socio-emotional learning, my goal is to support the development of the emotional and social skills of my students. I see an opportunity for every child to learn at their own level, especially in self-regulated learning environments. This research project is an opportunity for me to learn from students and teachers how theoretical concepts work in reality and how I and other teachers can create a learning environment for students to learn well on social and emotional topics.
The teachers also support each other during the project, as one teacher put it: “And the nice thing is that we actually all do this in the class and that we can also discuss it with each other again: ‘Tell me, how did that go for you?’ and ‘it didn’t go well for us today’, or something like that, so that we can have a look but also always know that you are there in the background and can ask you.” So, to a certain degree, our idea of a learning community is developing, although not across different schools. However, we also noticed that fully taking on a role as co-researchers would require more time and engagement than most teachers are willing and able to invest in the project given their copious responsibilities at the school. So, we are including them in all aspects of the research as much as we can, but while the teachers have the responsibility for the implementation of the materials in their classrooms, we as researchers bear the responsibility for the research aspect of the project and take on most of it. Reflection on both aspects is conducted together and our respective experiences and thoughts about them inform each other.
The students are also project stakeholders and involved in the redesign of the materials. In some classrooms the teachers decided with us to tell the students that this was not the final version of the materials and that we wanted their expertise and feedback on what to keep and what to change so that as many children as possible could benefit from the materials in the future. Other teachers opted not to give the students this perspective of the materials at the beginning so that they did not feel enticed to find mistakes instead of concentrating on their learning process. In this way, some students were involved in the process of redesigning more explicitly and continuously from the beginning, whereas others are consulted about their opinions about the materials during the focus groups or during ethnographic observations.
When students were asked in the focus groups about having Citka as a researcher in their classroom they found several positive aspects to it. One student emphasized mainly the opportunity of contributing to an improvement of the materials: “I think that’s good, because that way one can improve something that one doesn’t think is so great, and then one thinks that others might think it’s not great either, that one can improve it so that everything/that it helps everyone.” Another student mostly perceived Citka to be a resource for learning: “Yes, I also think it’s quite good. Because now, for example, we have a third or fourth person there to support us. Sometimes there are only two people in the class, meaning two teachers.” This resonates with the teacher’s perception of Citka as a “part of the team.”
We wanted to actively involve as many the students as possible in the process, especially those with diagnosed special educational needs, so that we could adapt the materials to their needs and make them more inclusive. This proved to be more difficult than expected as several parents of students with diagnosed special needs did not give their consent for their children to be interviewed. However, as students in the two comprehensive integrated schools in Hamburg are a very diverse group, we still have the opportunity to include many different voices in our research.
Beyond a description of our research-practice partnerships, these paragraphs also give first indications of a personal and political dimension of the project and the changes that take place in the different participants (both practitioners and researchers). We will be able to write about the developments in more detail after concluding the project. The voices we included show how we are still finding and negotiating our roles and expectations within the research process—while we are involved in it from different perspectives and embedded in different institutions with their requirements and rhythms. We are all learning and we are developing something together. At the same time, our relationships are growing.
Altogether, we allocate our project to DBR but at the interface with AR because of its focus on the design of a product, its research-practice partnerships, its inclusion of the personal and political dimension and its aim at scaling up and theory development (Figure 4). Locating the SeELe project in the context of DBR and AR.
Conclusion
In this article we have shown how DBR and AR approaches have a lot in common and how educational researchers that implement either could benefit from an exchange of experiences and ideas. This is specially the case in projects that aim at developing and adapting interventions that are meant to contribute to more equity and agency and to include the voices of all relevant stakeholders and participants, explicitly also those of hitherto marginalized groups. Including relational, political and personal aspects while negotiating the connection to more traditional research is a challenge we might be able to support each other with. And, depending on the context, focus, and original impulse for a project, researchers might want to switch approaches once in a while and use their experiences with one to inform their implementation of the other. Either way, we hope that we have made the case for DBR being a potentially helpful framework for participation and action in educational research.
Footnotes
Acknowledgments
We thank the teachers and students who share their classrooms and experiences so generously with us. Christine Schmalenbach thanks Dr Alexa Brase for the conversations on DBR and its relationship to other approaches to research that gave her food for thought. We thank Nils Oliver Fehrmann for his support in the recording and transcription of interviews.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The project is funded by a grant from the foundation Kurt und Käthe Klinger Stiftung and a scholarship from the foundation Stiftung der Deutschen Wirtschaft (SDW) for Citka Ashouri.
