Abstract
Citizen science is a process that seeks to involve the public in scientific projects through systematic data collection and observations of natural phenomena. Environmental education centres on the relationship between humans and nature and has attracted the attention of more and more educators because of its increasingly important educational value. This paper reports an empirical study on environmental education projects featuring observations of natural phenomena and using local environmental resources. Using the Spring Swallow Survey project as an example, the design ideas and evaluation framework for the project are analysed. The implementation effect of the project is examined quantitatively and qualitatively. Finally, the resource package used in the project is revised and redeveloped according to the effect evaluation data.
Introduction
The term ‘environmental education’ was coined at the United Nations Conference on the Human Environment held in Stockholm in 1972. Afterwards, the understanding of environmental education remained at the level of formulating specifications to restrict people's behaviour. The Tbilisi Declaration adopted at the world's first intergovernmental conference on environmental education in the Soviet Union in October 1977 marked a stage of vigorous development of the field. The Rio Declaration on Environment and Development and the Agenda 21 adopted by the United Nations in Brazil in June 1992 showed that human understanding of environmental education had expanded in terms of both time and space. Humans should not only solve present problems facing our species but also consider the impact of current actions on the future. This indicates that environmental education is linked to sustainable development (Cui, 2007). We can see that, since the 1970s, international environmental education has undergone a process beginning from its rise to its fuller development and is attracting growing public participation.
Citizen science allows the public to participate in scientific projects by systematically collecting data and observing natural phenomena. Science research is thus no longer the privilege of a few, but is open to all. The earliest modern citizen science project was the Christmas Bird Count event in the United States, which started in 1900. Since then, the count has become the most important data source for research on bird dynamics in North America (Silvertown, 2009). With the rapid development of such new technologies as the internet and big data, many possibilities have arisen for citizen science, especially in ecological and environmental science.
Educational ideas and various successful cases from abroad have inspired environmental educators in China. However, as foreign educational resources were introduced, our designers of environmental education activities were puzzled by many issues. For example, how are we to localize environmental education, making our own environmental education projects more down to earth and reflective of local cultural connotations and characteristics? How are we to guide young people to participate in citizen science?
The countryside is rich in high-quality natural educational resources. Mountains, streams, paddy fields, beautiful forest scenery, the quiet environment, rich species of birds, insects and vegetation and complex ecology all constitute natural educational resources. To tap and use natural resources in the countryside, we have tried to integrate advanced international educational ideas into rural education, combine environmental education and citizen science, and organize activities to let the public, especially children, gain a clearer understanding of the land where they are raised, experience the beauty of the countryside, nature and ecology, find the meaning of life, and create a suitable environment for people to live in harmony with nature.
For this reason, we have chosen the local environment for this research, and designed and conducted the Spring Swallow Survey project. Jointly launched by the Hubei Science and Technology Museum, the Cuncaoxin Rural Environment Protection Association and the Jingshan County Education Bureau, the project has different aspects, including knowledge, experience and methodology. We organized primary and middle school students in Jingshan County to observe and record migrant spring swallows, including barn swallows (
This paper tries to explore the value of citizen science activities as resources for environmental education by performing a case study on the Spring Swallow Survey project. It analyses the design ideas and effect evaluation data of the project. We hope that, through the project, we can find more ways for young people to participate in citizen science projects.
In this project, we do not restrict bird-watching activities to watching and recognizing birds, writing about birds and drawing pictures of them. Instead, we ask the students to record original data in a scientific way, learn basic statistics and analysis, and use scientific perspectives to understand the environment just as scientists do. The project advocates the design vision that ‘our countryside and our environment are protected by ourselves’, which allows the public to explore nature in a simple and easy way, discover the value of nature and ecology, and develop a deep love for and imagination about nature.
A logical model of project design
The logical model designed for the project aims to provide a graphical or narrative description of its resources and expected objectives. It includes five parts (resources, activities, outputs, effects and impacts), which constitute a complete process (McCawley, 2001). The available natural resources are used to design activities, which are then carried out to produce some direct and measurable results, such as the number of participants, the length of participation, the number of teachers trained, the results of curriculum development, and so on.
Through these activities, the results expected by the designers are obtained, such as the answers to such questions as:
How were participants and the environment affected?
Was environmental knowledge increased?
Were the participants interested in nature and scientific research?
What changes occurred in individuals, groups, communities and ecosystems?
These results are measured through short-, medium- and long-term effects that lead to the ultimate impact. Based on this, we designed the logical model of the Spring Swallow Survey project (Figure 1).
The logical model of the Spring Swallow Survey project
In the late 1950s, project evaluation began to permeate from formal education into the field of informal education. As an essential research procedure in activity planning and implementation, project evaluation is not only the assessment of the final effect of a project, but also one of the means to ensure that the project achieves its expected goals. The formulation of evaluation indicators and the establishment of a framework for them play an important role in the entire evaluation process. For original education projects, the evaluation includes not only the analysis of data collected from the project objectives, but also the analysis of project planning and its educational effect, which is a relatively complex process. For the Spring Swallow Survey project, the evaluation object featured the educational effect on the audience and the details of project implementation. The purposes of evaluation are to optimize the project design and then improve the project strategy based on data analysis; assess the effectiveness of the educational effect by collecting data and information; and, more importantly, to help project designers reflect on the project's expectations and future plans. Therefore, it is necessary for project designers and implementers to evaluate the effectiveness of the project and ultimately improve its quality.
Qualitative analysis
Qualitative research is defined as:
a kind of activity that takes the researcher himself as a research tool and uses various methods of data collection (interview, observation, physical analysis) to conduct in-depth and holistic inquiry into the research object, form conclusions and theories from the original data, and obtain explanatory understanding of the behaviour and meaning of the research object through interaction (Chen, 2000, p. 12).
This kind of research traces the occurrence and development of things and is easy to conduct. As an evolving process, it does not validate hypotheses in advance, but forms evaluation questions and conclusions after in-depth investigation on the ground (Chen, 2010). Qualitative research arises from original data and, through general induction, the concept and theme of analysis are formed from the perspective of the audience. Then the final conclusion of the evaluation is produced by combining the analysis and interpretation of the evaluator.
In the Spring Swallow Survey project, because it was the first time that swallow-related data were systematically collected and the reproduction of spring swallows was systematically observed by primary and secondary schools in Jingshan County, the students and teachers participating in the project initially did not understand the essence of participating in such citizen science activities. There was doubt about whether the objectives of the project could be achieved. Therefore, we used qualitative analysis. We used quantitative methods to collect empirical data for statistics and induction, employed qualitative methods to analyse the original data and various interviews, and finally arrived at the conclusion of the evaluation of the project.
Generic learning outcomes
Generic learning outcomes (GLO) are the results of the Learning Impact Research Project commissioned by the Council for Museums, Archives and Libraries. The project was undertaken by the Research Centre for Museums and Galleries at the University of Leicester, and lasted from 2001 to 2004. GLO are used in some large projects of British museums to assess learning in the museum environment. They are also used by cultural institutions to plan and evaluate activities (Graham, 2013). GLO cover five dimensions (Figure 2): knowledge and understanding; skills; attitudes and values; enjoyment, inspiration and creativity; and activity, behaviour and progression. Each part covers specific contents (Hooper-Greenhill, 2004). GLO do not provide a set of operational methods for evaluation, but do offer a method and a reference template for analysing data. It is difficult to test the effect and influential factors of learning in an informal environment. At present, no authoritative system for the assessment of environmental education is available in China. Therefore, when establishing the evaluation framework in the Spring Swallow Survey project, we made reference to the five dimensions of GLO and set up six aspects based on the project's needs and conditions to evaluate the implementation effect (Table 1). We used general qualitative and quantitative approaches to collect and analyse data, and then summarized the objectives that the project could achieve.
Five dimensions of GLO
Evaluation indicators framework of the Spring Swallow Survey project
In a project evaluation framework, the indicators are measures reflecting the final results of the evaluation and the most direct source of data. Therefore, the design of the indicators has the most direct impact on the evaluation results (Li, 2013). Based on the objectives to be achieved by the Spring Swallow Survey project and drawing on qualitative analysis and GLO evaluation theory, we consulted Li's ‘On the construction of an evaluation index system for the audience of original exhibitions’ (2013), Zhang and Zheng's ‘Research on the effect evaluation framework of large-scale popular science activities’ (2013), Wu's ‘Establishment and application of an environmental education effect evaluation system’ (2016) and Academician Wei Yu's relevant research on brain science and science education. We then established the evaluation framework for this project (Table 1). The framework is divided into three levels and includes two Level-1 indicators, 10 Level-2 indicators and 49 Level-3 indicators.
Level-1 indicators form the target layer, which is a classification of the scope and type of evaluation. In other words, these indicators should attend to the effect of planning and design, and also need to test the final effect of implementation. Level-2 indicators form the guideline layer, which corresponds to the direction of testing of Level-1 indicators. For ‘planning and design’, the Level-2 indicators include four aspects: the design of themes; the selection of implementation sites; analysis of the audience; and the design of handbooks. They cover multiple aspects that need to be examined in project design and have a certain guidance value for future project implementation. Compared with the evaluation of the effects of formal learning, that of environmental education projects has always been challenging. Using GLO and Wei Yu's ‘social–emotional competence’ assessment, we established six Level-2 indicators for the ‘implementation effect’ in the Spring Swallow Survey project. They are sense and perception; the acquisition of knowledge; the mastery of skills; expressions of attitude; changes in behaviour; and the performance of social–emotional competence. Level-3 indicators form the scheme layer and offer a detailed explanation of Level-2 indicators. They are measures to be taken for the ultimate implementation of the evaluation indicators.
Research methods and evaluation tools
In accordance with the established indicators and layers of the evaluation framework, we designed and applied evaluation tools before, during and after the Spring Swallow Survey project. The evaluation required both quantitative analysis and qualitative description. Multiple evaluation angles and methods guaranteed the integration of quantitative and qualitative evaluations. The evaluation tools in our project mainly included questionnaires, observation, interviews, naturalistic observation notes and statistical charts. The quantitative data were processed by SPSS and Excel software, and the qualitative description was analysed using the general inductive approach.
Literature search
We conducted a literature search on studies about the planning and effect evaluation of natural science education projects so as to know about related research and research progress. We designed the localized Spring Swallow Survey project based on swallow survey projects worldwide and developed the evaluation index framework of the project based on relevant evaluation theory.
Questionnaire survey
Questionnaires are tools for collecting data in social surveys. They include sets of purposeful, systematic and orderly questions. In different implementation stages of the Spring Swallow Survey project, we designed pre-test questionnaires and mid-term questionnaires. Targeting teachers and students, the questionnaires were distributed and collected immediately when the respondents had finished answering. The questionnaires served as support data for the project.
Research and interviews
In line with progress in the implementation of the Spring Swallow Survey, we designed interview questionnaires in the middle and later stages. We interviewed participants at different levels face to face and collected their evaluations of the project. Through interviews at different stages, data integrity, diversity and objectivity were guaranteed, providing strong support for the final research conclusions.
Data collection
We adopted qualitative analysis for the evaluation of the Spring Swallow Survey project. Statistical analysis and induction were performed on the empirical data, and the raw data and interview information were analysed. As a result, the project evaluation was based on the collection of the raw data. In the project, we mainly collected naturalistic observation notes and statistical charts. Naturalistic observation notes record what one sees, hears and thinks in the form of both pictures and text. Through the notes, it is easy to analyse whether the students' natural science literacy, language expression and painting ability have improved. In addition, the primary purpose of the Spring Swallow Survey project was to encourage the public to participate in a citizen science project. The students' survey data were one of the core achievements of the project. Therefore, the analysis of the statistical charts of the survey is particularly important.
General induction
General induction provides a simple and clear set of procedures for analysing qualitative evaluation data. Using that method, we can draw reliable and effective evaluation conclusions in the context of focused evaluation questions. The purposes for using general induction are to ‘condense raw textual data into a brief, summary format; establish clear links between the evaluation or research objectives and the summary findings derived from the raw data; and develop a framework of the underlying structure of experiences or processes that are evident in the raw data’ (Thomas, 2006). The process of general inductive analysis is ‘preparation of raw data files (data cleaning)—close reading text— creation of categories—overlapping coding and uncoded text—continuing revision and refinement of category system’ (Thomas, 2006). Although the general induction method does not have the strong theoretical and modelling basis of other analytical strategies, it provides a simple and clear way to reach conclusions on the evaluation questions. Therefore, many evaluators prefer to use general induction to reduce the cumbersome use of other ways to conduct qualitative data analysis. In the Spring Swallow Survey project, we used the general inductive method to conduct qualitative evaluation of interview questionnaires and naturalistic observation notes, from which we were able to use the open-coding method and finally draw evaluation conclusions.
Evaluation data analysis
Our analysis of evaluation data involved pretest data analysis and analysis of questionnaire data, survey results and qualitative data.
Pre-test data analysis
When the Spring Swallow Survey project was launched, 261 primary and secondary school students from seven schools were surveyed through pre-test questionnaires. The questions covered five sections (Table 2).
Statistical results of the pre-test questionnaire of the Spring Swallow Survey
Statistical results of the pre-test questionnaire of the Spring Swallow Survey
From Table 2, we see that the participants had good awareness of environmental protection, which may be related to many years of birdwatching activities in Jingshan County. They generally had basic knowledge of nest habitats, but had little biological knowledge of spring swallows, such as of their appearance and feeding habits, and especially details about their reproduction. Judging from data missing from the questionnaires, questions with an efficiency ratio of lower than 95% all indicated that they were difficult for the students. Nearly 14 students claimed that they did not understand the meaning of the questions (for example, the total number of nests), indicating that they were relatively young and had scant knowledge of ecology, mathematical statistics, poems related to swallows and the use of survey tools. Based on the pictures drawn by the students, only seven students could draw the bowl-shaped barn swallow nest with relative accuracy, only one student was able to draw the red throat of the barn swallow, and no student was able to draw any feature of the red-rumped swallow and its nest. Before participating in the Spring Swallow Survey project, even in Jingshan County, where birdwatching activities are extensive, few students could make careful observations of the swallows around them or knew much about the birds.
In the project, three primary schools and two secondary schools were selected from the seven pilot schools for project tracking. A total of 42 teacher questionnaires and 102 student questionnaires were distributed, with an effective response rate of 100%.
Five of the nine items in the teachers' interim questionnaire (Figure 3) were about ‘planning and design’, and four were about the ‘implementation effect’. A 0–10 scale was adopted to measure the respondents' views, with 0 being the lowest score and 10 the highest. According to the results from the 42 questionnaires, all teachers acknowledged the educational effect of the project, giving an average rating of more than 9 points. Although the score for project planning and design was over 8.36, it was still weak compared with that for the implementation effect. In particular, the rationality of handbook design needs further improvement.
Average scores on teachers' interim survey questionnaire
According to the satisfaction survey of the students (Table 3), 76.5% of the students assigned five smiling faces (representing 5 points in a 0–5 scale) to the project, showing that the project was recognized as worthwhile by most respondents. However, the average score showed that more than half the 7th-graders gave the project a very low score, for reasons that remain unknown. In the future, we will continue to measure 7th-graders' enthusiasm for participating.
Popularity of the project among students
The results reflected in the students' interim questionnaire (Table 4) were basically consistent with data from the teachers' questionnaire. The evaluation of the implementation effect was positive. The students could make active observations and recordings, offer assistance and enjoy sharing. However, the questionnaire was challenging for them; 61.8% needed the guidance of teachers to understand and answer the questions.
Statistics on students' interim survey questionnaire
From the three surveys conducted by 147 students from seven schools, we collected data on the number of barn swallows and red-rumped swallows; their migrating time and locations; their clustering time and locations; the total number of nests (Table 5); and the swallows' nesting environment, location and materials.
Statistical data on barn swallows and red-rumped swallows
Statistical data on barn swallows and red-rumped swallows
It is evident that more barn swallows than red-rumped swallows migrated to Jingshan County for breeding in 2016. They usually migrated in middle and early March, while red-rumped swallows usually migrated from late March to early April. The entire breeding period was approximately six months long. According to the statistics, few swallows could breed two nests of babies one year. The breeding rate could not be calculated accurately because of limitations in the data. Both barn swallows and red-rumped swallows migrated south after clustering in late August or early September. These data are basically consistent with other relevant studies in China and elsewhere.
Based on the statistical data, we plotted data comparison ‘radar maps’ of barn swallows and red-rumped swallows (Figure 4) and the swallow nest distribution map of Jingshan County (Figure 5) in 2016. By combining the two, we found that barn swallows and red-rumped swallows had completely different distribution densities. In Sanyang, Songhe and Yongxing towns in eastern Jingshan County, the number of barn swallows was much higher than that of red-rumped swallows, while in Yonglong and Yangji towns in the western part of the county the number of red-rumped swallows and their nests was much larger than that of barn swallows. This is an interesting phenomenon. Whether the distribution is related to local habitats or geographical characteristics is still unknown, and further investigation is needed to confirm or refute some speculations.
Data comparison radar maps of barn swallows and red-rumped swallows (2016)
Distribution map of swallow nests in seven pilot towns in Jingshan County (2016) Note: The map shows the statistical data from the three surveys conducted by 147 students from seven schools in Jingshan County. On the map, the seven magnifiers represent the distribution of nests in the areas surveyed by the seven schools. For the convenience of the reader, the surveyed areas are enlarged. Each point represents a nest and is coded by a separate number. A total of 146 barn swallow nests and 73 red-rumped swallow nests are marked on the map.
Barn swallows and red-rumped swallows stay in Jingshan County throughout their breeding period. Their nesting habits are different from those of common forest birds. They do not build nests in trees or woodlands, but in the surrounding environment where humans live. According to statistics on their nesting habitats (Table 6), 96% of swallows build nests in places of residence or shops. Among them, 39% build nests in wall corners, 59% build under eaves and 100% build on cement. In addition to building new nests, swallows also return to and reuse their old nests. Most residents of Jingshan County welcome swallows. However, swallows like to defecate directly outside their nests or carry the excrement outside. This makes the area around the nests dirty and smelly. Thus, 11% of the residents did not want swallows to build nests in their homes (Table 7).
Statistical data on swallow nest habitats
Popularity of swallows with local residents
By using general induction, we classified the interview questionnaires of 17 participants at different levels and 89 students' naturalistic observation notes and analysed them by using open coding based on the evaluation framework. The following conclusions are drawn.
Students have learned about swallows, their habitats and environment
Both interviews and naturalistic notes reflected a common feature: before the implementation of the project, students who participated in the survey had almost no ecological knowledge of barn swallows and red-rumped swallows. After participating in the project, they were able to identify the two species and their nests at a glance and accurately draw the differences between their nests. They learned about their shape characteristics, their reproductive process and the ecological environment in which they lived. For example, swallows perch on wires, but not in trees; they like to nest under eaves, but not in trees; adult barn swallows have black beaks, while young birds have yellow beaks; and so on.
Students have learned about the process of investigative learning
Students participating in the project had not been exposed to similar survey projects before. At first, they struggled with data collection, recording and analysis. By the end of their participation, they had learned to collect simple statistics, conduct simple analysis, draw map sketches, mark different items with different colours and plot a complete distribution map of nests. The project enabled them to understand the research approaches behind scientific projects.
Students have cultivated the habit of observing changes in surrounding species
Observing the subtle changes during the entire reproductive process of swallows, from mating, nesting, hatching to nurturing, made it possible for students to learn about the growth of the birds and the role of parent swallows and feel the beauty of life.
Students have access to excellent material
In participating in the project, the students heard a lot of true stories. For example, there were three nests in one resident's house, but only one was used by swallows. The other two were occupied by sparrows, and the swallows had to build another nest. The students also asked many interesting questions. For example, when two nests are connected, is there a door at the connecting part?
Students have enhanced their awareness of environmental protection
Many phenomena attest to the enhancement of the students' environmental awareness. For example, in the past, swallow nests at the schools would be poked off walls and eaves by someone, but there are few such instances now. If students find someone destroying swallow nests, they will stop them. They will also tell local residents to make a small scaffold to catch swallow excrement, which will protect the nests and solve the hygiene problem as well. This shows that the students have changed from being merely part of an audience to being active communicators and actors in environmental protection.
Teachers need to make extra efforts
Most students expressed strong interest in the project. They initially participated in the project out of curiosity and for fun. Parents also supported their children's participation in similar activities. However, most activities were carried out outdoors. The students' safety must be guaranteed in such cases. Therefore, a sufficient number of teachers were needed to assist the students in completing the survey. Moreover, most of the students were left-behind children, and few had suitable access to the internet. Therefore, most of them cannot get guidance from their parents and obtain relevant information through the internet, as children in cities do. When filling out statistical forms and drawing distribution maps, the students were constrained by their limited knowledge. The teachers needed to make additional efforts to assist them. Especially in the primary schools, most students were led by teachers to complete the statistical tasks, which increased the workload and pressure on the teachers and hindered the progress of the project.
An activity toolkit with more resources is needed
To carry out environmental education with scientific research content in rural areas, teachers, as the main activity guides, need a large amount of relevant knowledge, which increases the difficulty of project promotion. An activity toolkit with more resources will inspire more effective knowledge and methods, so that teachers can more easily communicate, and the implementation effect of activities can be enhanced.
Reflections and improvements
Based on our evaluation of the first Spring Swallow Survey project, we believe that improvements can be made in the activity resource pack and through the development of derivative products.
Improvement in activity resource packs
From the implementation effect of the project, we found that handbook design was the biggest obstacle to the development of the project. To address this problem, we modified the 2016 handbook (Figure 6). The students' handbook was broken down into five separate brochures covering the reproductive cycle of spring swallows, including
Spring Swallow Survey handbook in 2017
Design process for the 2017 students' handbook for the Spring Swallow Survey
Developing derivative products of educational projects is an effective way to derive and expand educational effects. Usually, the cultural and knowledge-related features contained in a project will be selected to develop innovative derivative products. The Spring Swallow Survey project is closely related to people's daily life, and derivative products for everyday life cover a wide range of areas and are also very popular. Therefore, we decided to use everyday toys as derivative products of the project. Centring on the habitual characteristics of spring swallows in their breeding period, we took inspiration from the magic cube to design swallow magic cubes as derivative products. We developed three derivatives for audiences of different ages: a simple magic cube, a puzzle magic cube and a rotating magic cube (Figure 7). All pictures used in our magic cubes were hand-drawn. They combined virtual and real aspects to describe different scenes. The swallow was the main character, and realistic techniques were adopted to represent swallows' appearance, features and flying postures, so that people could recognize them at a glance. Freehand techniques were used to depict the scenes and highlight seasonal changes, thus ensuring that the entire picture would produce a sense of reality and perfectly present the artistic beauty of the subject.
Instructions for the swallow magic cubes
As children are growing up, they can experience a phenomenon called ‘nature deficit disorder’. This is not a medical diagnosis, but provides a perspective to help us think about the impact of nature on people, especially children (Louv, 2014). To repair the fracture between humans and nature, the design of environmental education projects should be rooted in the native land and closely related to the ecology of each region.
Birds are among the most sensitive indicator species of the condition of an ecosystem, and their habitats and migration routes will change along with changes in the global environment (Li et al., 2012). If citizen science activities are introduced to local environmental education projects, it will be possible for the public to monitor the composition, quantity, changing trends and group characteristics of native bird species over a long period. Data on those factors reflect changes in the quality of the ecological environment of bird habitats. Citizen science activities will also allow the public to observe local bird species' behaviour and to record, analyse and archive data. People will experience the fun of scientific research and feel the beauty and power of nature. The scientific literacy and environmental awareness of all people involved in the activities can thus be enhanced.
Integrating citizen science activities into the design of environmental education projects requires project implementers to have a certain level of scientific literacy, which raises the threshold for project implementation. If we want to expand the coverage of a project and the dissemination of its results, we should use effective tools to transmit deep professional knowledge in order to make up for any lack of breadth and depth in the implementer's knowledge. Therefore, by designing a standardized, interesting and easy-to-operate toolkit with relevant contents needed in a project, and selecting the cultural and knowledge-related aspects of the project to develop creative derivative products, we can significantly enhance the implementation effect and dissemination scope of the project, allowing many more people to participate in it.
Funding
This study is funded by the Research on Relevant Standards and Specifications of the Modern Science and Technology Museum System project (2015BAK33B01) and the 2017 national science and technology think tank (Hubei) Research on the Construction, Operation and Innovation Models of Science and Technology Museums project.
Footnotes
Author biographies
Nuxue Jiang is a senior engineer of Hubei Science and Technology Museum. Her research interest is in the development of science education projects and science communication.
Lei Zhang is a master's student of Hubei Science and Technology Museum. Her research interest is in the artistic design of environment and natural science painting.
Cheng Jing is a master's student of Hubei Science and Technology Museum. Her research interest is in natural science education and the application of its design technology.