Abstract
This study explores, identifies, and reports on ten key questions at the frontiers of the field of classroom analysis. To achieve this goal, the research team undertook a multi-step exploratory process. First, the team compiled and extracted a list of popular themes in classroom analysis in China and the world from a large body of existing literature, with a particular focus on studies published in the last decade. Second, the team organized multiple rounds of internal expert discussion and argumentation to narrow the compiled themes to a representative selection. Third, surveys were administered to several international scholars who specialized in classroom analysis to solicit their written opinions and critiques on the selected questions from the previous rounds. Eventually, following several iterations, the research team identified ten key questions at the forefront of international classroom analysis to be reported below.
Disclosing and presenting these questions can help researchers’ ongoing exploration and innovation at the frontiers of classroom analysis through summarizing existing studies, propelling China's empirical classroom research on elementary and secondary education to the forefront of the world. This article was delivered as an oral presentation during the 20th Shanghai International Curriculum Forum (hereinafter referred to as the Forum), which took place at East China Normal University (ECNU) in Shanghai in November 2022. The theme of the Forum was “Technology-Empowered Classroom Analysis: Unraveling the Black Box and Reconstructing Classrooms.” During the Forum, scholars from institutions in China and beyond converged to share their insights into classroom analysis and classroom research.
Taken together, the ten key questions at the frontiers of classroom analysis are centered around three themes. The first theme pertains to values, such as what values should guide classroom analysis. The second theme concerns implementation and practice: As classrooms are often shrouded in “mystery,” this theme consists of questions of what should be done to unravel the “black box” of the classroom and capture what is happening inside. The third theme has to do with teaching practice and educational research, that is, how data can be transformed into evidence to better support teaching, educational research, and other school activities. The following sections focus on each of the three themes by presenting and probing each question within each theme, with the goal of offering recommendations for the future direction of the field and related research. Table 1 provides a list of the ten questions proposed and addressed in this article.
A list of the 10 key questions at the frontiers of classroom analysis.
Theme I: Clarifying values of classroom analysis
To clarify the values of classroom analysis, two questions are proposed. The first question regards the guiding values of classroom analysis. Answering this question involves addressing the crucially relevant question of what constitutes an “ideal” classroom. Currently, there is a lack of a uniform and definite answer to what the guiding values and criteria of an “ideal” classroom should look like. This issue was on full display during the Forum, where experts adopted various perspectives when elaborating on what high-quality classroom dialogue, effective group work, and scientific argumentative discourse that facilitates intellectual and cognitive development ought to look like. As such, there is no such thing as a uniform understanding or standard of a “good” classroom in the academic community. This also prompts the question of whether the Western criteria of a quality classroom are applicable to elementary and secondary classrooms in China. The international experts invited to attend the Forum advocated for classroom activities such as dialogue, discussion, argumentation, and group work (Howe & Abedin, 2013; Rapanta & Felton, 2022). However, as their well-established experience was rooted in the context of Western education, we need to ask whether it actually applies to Chinese classrooms, which differ greatly from a Western one in terms of student–teacher ratio and other aspects of educational resources. This is the first question we need to address before we engage in any kind of classroom analysis, because if we do not even know what an “ideal” classroom should look like in the Chinese context, we are in no position to determine the quality or value of various learning activities taking place in a Chinese classroom.
Clarifications of the criteria of productive classroom teaching and learning bring us to the second question, that is, how to construct a high-quality classroom analysis framework. During the Forum, Yang (2021) shared a set of criteria developed by our research team at ECNU regarding high-quality classroom analysis. This set of criteria involves three levels: classroom efficiency, classroom fairness, and classroom democracy. Each level comprised three dimensions, for a total of nine dimensions. Classroom efficiency encompasses learnability, effectiveness, and enjoyment, which concern whether learning is constructive, whether teaching is focused, and whether learning is pleasurable, respectively. Classroom fairness involves allocation, procedure, and interaction, which pertain to equal opportunities, just procedures, and equitable dialogues, respectively. Classroom democracy consists of safety, autonomy, and cooperation, which represent safe atmosphere, self-regulated learning, and collaborative activities, respectively. Whether these three levels and nine dimensions make up a set of criteria for high-quality classroom analysis that can help us identify and discern the quality of classroom activities in practice is a pressing issue that needs to be studied by researchers.
Theme II: Revealing “truth” about classrooms
Based on the foregoing questions centered around the values of classroom analysis, the research team proposed four questions, focused on implementation and practice, to reveal the truth about the “black box” of a classroom. First, how to collect and construct multimodal data to bring us closer to the truth about a classroom? The notion of multimodal data was mentioned in many of the reports presented throughout the Forum. The concept of multimodal data holds that there are various types of data, including verbal and non-verbal data (Jaipal, 2010). In their respective reports, An (2023) and Xu and Shen (2022) shed light on courseware and lesson cases, which are also categorized as classroom data. Existing classroom studies have devoted greater attention to verbal data (e.g., Hennessy et al., 2020); however, despite its significance, verbal data are just one of the many types of classroom data, which also include non-cognitive data, such as emotional experiences, motivations, and behavioral data such as facial expressions, gestures, and body postures. A classroom is generally a highly complex environment comprising multiple individuals, each engaging in internal cognitive and noncognitive activities that cannot be directly captured by the naked eye. This raises the question of how such verbal, textual, behavioral, physiological, and emotional data, among others, can be obtained. In the field of neuroscience, researchers have initiated attempts to gather multimodal data by asking students to put on wearable devices designed to capture their brain activities, heartbeats, and electrodermal activities during the class (Babiloni & Astolfi, 2014; Cui et al., 2012). Educational researchers have rarely taken up this type of research rooted in neuroscience and the question remains as to whether it is applicable to authentic classroom contexts.
Second, the apparent importance of obtaining multimodal data raises the crucial question of how to enrich classroom data by expanding the diversity of technology. In his report, Yang (2021) mentioned that the quality of classroom videos is paramount insofar as it directly dictates the type, quantity, and quality of classroom data that are subsequently available. The quality of classroom videos extends beyond audio quality, which concerns whether the language used by teachers and students can be heard clearly and can fully support subsequent transcription and analysis work. It also relates to graphics quality, which reflects whether the classroom behaviors—including facial expressions, demeanor, and body language, among other things—of teachers and students are clearly visible in the videos. In other words, high-quality classroom data cannot exist without the support of technology. Among the schools currently partnering with the International Classroom Analysis Laboratory (ICAL) at the Institute of Curriculum and Instruction at ECNU, several have set up dedicated video-recorded classrooms. Each of these classrooms is equipped with multiple cameras that can clearly capture the classroom behaviors of teachers and students from various angles and, in some cases, specifically record students’ group work behaviors. Thus, while multimodal data are paramount, it is just as important to set up technology-empowered equipment to gather such data.
Third, how to enhance the automatic coding of classroom data? During the Forum, both Chen et al. (2015) and Yang (2021) mentioned automatic coding in their reports. What makes automatic coding indispensable? To answer this, it is necessary to elucidate the concept opposite to that of automatic coding, namely, manual coding. Manual coding is no stranger to researchers (e.g., Hennessy et al., 2020; Mercer, 2010). Over the last two decades, Cui and his team of researchers have conducted an extensive number of studies surrounding classroom observations, which involve live coding of classroom discourse or behaviors (Cui, 2012). These studies are highly valuable and have laid a solid foundation for classroom analysis, which often involves coding based on classroom videos or audios. That said, as human processing of data is very limited by nature, the availability and capacity of experts are often insufficient to match the number of classroom videos that need to be analyzed. Moreover, teachers and students often require timely feedback, but analytical procedures such as transcription, coding, and statistical analyses during manual analyses of classroom videos tend to consume a large amount of time and effort (Chen, 2020). Timely feedback on classroom teaching is particularly important for teacher professional development (PD) or collaborative reflection sessions, during which classroom observers or experts are expected to provide feedback on the teacher's performance. During these PD or reflection sessions, observers or experts often share their subjective interpretation of classroom events, but they rarely back up their claims by drawing on evidence or results obtained through manual coding. One of the reasons was that it was very difficult to complete manual coding of classroom data prior to these sessions.
Automatic coding, namely, coding that uses artificial intelligence to automate the process of assigning a turn or event to a pre-defined category, has the potential to liberate manpower and substantially reduce the amount of time researchers need to transcribe and code videos (Chi, 1997). However, automatic coding is beset by a multitude of potential problems. For instance, it may achieve the goal of analyzing a copious amount of data at the expense of the depth of such analyses. To date, classroom analyses have tended to focus on qualitative research. Xiao's (2022) report offered several remarkable examples of qualitative case analyses. Although qualitative analyses generally involve smaller data sizes due to their highly time-consuming nature, they tend to go into great depth and reveal extensive characteristics about the classroom and the sociocultural context in which it is situated (Mercer, 2010). Conversely, automatic coding can handle and analyze a vast amount of data within a short period of time. That said, where manual coding can achieve high-inference analysis, automatic coding is typically conducted in a low-inference manner and characterized by a relatively rudimentary coding framework and the inability to accurately detect subtle nuances in the language (Hennessy et al., 2020). As such, manual coding can offer more in-depth insights into the classroom activities of teachers and students, including the cognitive levels attained by students and the adequacy of the evidence and explanations used in their arguments. Given their respective advantages, automatic and manual coding should form a mutually complementary relationship in which they are used in tandem. Automatic coding is essential when researchers need to provide teachers with timely feedback, while manual coding is indispensable when it becomes necessary to analyze classroom data in depth.
The last question in this theme is how to build a large-scale, non-invasive classroom analysis system. Methods involving setting up video recorders in the classroom and putting wearable devices on teachers and students are invasive and might interrupt the natural flow of teaching and learning. Under these circumstances, teachers and students behave and conduct themselves differently in the classroom in terms of both their verbal and non-verbal behavior than when they would under usual circumstances. Hence, if the goal is to uncover the real classroom and collect the most “authentic” data, we need to contemplate how best to gather a large amount of classroom data in a non-intrusive and routine fashion, and whether doing so is achievable with our current technological capacity or what kinds of technological solutions are needed to achieve this goal in the future. It is imperative that we answer these questions in future studies.
Theme III: Transforming data into evidence
The goal of classroom analysis is by no means to analyze data per se; its purpose and value can only be truly realized if the analytical results can be fed back into pedagogical practices and substantively improve classroom efficiency and the quality of teaching and learning. When transforming data into evidence, the first question that needs to be addressed is an ethical one, namely, how to construct and enforce a code of ethics for classroom analysis. During the Forum, many experts pixelated the faces of teachers or students shown in videos or pictures when presenting their analytical results. In doing so, researchers were following a code of ethics for classroom analysis. Individuals have a right to privacy, and it is especially vital to protect the privacy of children and minors, who are particularly vulnerable. How, then, can researchers protect the safety and privacy of students and teachers while maximizing the amount of classroom data captured, extracted, and analyzed in elementary and secondary schools? What code of ethics should they establish for classroom analysis to achieve this goal? Admittedly, the ethical development of pedagogy as a discipline may be trailing behind other academic fields, such as psychology. In the field of psychology, before research projects commence, they first need to pass stringent ethical reviews involving clear stipulations as to what researchers can and cannot subject the participants to in the research or experimental setting. However, when researchers use video recorders to collect data in elementary and secondary schools, they seldom give much thought to matters such as protecting the privacy of students and teachers and ensuring that the data will not be leaked. Thus, in the field of classroom analysis, efforts need to be taken to improve and strengthen ethical restrictions and regulations and publish a more sophisticated, binding, and uniform code of ethics.
With a powerful code of ethics in place, the next question to address is how to use classroom data to build a profile of students and teachers. Despite the increasing attention it has gained among educational researchers and practitioners in recent years, profiling remains a highly contested concept. Many scholars perceive profiling as a process that extracts and makes abstract generalizations about an abundance of teacher and student data, inevitably erasing the unique characteristics and personalities of individuals due to the overemphasis on commonalities. Therefore, when profiling teachers and students, researchers should try to keep a balance between their commonalities and individualities. In other words, they should reflect on how to retain as many individual differences and unique characteristics as possible, while extracting information that is applicable to the majority of students and teachers.
Transforming data into evidence also prompts the question of how to build a model to meet users’ needs for teaching, learning, evaluation, and management. Answering this question is integral to feeding the results of classroom analysis back into teaching. During the Forum, Chen et al. (2015) presented a report on an in-depth study he conducted about enhancing teachers’ professional competencies through classroom analysis. In this report, he elucidated and detailed the outcomes that he and his team had achieved regarding their research on classroom language and dialogue. Over the last few years, the ICAL has formed close partnerships with more than 100 elementary or secondary schools across China through the “artificial intelligence + online-merge-offline” (AI + OMO) and the “Artificial Intelligence Classroom” (AIC) initiatives. The goal is to support schools in conducting educational research and improve the quality of classroom teaching by undertaking classroom analysis and generating timely reports in the process. Here, the pertinent question facing secondary and elementary schools is how classroom analysis can promote teaching, learning, evaluations, and educational research. As classroom analysis uses data obtained from authentic classrooms, its results should also be incorporated back into real-life classrooms. Indeed, classroom analysis loses all value if it is detached from the real-life settings of teaching and educational research. Therefore, although it is grounded in empirical research, its application should be persistently practice-oriented, and the quality of analysis should be evaluated against practical outcomes.
Having emphasized the importance of practice-oriented classroom analysis, we once again merge theory and practice with the final question: How to simultaneously achieve data-driven and theory-driven classroom analysis? Although classroom analysis uses practice as its source of data, its analytical orientations and framework require theoretical guidance. In contrast to the “bottom-up” approach to research, which is inductive in nature and ascends from the individual to the general, the “top-down” approach is deductive and descends from the general to the individual. While the constitution of a good classroom has been heavily theorized in both Western and Chinese classroom analysis, empirical data drives researchers to ceaselessly refine and improve these theories. The studies conducted by the ICAL aim to build on the essence of existing research in both Chinese and Western classroom analysis, use theories as a guide to formulate the criteria for high-quality classroom analysis, and develop an AI-powered system for analyzing classroom data to continuously respond to and improve relevant theories. The ten questions proposed and discussed in this article are the product of extensive theoretical argumentation and expert discussions undertaken by the research team. It is hoped that they can offer some level of guidance to subsequent classroom studies and that the ICAL can better address these questions based on more theoretical and empirical research in the future.
Takeaway message
These ten questions revolve around three areas—namely, “clarifying values of classroom analysis,” “revealing truth about classrooms,” and “transforming data into evidence.”
These questions can be harnessed to propel in-depth analysis of classrooms in China and abroad.
This study frames ten questions at the frontiers of research in order to advance the direction of future research in classroom analysis.
