Sage Journals: Discover world-class research

Abstract

The purpose of this study was to determine the topics being studied, theoretical perspectives being used, and methods being implemented in current literacy research. A research team completed a content analysis of nine journals from 2009 to 2014 to gather data. In the 1,238 articles analyzed, the topics, theoretical perspectives, research designs, and data sources were recorded. Frequency counts of these findings are presented for each journal. Chi-square tests of independence revealed statistically significant differences among the topics, theoretical perspectives, designs, and data sources across the nine journals. These results suggest that the field of literacy research may be fragmented, which has been a concern for literacy researchers since the paradigm wars of the 1980s and 1990s. We urge the literacy research community to continue to demand rigorous research, but to do so in a way that appreciates the power in viewing and studying teaching and learning from diverse perspectives, using diverse methods, and with recognition that a foundational aspect of rigorous research is the match between research questions asked and research methods used.

Keywords

literacy research content analysis trends topics

It is the job of literacy scholars to stay current in the field of literacy research. However, staying current is becoming increasingly difficult as the field has experienced rapid and continued growth in scope, volume, perspectives, epistemologies, and methodologies (Beretvas, 2004; Duke & Mallette, 2011; Kamil, Afflerbach, Pearson, & Moje, 2011; Lomax, 2004). This expansion has resulted in heated debate about appropriate perspectives and methods, as evidenced by the “reading and paradigm wars” of the 1980s and 1990s, which pitted whole language against phonics and quantitative research against qualitative research (Kamil, 1995; Kamil, Afflerbach, et al., 2011; Pearson, 2004; Stanovich, 1990). The new millennium brought No Child Left Behind (NCLB; 2002) and the Report of the National Reading Panel (National Institute of Child Health and Human Development [NICHD], 2000), which heightened debate about epistemology and methodology (e.g., Cunningham, 2001; Shanahan, 2003; Tierney, 2014; Yatvin, 2002). Some feared that these debates would lead to fragmentation in the field of literacy where researchers with different epistemologies who use different research methods would separate and isolate themselves (e.g., Stanovich, 1990). Indeed, research organizations such as the National Reading Conference (now the Literacy Research Association) experienced divides as researchers left the organization to form other groups such as the American Reading Forum and the Society for the Scientific Study of Reading (Duke & Mallette, 2001).

In 2001, Duke and Mallette hypothesized that the field of literacy research would welcome “ecological balance,” an understanding of and appreciation for different paradigms and methodologies, as opposed to fragmentation. More recently, in the preface to the fourth volume of the Handbook of Reading Research (Kamil, Pearson, Moje, & Afflerbach, 2011), Kamil, Afflerbach, et al. (2011) expressed that the field has experienced a shift toward mixed-methods designs: “The mantra of what works for whom, and when is increasingly taken up by those interested in student learning” (p. xviii).

In light of this sometimes-fractious history, we conducted a content analysis of literacy journals to develop a picture of what is currently being published in the field of literacy. In our investigation, we answer the following questions:

Research Question 1: What are the topics, theoretical perspectives, research designs, and data sources in the articles published in nine literacy journals over the last 6 years?

Research Question 2: Is there a relationship between the topics, theoretical perspectives, research designs, and data sources and the journals in which they are published?

Related Research

In conducting this analysis, we follow other researchers who have explored trends and issues in literacy research. Past content analyses have often been organization-specific, such as the retrospective analysis of the National Reading Conference’s publications, conducted by Baldwin and colleagues (1992), which summarized 40 years of trends in the Journal of Reading Behavior (JRB; now the Journal of Literacy Research, JLR) and the NRC Yearbook (now Literacy Research: Theory, Method, and Practice). Baldwin et al. found that comprehension was the most widely researched topic, with methodological analysis shifting from a steady stream of ANOVA and correlational research to more qualitative studies. Furthermore, they identified patterns of increased co-authorship, along with a critical examination of some aspects of professionalism, including the relationship between researchers and classroom teachers.

Several content analyses of single journals over long periods of time focused on approaches and authorship changes. For example, Dunston, Headley, Schenk, Ridgeway, and Gambrell (1998) examined 694 research studies in 20 years of the NRC Yearbook, 189 of which were qualitative studies, 464 of which were quantitative studies, and 41 of which were studies that used both quantitative and qualitative analysis procedures. Of the 11 topics of research interest identified, six topics emerged as most frequently researched over the 20-year time span: (a) adult/college/family literacy, (b) beginning/early/emergent literacy, (c) comprehension, (d) instruction, (e) students, and (f) teachers.

Similarly, Guzzetti, Anders, and Neuman’s (1999) analysis of the JRB/JLR spanned 30 years and focused on changes in authorship (i.e., number of authors for each article and the gender of these authors), participants in each study (i.e., gender, socioeconomic status, age, and ethnicity), and the research topics and methods. Findings from the analysis demonstrated that over the course of these 30 years, articles in JRB/JLR typically had multiple authors, indicating an interest in collaboration among literacy researchers. In addition, there were an equal number of male and female authors throughout the history of the journal. Last, researchers discovered that the majority of articles were quantitative rather than qualitative, and focused on elementary school children and on topics such as readers rather than on text, instruction, or assessments.

Taking a different approach, McKenna and Robinson (1999) analyzed the citation frequency of JRB/JLR articles in the Social Science Citation Index (SSCI) as a means to establish the journal’s impact factor. They discovered that JRB/JLR’s most influential volumes were published from 1973 to 1984, with citations in the SSCI tapering off after 1985, a finding that correlated with the most-cited articles, one in Volume 11 and one in Volume 13. McKenna and Robinson surmised that an increase in the overall number of journals from 300 in 1969 to 780 in 1999 might have influenced the decline in the journal’s citations after 1985.

More recently, content analyses have focused on the increasing diversity of approaches in literacy research. Still and Gordon (2011) completed a content analysis of Association of Literacy Educators and Researchers’ (ALER’s; formerly the College Reading Association) conference programs from 1960 to 2010 to commemorate the 50th anniversary of the organization, offering insights into the historical context and professional implications for changes in research topics that reflected a greater interest in diversity and a sustained commitment to the meaning-making role of reading. In addition, Morrison and his colleagues (2011) conducted a content analysis covering the authors, methods, participants, and topics in ALER’s journal, now Literacy Research and Instruction (LRI), from 1961 to 2011. In their analysis of 1,099 articles, they observed an increase in empirical research articles in later volumes, along with a greater frequency of qualitative research methods, which they believed signaled an increasing demand for high-quality research as well as a greater tolerance for diverse research methods—a finding that is in line with Guzzetti et al.’s (1999) findings. Schumm, Lewis-Spector, Price, and Doorn (2014) expanded on this effort with their content analysis of LRI’s treatment of preservice teacher education in literacy. The researchers identified 71 articles related to preservice teacher education, about 6.5% of the 1,099 articles published between 1961 and 2011 in LRI. Interestingly, and specifically related to the current study, few of the identified articles identified theoretical perspectives.

Two recent content analyses also look at single journals over time. Reutzel and Mohr (2015) looked at the ways Reading Research Quarterly (RRQ) informed the field of literacy over the past 50 years. Using quantitative approaches, the researchers analyzed research articles in 198 issues of RRQ focusing on the topics studied. They also analyzed the relative standing of RRQ in relation to other, similar publications. Reutzel and Mohr’s approach is unique in that they included qualitative analysis of editorials over the past 50 years to determine the goals of RRQ’s editors. The researchers found that instructional practices and programs, comprehension, and reviews of research were the most common topics of literacy research published in RRQ over the last 50 years, and were surprised to find that oral language development was the topic of only seven articles, in their sample, during this time period. They also identified what they called the five pillars of RRQ’s contributions to literacy research: RRQ is a “go-to” journal for high-quality research; it is a nexus of professional dialogue; over the years, it has mirrored the evolution of the field of reading; it is a top-tier journal; and its mission is to provide research that informs practice.

Bauer and Theado (2015) conducted a content analysis of the Journal of College Reading and Learning (JCRL). These researchers identified and analyzed 62 articles from 2005 to 2013 to determine which pedagogical practices have affected literacy learning for college-aged readers over the years. They also analyzed the theoretical frameworks researchers used to guide their interpretations of their results. They identified three areas that heavily influenced college students’ reading instruction: metacognition, self-regulation, and self-efficacy/motivation. Furthermore, they identified three main theories undergirding researchers’ inquiry: social constructivist, social learning, and social cognitive theories. Bauer and Theado worried about the mismatch between the researchers’ purported frameworks of literacy as social practice that guide the inquiries and the emphasis in these articles on how students should learn, with literacy events categorized as individual, text-based acts. Such a mismatch may be said to reflect the need for a critical discourse about research methods in the field of literacy (Duke & Mallette, 2001).

The bulk of the literature on content analysis has centered on the topics, research methods, and shifts of one or two publications over time—often decades—and helps develop a picture of each journal’s contributions in an effort to capture how its past might influence its future. Despite this pattern of looking back to look forward, few studies have examined the contents of several journals. One study that looks at literacy scholarship over several journals was conducted by Knudson, Onofrey, Theurer, and Boyd-Batstone (2002), who examined literacy research published in three journals: Journal of Research and Development in Education (JRDE), Journal of Educational Research (JER), and Journal of Experimental Education (JEE) over a 10-year span. Results from this study showed trends in literacy research as well as demographics of authors and participants. JRDE had more experimental designs during the last half of the decade and focused more on elementary and college students, whereas JEE was entirely empirically based and had twice as many male single-authored studies as female single-authored studies. In addition, nearly 80% of the articles focused on elementary education. JER, however, experienced a sharp decline in the number of literacy articles after 1996 and, similar to the two previous journals, focused on elementary-age participants. However, the selection of these particular journals is peculiar as none of them is explicitly devoted to literacy research.

The research reported here builds upon this previous work but takes a different approach, analyzing the content of nine publications over a 6-year period, including JLR, LRI, RRQ, Research in the Teaching of English (RTE), Scientific Studies of Reading (SSR), Journal of Research in Reading (JRR), Reading Psychology (RP), Reading and Writing (R&W), and Reading and Writing Quarterly (RWQ). Because so many past content analyses have looked at one journal over a long period of time, we opted to look at nine journals over a short period aiming for a snapshot of the current field rather than examining trends over time, as others have done. By focusing on research conducted over the past 6 years, we also hoped to identify patterns that would suggest the ecological balance Duke and Mallette (2001) hypothesized emerging in recent literacy scholarship.

Perspective

In analyzing multiple journals, we aimed to capture a broader view of literacy scholarship in terms of audience and purpose than was evident in previous content analyses. Because conceptions of literacy are constantly evolving (Leu, 2000), it is critical to understand how prior definitions of literacy form the basis of new definitions. For this reason, we used a history of science perspective, which maintains that paradigms, as basic theoretical understandings and assumptions held by groups of scholars within the same discipline, inform scientific knowledge at specific times (Fleck, 1979; Kuhn, 1962). Fleck called groups of scholars within the same discipline working in similar time periods “thought collectives.” He described how thought collectives develop “thought styles,” which parallel Kuhn’s paradigms in that both are undergirded by common understandings and assumptions held by a community of scholars. From this perspective, significant scientific progress takes place when there are paradigm shifts. Kuhn explained that when a scientific community faces a series of anomalies that cannot be explained by the current paradigm, the result is a crisis. To resolve the crisis, another paradigm, driven by advances in research, must take its place; thus, a paradigm shift occurs (Kuhn, 1962).

The scholarship published in high-profile literacy journals represents the current conversation in the field—the “paradigm” or set of paradigms of the current literacy research “thought collective” (or thought collectives). Although a history of science perspective, by nature, takes a long view, looking at progress over time and the scientific revolutions that have taken place, it is important to carefully examine the current state of a thought collective to capture the paradigm or set of paradigms that exist and dominate. This allows scholars in the field to see the existing paradigms and thought collectives, which can lead to more collective understanding and ultimately more scientific progress. Such is the goal of the current project. We documented and present the topics, theoretical perspectives, research designs, and data sources in nine literacy research journals over the last 6 years to present a picture of the current paradigms and thought collectives that exist in the field of literacy research.

Method

To investigate the research questions, we conducted a content analysis (Holsti, 1969; Krippendorff, 2004). Hoffman, Wilson, Martínez, and Sailors (2011) explained, “content analysis is a flexible research method for analyzing texts” (p. 29). We used this method to examine the topics, theoretical perspectives, research designs, and data sources found in articles published in nine literacy journals. Key considerations when engaging in content analyses are sampling, objectivity, and systematicity (Holsti, 1969; Krippendorff, 2004). Sampling is important because it determines the validity of the content analysis. Can the documents analyzed answer the research questions presented? Objectivity requires researchers to eliminate, as much as possible, personal predispositions by following rules and procedures (Holsti, 1969). This aspect of content analysis speaks to reliability (Krippendorff, 2004). Would other researchers who analyzed the same texts draw the same conclusions? Systematicity, which is closely related to objectivity, refers to the extent to which the researchers are systematic in all processes of the study, including sampling and coding (Holsti, 1969). In the following, we demonstrate how we strove to be systematic and objective in all aspects of our content analysis, including our sampling of texts.

Our sampling occurred in two stages. Because we wanted to look at the most current literacy research being published, we originally limited our study to the most recent 5 years for which complete volumes were available: 2009-2013. In this stage, to determine the journals, the lead researcher informally surveyed literacy colleagues. He asked 20 literacy scholars from across the United States for their opinion of the five most influential literacy research journals in the field. Although most of the respondents replied to this request accordingly, others indicated that they were unable to limit their response to only five; in these cases, all responses were recorded. We took the results of this informal survey and removed journals that (a) served primarily a practitioner audience (The Reading Teacher, Language Arts, and Journal of Adolescent and Adult Literacy) and (b) were not devoted specifically to literacy research (Elementary School Journal and Journal of Educational Psychology). This left us with five journals: JLR, LRI, RRQ, RTE, and SSR.

In Phase 2, this list was compared with impact factor databases: Thomson Reuters’ Journal Citation Reports and SCImago Journal Rankings (www.scimagojr.com). We then included literacy journals that were at the top of these lists but did not emerge from the informal survey of colleagues. Therefore, we added JRR, RP, R&W, and RWQ. This selection process allowed us access to an additional year of complete volumes. Because the purpose of this content analysis was to present a current picture of literacy research, we included the most recent year in our analysis, giving us a 6-year “snapshot” of the field of literacy research.

Data Sources

The nine journals included in this content analysis are presented in Table 1, which displays the journals as well as their impact factors and ratings.

Table 1.

The Nine Journals Analyzed and Their Impact Factors.

Journal	Reuters impact factor	SCImago journal rank
Journal of Literacy Research (JLR)	0.656	1.32
Journal of Research in Reading (JRR)	1.470	0.81
Literacy Research & Instruction (LRI)	not listed	0.47
Reading Psychology (RP)	not listed	0.44
Reading Research Quarterly (RRQ)	2.880	2.65
Reading and Writing (R&W)	1.630	1.34
Reading & Writing Quarterly (RWQ)	0.697	0.44
Research in the Teaching of English (RTE)	0.871	0.84
Scientific Studies of Reading (SSR)	2.950	2.14

Every article in each of the nine journals in this study from 2009 to 2014 was analyzed. These analyses included researchers documenting the (a) topic, (b) theoretical perspective, (c) research design, and (d) data sources from each article. It is important to note that to reduce researcher bias and move toward objectivity, the researchers only reported what the authors of the articles within the journal listed for each of the areas. For example, if the authors did not explicitly state a theoretical perspective, the researchers did not seek to infer one but rather coded this “not specified.” In addition, researchers coded up to three main topics using the keywords, title, and abstract of the article, and up to three data sources as stated in the “Method” section. Therefore, the frequency counts for topics and data sources are not consistent with the number of articles because many articles had more than one topic and/or more than one data source. The study was limited to articles; that is, editorials, book reviews, interviews, policy statements, and the like were not analyzed. We used an Excel spreadsheet to catalog these data (see Figure 1).

Figure 1.

Excel headings and the dropdown options in the spreadsheet.

Data Analysis

Developing codes

To begin the coding process, the lead researcher created codes using a variety of sources. To create potential topic codes, the lead researcher used the most current “What’s Hot” list (Cassidy & Grote-Garcia, 2013) and the table of contents of current comprehensive literacy research synthesis texts: Handbook of Reading Research (Kamil, Pearson, et al., 2011), Handbook of Research in the Teaching of the English Language Arts (Lapp & Fisher, 2011), and Best Practices in Literacy Research (Morrow & Gambrell, 2011). To create potential theoretical perspectives used, the lead researcher consulted Theoretical Processes and Models of Reading (Alvermann, Unrau, & Ruddell, 2013) and Lenses on Reading (Tracey & Morrow, 2012). To create potential research design and data sources codes, the lead researcher consulted research design and educational research texts (Creswell, 2007, 2013). The research team used these codes in an initial round of data analysis of 949 articles from nine journals.

During a research team meeting following the first round of analysis, it was evident that the predetermined categories used to code the topics, theoretical perspectives, and research designs were not expansive enough. For example, the code “Other” was applied for nearly half of the topics and a large number of theoretical perspectives. Accordingly, the lead researcher compiled the list of topics and theoretical perspectives that were coded “Other” to pull out additional codes that could more comprehensively capture the topics and theoretical perspectives being studied. After the “Other” topics and theoretical perspectives were recoded with these additional codes, fewer than 7% of the articles remained in the “Other” category. The lead researcher also defined each topic code (initial and revised), so there was a shared understanding among the team (see the revised coding sheet in Figure 2).

Figure 2.

Updated codes with definitions for topics and “rules” for applying codes.

During a research team meeting following the first round of analysis, the team decided to code topics based on keywords identified in the article, allowing up to three topics per article. If an article did not have keywords, then the title and abstract were used to determine the topics covered by the article. The team also decided that unless the theoretical perspective was explicitly named in the article, it was to be coded “Not Specified” to ensure that we were not making inferences based on other information in the article. In addition, the research team decided that the “rule” for coding research designs was that it must be named (e.g., This is a ___ study or This study used a ___ design or This study’s design is ___), and if the design was not explicitly specified, the study would be examined and categorized generally as Qualitative, Quantitative, Mixed Methods, or Non-Empirical. Again, this decision increased the researchers’ consistency in understanding and reliability in coding for the second round of analysis because of these systematic processes and movements toward objectivity (Holsti, 1969; Krippendorff, 2004).

After the second round of the analysis, we analyzed inter-coder consistency to establish reliability among coders using a random number generator to select 10% of the issues. From these selected issues, each researcher coded articles in a journal previously coded by another team member and then compared the results with the original researcher’s codes. This inter-coder consistency check revealed 87% agreement between coders, and relatively rare discrepancies were discussed and reconciled.

To answer the first research question (What are the topics, theoretical perspectives, research designs, and data sources in the articles published in nine literacy journals over the last 6 years?), we calculated frequency counts for topics, theoretical perspectives, research designs, and data sources in each article (N = 1,238). To examine our second research question (Is there a relationship between the topics, theoretical perspectives, research designs, and data sources and the journals in which they are published?), we first ran four chi-square tests of independence to determine whether there was a relationship between these variables (i.e., topics, theoretical perspectives, designs, and data sources) and the journals in which they were published. The chi-square tests also allowed us to determine whether the nine journals were significantly different from one another based on the variables. If there exists an “ecological balance” in the field (Duke & Mallette, 2001), then we would expect to find no association between the variables and the journals, as similar topics, theoretical perspectives, designs, and data sources would be published in each of the journals.

Results

Frequency counts revealed the topics, theoretical perspectives, research designs, and data sources that were most common in nine journals from 2009 to 2014. In this section, we first present the most frequent codes for each of these components and then present the results of the chi-square tests of independence.

Topics

Frequency counts demonstrated that the five topics most written about in these nine journals were (a) Comprehension, (b) Bilingual/ELLs (English language learner), (c) Phonemic/Phonological Awareness, (d) Instruction, and (e) Writing. A chi-square test of independence was conducted to examine the relationship between article topics and the journal in which they were published. The relationship between these variables was significant, χ²(280, N = 2,967) = 1,620.20, p < .001. The effect size, V = .27, indicated a moderate effect (Cohen, 1988). Table 2 presents the frequency counts for each topic in each of the journals. The chi-square results indicate that these topics were not evenly represented across the journals.

Table 2.

Frequency of Topics by Journal.

Topics	Journals									Total
Topics	JLR	RRQ	LRI	SSR	RTE	JRR	RP	R&W	RWQ	Total
Adolescent literacy	18	6	12	0	20	5	12	23	4	100
Adult readers	7	6	8	5	12	17	14	32	0	101
Bilingual/ELLs	14	12	10	10	18	50	18	97	8	237
Cognition	1	12	0	11	0	9	3	25	1	62
Comprehension	7	31	26	28	5	34	45	61	9	246
Culture	26	16	4	2	15	6	6	10	13	98
Development	0	1	0	10	0	4	3	17	2	37
Disciplinary/content area reading	8	11	10	2	11	0	9	1	0	52
Emergent/pre-literacy	7	6	9	1	0	16	13	47	0	99
Family literacy	3	0	3	6	0	6	5	13	6	42
Fluency	2	6	11	19	1	8	13	29	2	91
Instruction	14	26	26	4	10	7	30	35	15	167
Lenses of readers	4	3	6	0	14	5	2	2	0	36
Literacy coaching	3	2	5	0	0	0	1	1	14	26
Motivation/engagement	6	13	8	2	2	11	23	13	2	80
New literacies	17	15	7	2	15	15	10	11	2	94
Phonics	0	2	1	1	0	4	3	18	0	29
Phonemic/phonological awareness	2	12	1	37	0	30	5	86	1	174
Policy	1	0	2	0	2	0	0	0	0	5
Professional Development	7	3	9	0	9	2	0	6	2	38
Reading processes	1	4	0	23	0	15	9	49	1	102
Assessment	4	4	11	4	2	5	8	7	11	56
Classroom context	2	3	0	0	1	3	2	2	3	16
Spelling	0	4	1	34	0	25	4	72	1	141
Struggling readers	5	7	16	24	4	23	15	53	15	162
Students with particular needs	0	2	1	10	0	5	4	30	4	56
Teacher characteristics	4	4	7	0	3	2	7	5	2	34
Teacher education	7	2	26	1	7	0	4	4	1	52
Teaching technique	0	1	0	0	0	0	0	0	0	1
Tutoring	0	2	5	0	0	0	1	1	0	9
Type of text	4	2	7	0	2	6	15	10	0	46
Vocabulary	7	17	7	28	0	18	12	58	3	150
Word recognition/decoding	1	1	2	19	0	11	4	41	3	82
Writing	11	7	17	2	35	14	5	68	5	164
Neuroscience	0	0	0	2	0	0	0	5	1	8
Other	9	8	3	13	0	3	12	23	3	74
Total	202	251	261	300	188	359	317	955	134	2,967

Note. JLR = Journal of Literacy Research; RRQ = Reading Research Quarterly; LRI = Literacy Research & Instruction; SSR = Scientific Studies of Reading; RTE = Research in the Teaching of English; JRR = Journal of Research in Reading; RP = Reading Psychology; R&W = Reading and Writing; RWQ = Reading & Writing Quarterly; ELL = English language learner.

Theoretical Perspectives

Theoretical perspectives were not specified in 75.69% (n = 937) of all the articles. In addition, 6.79% (n = 84) used a theoretical perspective outside our codes and 3.23% (n = 40) of the articles presented multiple theoretical perspectives. In the remaining articles, the most frequently reported theoretical perspectives were (a) Sociocultural, (b) Cognitive/Information Processing, and (c) New Literacies. A second chi-square test of independence was performed to examine the relationship between the theoretical perspectives and the journal. The relationship between theoretical perspectives and journal was significant, χ²(200, N = 1,238) = 526.43, p < .001. These results also indicated a moderate effect size, V = .28 (Cohen, 1988). The frequencies of the theoretical perspectives are represented in Table 3.

Table 3.

Frequency of Theoretical Perspectives by Journal.

Theoretical perspectives	Journals									Total
Theoretical perspectives	JLR	RRQ	LRI	SSR	RTE	JRR	RP	R&W	RWQ	Total
Behavioral	0	1	0	0	0	0	0	0	0	1
Cognitive/information processing	1	14	0	0	0	0	0	3	0	18
Constructivism	0	1	5	0	1	0	1	1	0	9
Critical literacy	0	0	0	0	3	0	0	0	0	3
Critical perspectives	2	1	0	0	1	0	0	0	3	7
Activity theory	0	0	0	0	0	0	1	0	0	1
Culturally relevant pedagogy	0	0	1	0	0	0	0	0	0	1
Ecological perspectives	0	0	2	0	2	0	1	1	1	7
Emergent literacy	0	4	0	0	2	0	1	0	0	7
Expectancy-value theory	0	0	3	0	0	0	0	0	0	3
Family literacy	0	2	0	0	4	1	0	0	0	7
Goal theory	1	0	0	0	0	0	0	0	0	1
Identity theories	2	1	0	0	5	0	0	0	0	8
Linguistic	0	0	0	0	1	0	0	2	1	4
Metacognition	0	0	0	0	0	0	1	0	0	1
Multiple theories	11	3	2	0	1	3	11	6	3	40
New literacies	5	2	3	0	1	2	1	1	1	16
Not specified	45	49	62	131	30	142	106	328	44	937
Pragmatic perspectives	0	0	1	0	0	0	0	0	0	1
Reader response	0	0	3	0	0	0	0	0	0	3
Simple view	1	0	0	0	0	1	3	5	1	11
Situated learning	2	0	1	0	1	0	0	0	0	4
Social constructivism	0	5	3	0	3	1	1	1	1	15
Sociocultural	10	10	12	0	8	1	0	3	1	45
Transactional theory	1	0	1	0	2	0	0	0	0	4
Others	12	7	14	5	7	9	4	22	4	84
Total	93	100	113	136	72	160	131	373	60	1,238

Research Designs

The most frequently reported research designs across all journals were (a) General Quantitative, (b) Experimental/Quasi-Experimental, (c) Non-Empirical, (d) Mixed Methods, and (e) Case Study. Another chi-square test of independence was run to determine whether there was a significant relationship between research design and journals. This relationship was statistically significant, χ²(144, N = 1,238) = 872.39, p < .001. The results from this chi-square indicated a moderate effect size, V = .32 (Cohen, 1988). These results are illustrated in Table 4.

Table 4.

Frequency of Research Designs by Journal.

Research designs	Journals									Total
Research designs	JLR	RRQ	LRI	SSR	RTE	JRR	RP	R&W	RWQ	Total
Case study	17	10	14	0	21	4	3	2	5	76
Content analysis	2	1	8	0	1	0	1	0	1	14
Correlational	0	8	0	1	0	3	2	9	0	23
Discourse analysis	2	4	0	0	4	2	0	1	0	13
Ethnography	3	6	2	0	2	2	0	0	2	17
Experimental/Quasi	2	24	14	29	4	26	13	89	1	202
Formative	2	1	1	0	2	0	0	1	0	7
Grounded theory	1	0	1	0	0	0	0	0	0	2
Literature review	1	0	3	1	0	1	6	1	1	14
Meta-analysis	1	14	1	4	0	3	0	3	0	26
Mixed methods	7	7	24	1	1	9	13	13	3	78
Narrative	2	0	0	0	0	0	0	0	0	2
Non-empirical	15	2	8	7	11	4	9	8	24	88
Phenomenological	0	0	1	0	1	0	0	0	0	2
Qualitative	7	1	18	0	23	8	10	2	5	74
Quantitative	16	11	12	81	1	89	68	227	15	520
Statistical modeling	8	8	4	12	1	9	6	17	3	68
Survey	0	2	1	0	0	0	0	0	0	3
Other	7	1	1	0	0	0	0	0	0	9
Total	93	100	113	136	72	160	131	373	60	1,238

Data Sources

Last, frequency counts indicated that the most commonly observed data sources across all the journals were (a) student assessments, followed by (b) observations/video transcripts/field notes, and (c) surveys/questionnaires. A fourth chi-square test of independence was run to determine whether there was a relationship between data sources and the journal. This was also statistically significant, χ²(56, N = 1,678) = 630.29, p < .001. This test also indicated a moderate effect size, V = .23 (Cohen, 1988). Table 5 demonstrates the frequencies of data sources represented in the different journals.

Table 5.

Frequency of Data Sources by Journal.

Data sources	Journals									Total
Data sources	JLR	RRQ	LRI	SSR	RTE	JRR	RP	R&W	RWQ	Total
Artifacts	32	11	40	1	31	11	11	27	3	167
Interviews	39	22	36	1	25	14	20	10	15	182
Observations	46	34	35	5	29	12	24	24	11	220
Student assessments	29	55	37	118	6	121	85	326	26	803
Studies/articles	2	15	6	5	0	3	4	7	3	45
Surveys	21	12	42	15	5	24	38	51	6	214
Teacher assessments	0	0	3	0	0	3	4	6	1	17
Other	2	1	1	3	2	4	6	9	2	30
Total	171	150	200	148	98	192	192	460	67	1,678

Thus, based on topics, theoretical perspectives, research designs, and data sources in nine literacy journals are all significantly different with moderate (Vs = .23 to .32) strengths of association.

Discussion

This study provides a content analysis of nine peer-reviewed journals, six of them representing literacy organizations: ALER (LRI), International Literacy Association (RRQ), Literacy Research Association (JLR), National Council for the Teaching of English (RTE), Society for the Scientific Study of Reading (SSR), and United Kingdom Literacy Association (JRR). The research team systematically analyzed 1,238 articles published from 2009 to 2014. The results provide a snapshot of current literacy research that demonstrates the paradigm or set of paradigms (Kuhn, 1962) that make up the present literacy research thought collective or thought collectives (Fleck, 1979).

Overall, the topics appearing most frequently in these journals include Comprehension (20% of articles), Bilingual/ELLs (19%), Phonemic/Phonological Awareness (14%), Instruction (14%), Writing (13%), Struggling Readers (13%), Vocabulary (12%), and Spelling (11%). Articles could receive up to three topic codes, so these percentages are not mutually exclusive. In addition, the chi-square tests of independence demonstrated significant differences in the topics these journals are publishing. In short, these journals are publishing articles on different topics at varying frequencies.

The frequency of topics published in different journals evidences the finding described above. Although Writing, for instance, was a common topic overall (164 articles were coded in this way), 63% of the articles on Writing were from two journals, R&W and RTE. This finding makes sense. Writing is in the title of the journal Reading and Writing, a journal clearly aimed at both reading and writing research, and RTE is the flagship journal of National Council for the Teaching of English, an organization that has a rich history of emphasizing writing. Likewise, Struggling Readers was also a common topic (162 articles were coded this way), yet it was only in the top three codes of one journal: RWQ. Again, this finding is not surprising: RWQ is a journal devoted to students with learning difficulties. Not all topics coalesced around particular journals, though. For example, Comprehension, Bilingual/ELLs, Phonemic/Phonological Awareness, and Instruction were common topics and had wide representation across these nine journals. Each of these topics was in the top three topics coded in three or more journals.

Because the current content analysis took a different approach than previous content analyses (we examined multiple journals for 6 years; previous analyses typically examined only one or just a few journals for many years), it is difficult to draw comparisons and impossible, with our data, to trace trends over time. However, previous content analyses found, like we did, that Comprehension and Instruction were common topics appearing in literacy journals (Baldwin et al., 1992; Dunston et al., 1998; Guzzetti et al., 1999; Morrison et al., 2011). Topics commonly found in other analyses that were less common in ours include Assessment (Morrison et al., 2011; Reutzel & Mohr, 2015), Teachers/Teacher Education (Dunston et al., 1998), and the Reader (Guzzetti et al., 1999).

Using this content analysis as a snapshot of the current “paradigm,” it appears that there have been three shifts (i.e., topics of study that are currently receiving much attention that previously were not). First, there appeared to be increased attention paid to special populations: Bilingual/ELLs was the second most common topic of study, and Struggling Readers was the sixth most common topic studied across the 1,238 articles we analyzed. Previous content analyses actually specifically called for increased attention to these populations (Guzzetti et al., 1999). In the late 1990s, Guzzetti and colleagues began to see and anticipated an increase in attention to technology, and that is part of the second shift that was inferred from this study. Although not in the top 10 most frequently studied topics in the current content analysis, New Literacies was a focus of study in 94 articles (8% of the articles reviewed). The final shift we saw when reviewing the topics found in this research in light of previous content analyses is increased attention given to writing: Our analysis revealed that Writing was the fifth most commonly studied topic. Two previous studies noted increased study of writing (Dunston et al., 1998; Morrison et al., 2011), but historically, writing has not appeared in the top five most frequently studied topics.

This content analysis found that 937 of the 1,238 (76%) articles did not explicitly name a theoretical perspective. This high percentage is noteworthy as literacy researchers explain that theoretical perspectives strengthen research because they provide a broader perspective, give an external source for anchoring data analysis, and help build theories of human and social behavior (Dressman & McCarthey, 2011; Unrau & Alvermann, 2013). However, theoretical perspectives are most commonly associated with qualitative research (Dressman & McCarthey, 2011; Unrau & Alvermann, 2013), which was less common than quantitative research in the journals we analyzed. Also, we recognize that theoretical perspectives are often not explicitly stated. That is, theoretical perspectives are frequently implied by the problem addressed, the research questions asked, the literature cited, the data collected, the analysis used, and the conclusions drawn. Nonetheless, our research team applied an explicit operationalization to document this often-implicit aspect of research. It is important, then, to recognize that the theoretical perspective code “Not Specified” in this study does not mean nonexistent. Rather, it means that the operational definition of theoretical perspective we used excluded implied theoretical perspectives.

The implied nature of theoretical perspectives aligns with the history of science perspective used in this study. Kuhn (1962) and Fleck (1979) demonstrated that thought collectives are composed of paradigms; that is, they share common understandings and assumptions. From this perspective, it might be unnecessary for scholars to explicate the theoretical perspective they are working from because it is shared among the thought collective. Fleck also explained that fields of study could have multiple thought collectives. Different thought collectives might align with particular questions. For example, a researcher studying digital literacies may not need to explicitly state she is coming from a New Literacies perspective. A researcher studying the opportunities afforded minority female students compared with majority male students might not need to explicitly state that he is using a critical perspective.

Across the nine journals in this content analysis, the most frequently used research designs included Quantitative (42% of articles), Experimental/Quasi-Experimental (16%), Non-Empirical (7%), Mixed Methods (6%), Qualitative (6%), and Statistical Modeling (6%). In addition, the chi-square tests of independence demonstrated significant differences in the research designs used in the research these nine journals are publishing. As literacy researchers might expect, particular journals tended toward particular research designs (Duke & Mallette, 2001). SSR and R&W published quantitative research almost exclusively. RP and JRR published some qualitative and mixed-methods research, but their contents were primarily quantitative. Conversely, RTE published primarily qualitative research. JLR, RRQ, and LRI demonstrated a balance of quantitative, qualitative, and mixed-methods research. RWQ also demonstrated a balance; however, 40% of its articles were non-empirical—a much larger percentage than any other journal in this study.

According to this analysis, literacy researchers most frequently use student assessments as a data source (n = 803), supporting the high incidence of quantitative studies. Researchers also frequently used observations (n = 220), surveys (n = 214), interviews (n = 182), and artifacts (n = 167). These comprise the primary research data sources used within these journals. Although the chi-square tests of independence indicated statistically significant differences in the data sources in all the tests, this finding is reflected in the previous discussion of research designs, which largely determine the data sources collected.

Although there appears to be an “ecological balance” (Duke & Mallette, 2001) of methods used in research published in some journals, the field still might be “fractious” (Stanovich, 1998) in that some journals only publish research that comes from a particular paradigm. Viewed more positively, these patterns in different journals from different organizations publishing research from different epistemological perspectives using different methodologies could represent “thought collectives” (Fleck, 1979). That is, researchers affiliate with, collaborate with, and publish for colleagues who share the same epistemologies and areas of expertise. Different thought collectives could be positive for the field. Fleck explained that thinking is a social activity, so the exchange of ideas across thought collectives can spur progress as paradigms are questioned and anomalies arise. Such intellectual and research activity might compel thought collectives to merge as paradigm shifts take place and knowledge is built. Does the current study expose a fractious field or multiple thought collectives? Are readers of RTE also readers of SSR? Or are RRQ, JLR, and LRI the outlets that promote inter-thought collective dialogue? We urge members and board members of these organizations and editors of these journals to carefully consider these questions. Regardless of the answers, what forums could the field of literacy research utilize or create to facilitate more inter-thought collective discussion?

Previous content analyses found shifts from quantitative methods to qualitative methods (Baldwin et al., 1992; Dunston et al., 1998; Guzzetti et al., 1999; Morrison et al., 2011). Our study sought a more nuanced view of designs than just Quantitative, Qualitative, and Mixed Methods, which was typical in previous content analyses of literacy research. Therefore, when we classify our data as Quantitative, Qualitative, and Mixed, there are some gray areas because certain designs (e.g., Formative and Content Analysis) use all three types at various times. However, a rough breakdown of the empirical articles in the nine journals in our study is 75% quantitative, 18% qualitative, and 7% mixed methods.

Certain journals have shifted toward more qualitative research. For example, JLR has published more qualitative studies and fewer quantitative studies than were described in Guzzetti et al.’s (1999) content analysis of issues published in 1969-1998, where they noted an increase in qualitative studies over time, but quantitative studies were still the majority. However, across all nine journals, it is clear that quantitative research is much more common. We should note that the numbers presented here are somewhat skewed due to the high publication count of R&W (30% of the articles analyzed came from this journal), which was predominately quantitative. Nonetheless, quantitative research is more prevalent than qualitative or mixed methods in the field of literacy research represented by the high-profile journals analyzed herein.

Limitations

One limitation is the specific journals analyzed in this content analysis. We attempted to be objective and systematic in selecting the journals. However, we recognize that there are other journals that are of high quality and publish influential literacy research and ideas. Regardless, the journals included in this content analysis, although certainly not inclusive of all literacy journals, are undoubtedly well known and impactful. Another limitation is the rules used to identify research designs and theoretical perspectives—that the authors had to explicitly state the design or theoretical perspective. We recognize that research designs and theoretical perspectives are not always explicitly stated but rather more implicitly described. Nonetheless, we erred on the side of objectivity, setting up rules and procedures that could be systematically followed, in the traditions of content analysis (Holsti, 1969; Krippendorff, 2004).

Conclusion

This content analysis enabled us to see the current paradigms (Kuhn, 1962) and thought collectives (Fleck, 1979) in the field of literacy research. As literacy researchers, we are encouraged by the wide range of topics being studied through a multitude of research designs. We urge the literacy research community to continue to demand rigorous research, but to do so in a way that appreciates the power in viewing and studying teaching and learning from diverse perspectives, using diverse methods, and with recognition that a foundational aspect of rigorous research is the match between research questions asked and research methods used (Duke & Mallette, 2011; Kamil, Afflerbach et al., 2011). Engaging in scholarly dialog across thought collectives will help us identify anomalies in our current understanding, which will advance us as a field by compelling additional research and subsequent paradigm shifts to help us move toward advanced understandings of literacy processes, literacy teaching, and literacy learning (Fleck, 1979; Kuhn, 1962).

Therefore, we encourage literacy researchers to engage in scholarly work in new thought collectives. Attend conferences that present epistemologies different from your own—and participate in the conversation, asking pressing questions and thoughtfully considering their responses. As a field, we must engage in this sort of cross-thought collective collaboration, or we will not move toward “ecological balance” (Duke & Mallette, 2001). We hope this content analysis will contribute to the field by demonstrating what topics are currently being studied (and which topics are not). We also hope to encourage conversation and action regarding research designs where literacy researchers are posing the most important questions of the day and studying them using rigorous and appropriate research designs that improve our work in helping all people live literate lives.

Footnotes

Authors’ Note

The George Mason University Content Analysis Team includes Jan Ainger, Mary Carmen Bartolini, Alicia K. Bruyning, Ellen Clark, Sarah Crain, Nisreen Daoud, Karen Sutter Doheney, Stacey Duff, Susan V. Groundwater, Jacqueline Heller, Amber Jensen, Lesley A. King, Jennifer Lindenauer, Joanna Newton, Allison Ward Parsons, Erin M. Ramirez, Jayne Sherman, and Peet Smith.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Seth A. Parsons, PhD, is an associate professor in the College of Education and Human Development at George Mason University. He teaches courses in the Literacy, Research Methods, and Elementary Education program areas. His research focuses on literacy teachers’ instructional adaptations, literacy teacher education and development, and students’ motivation and engagement in literacy.

Melissa A. Gallagher, PhD, is an assistant professor in the College of Education at the University of Louisiana at Lafayette. She teaches Mathematics Methods courses. Her research focuses on teachers’ knowledge and practices in mathematics, students’ vocabulary development, and best practices to support English learners.

The George Mason University Content Analysis Team is a remarkable team of professors and doctoral students who are committed to learning more about literacy research, so they can better serve teachers and students.

References

Alvermann

D. E.

Unrau

N. J.

Ruddell

R. B.

(Eds.). (2013). Theoretical models and processes of reading (6th ed.). Newark, DE: International Reading Association.

Baldwin

S. R.

Readence

J. E.

Schumm

J. S.

Konopak

J. P.

Konopak

B. C.

Klingner

J. K.

(1992). Forty years of NRC publications: 1952–1991. Journal of Literacy Research, 24, 505-532. doi:10.1080/10862969209547793

Bauer

Theado

C. K.

(2015). Examining the “social turn” in postsecondary literacy research and instruction: A retrospective view of JCRL scholarship, 2005-2013. Journal of College Reading and Learning, 45, 67-84. doi:10.1080/10790195.2014.0950869

Beretvas

S. N.

(2004). Using hierarchical linear modeling for literacy research under No Child Left Behind. Reading Research Quarterly, 39, 95-99.

Cassidy

Grote-Garcia

(2013). Common core state standards top the 2014 what’s hot, and what’s not survey. Reading Today, 31(1), 12-16.

Cohen

(1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.

Creswell

J. W.

(2007). Qualitative inquiry and research design: Choosing among five approaches (2nd ed.). Thousand Oaks, CA: Sage.

Creswell

J. W.

(2013). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand Oaks, CA: Sage.

Cunningham

J. W.

(2001). Essay book review of The National Reading Panel Report [Review of the book Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction by The National Reading Panel.]. Reading Research Quarterly, 36, 326-335. doi:10.1598/RRQ.36.3.5

10.

Dressman

McCarthey

S. J.

(2011). Toward a pragmatics of epistemology, methodology, and social theory. In Duke

N. K.

Mallette

M. H.

(Eds.), Literacy research methodologies (2nd ed., pp. 441-463). New York, NY: Guilford Press.

11.

Duke

N. K.

Mallette

M. H.

(2001). Critical issues: Preparation for new literacy researchers in multi-epistemological, multi-methodological times. Journal of Literacy Research, 33, 345-362. doi:10.1080/10862960109548114

12.

Duke

N. K.

Mallette

M. H.

(Eds.). (2011). Literacy research methodologies (2nd ed.). New York, NY: Guilford Press.

13.

Dunston

P. J.

Headley

K. N.

Schenk

R. L.

Ridgeway

V. G.

Gambrell

L. B.

(1998). National reading conference research reflections: An analysis of 20 years of research. Yearbook of the National Reading Conference, 47, 441-450.

14.

Fleck

(1979). Genesis and development of a scientific fact. Chicago, IL: The University of Chicago Press.

15.

Guzzetti

Anders

P. L.

Neuman

(1999). Thirty years of JRB/JLR: A retrospective of reading/literacy research. Journal of Literacy Research, 31, 67-92.

16.

Hoffman

J. V.

Wilson

M. B.

Martínez

R. A.

Sailors

(2011). Content analysis: The past, present, and future. In Duke

N. K.

Mallette

M. H.

(Eds.), Literacy research methodologies (2nd ed., pp. 28-49). New York, NY: Guilford Press.

17.

Holsti

O. R.

(1969). Content analysis for the social sciences and humanities. Reading, MA: Addison-Wesley.

18.

Kamil

M. L.

(1995). Some alternatives to paradigm wars in literacy research. Journal of Reading Behavior, 27, 243-261. doi:10.1080/10862969509547880

19.

Kamil

M. L.

Afflerbach

P. P.

Pearson

P. D.

Moje

E. B.

(2011). Preface: Reading research in a changing era. In Kamil

M. L.

Pearson

P. D.

Moje

E. B.

Afflerbach

P. P.

(Eds.), Handbook of reading research (Vol. 4, pp. 13-27). New York, NY: Routledge.

20.

Kamil

M. L.

Pearson

P. D.

Moje

E. B.

Afflerbach

P. P.

(Eds.). (2011). Handbook of reading research (Vol. 4). New York, NY: Routledge.

21.

Knudson

R. E.

Onofrey

K. A.

Theurer

J. L.

Boyd-Batstone

(2002). A decade of literacy research in three education journals. Reading Improvement, 39, 119-123.

22.

Krippendorff

(2004). Content analysis: An introduction to its methodology (2nd ed.). Thousand Oaks, CA: Sage.

23.

Kuhn

T. S.

(1962). The structure of scientific revolutions. Chicago, IL: The University of Chicago Press.

24.

Lapp

Fisher

(Eds.). (2011). Handbook of research on teaching the English language arts (3rd ed.). New York, NY: Routledge.

25.

Leu

D. J.

(2000). Literacy and technology: Deictic consequences for literacy education in an information age. In Kamil

M. L.

Mosenthal

P. B.

Pearson

P. D.

Barr

(Eds.), Handbook of reading research (Vol. 3, pp. 743-770). New York, NY: Routledge.

26.

Lomax

R. G.

(2004). Whither the future of quantitative literacy research? Reading Research Quarterly, 39, 107-112.

27.

McKenna

M. C.

Robinson

R. D.

(1999). The impact of the Journal of Reading Behavior on reading scholarship. Journal of Literacy Research, 31, 93-104. doi:10.1080/10862969909548038

28.

Morrison

T. G.

Wilcox

Billen

M. T.

Carr

Wilcox

Morrison

Wilcox

R. T.

(2011). 50 years of literacy research and instruction: 1961–2011. Literacy Research and Instruction, 50, 313-326. doi:10.1080/19388071.2011.602924

29.

Morrow

L. M.

Gambrell

L. B.

(Eds.). (2014). Best practices in literacy instruction (5th ed.). New York, NY: Guilford Press.

30.

National Institute of Child Health and Human Development. (2000). Report of the National Reading Panel: Teaching children to read (NIH Pub. No. 00-4769). Washington, DC: U.S. Government Printing Office.

31.

No Child Left Behind Act of 2001 Pub L. no. 107-110. (2002). Retrieved from http://www.ed.gov/policy/elsec/leg/esea02/index.html

32.

Pearson

P. D.

(2004). The reading wars. Educational Policy, 18, 216-252. doi:10.1177/089904803260041

33.

Reutzel

D. R.

Mohr

K. A. J.

(2015). 50 years of Reading Research Quarterly (1965-2014): Looking back, moving forward. Reading Research Quarterly, 50, 13-35. doi:10.1002/rrq.87

34.

Schumm

J. S.

Lewis-Spector

Price

Doorn

(2014). A content analysis of College Reading Association/Association of Literacy Educators and Researchers teacher education publications: Past, present, and future. Literacy Research and Instruction, 53, 225-244. doi:10.1080/19388071.2013.847132

35.

Shanahan

(2003). Research-based reading instruction: Myths about the national reading panel report. The Reading Teacher, 56, 646-655.

36.

Stanovich

K. E.

(1990). A call for an end to the paradigm wars in reading research. Journal of Reading Behavior, 22, 221-231. doi:10.1080/10862969009547708

37.

Stanovich

K. E.

(1998). Twenty-five years of research on the reading process: The grand synthesis and what it means for our field. Yearbook of the National Reading Conference, 47, 44-58.

38.

Still

K. L.

Gordon

J. P.

(2011). Five decades of ALER conference session presentations, 1960-2010. Literacy Research and Instruction, 50, 327-347. doi:10.1080/19388071.2011.603598

39.

Tierney

R. J.

(2014). Mapping the challenges and changes in literacy research. Yearbook of the Literacy Research Association, 63, 32-47.

40.

Tracey

Morrow

L. M.

(2012). Lenses on reading: An introduction to theories and models (2nd ed.). New York, NY: Guilford Press.

41.

Unrau

N. J.

Alvermann

D. E.

(2013). Literacies and their investigation through theories and models. In Alvermann

D. E.

Unrau

N. J.

Ruddell

R. B.

(Eds.), Theoretical models and processes of reading (6th ed., pp. 47-90). Newark, DE: International Reading.

42.

Yatvin

(2002). Babes in the woods: The wanderings of the national reading panel. Phi Delta Kappan, 83, 364-369.

A Content Analysis of Nine Literacy Journals,2009-2014

Abstract

Keywords

Related Research

Perspective

Method

Data Sources

Data Analysis

Developing codes

Results

Topics

Theoretical Perspectives

Research Designs

Data Sources

Discussion

Limitations

Conclusion

Footnotes

Authors’ Note

Declaration of Conflicting Interests

Funding

Author Biographies

References