Abstract
The Congressional Research Service (CRS) is the nonpartisan research and analytical arm of the U.S. Congress and is physically and structurally located within the Library of Congress. Despite the role of CRS as a critical information provider and technical assistant to actors crafting and implementing federal policy, CRS reports, which serve as a key mechanism of information dissemination and influence by the organization, only became widely available to the public following the adoption of the Consolidated Appropriations Act of 2018. This study uses content analysis to examine CRS’s education-related reports from the 115th Congress to explore what sources are directly and indirectly informing federal policymaking based on references in CRS reports. Findings offer insights into research utilization in the federal education policy process and contribute to the growing literature on intermediary organizations and knowledge brokering.
Keywords
Introduction
The Congressional Research Service (CRS) serves as the nonpartisan research and analytical arm of the U.S. Congress, acting primarily as an information provider to members of Congress and congressional committees. Referred to as “Congress’s think tank” (Williamson, 2007) and housed in the Library of Congress, CRS “operates solely at the behest of and under the direction of Congress” (Library of Congress, n.d.) and, per its mission, provides “the highest quality of research, analysis, information, and confidential consultation” (Mazanec, 2019). This role of CRS has become increasingly distinctive as decades of disinvestment in the federal government’s capacity to conduct research and analysis has presented increased opportunity for lobbyists and partisan organizations to influence the federal policymaking process (Baumgartner & Jones, 2018; Drutman & Teles, 2015). Furthermore, recent work has noted that CRS maintains a precarious relationship with the U.S. Congress due to fear of “political retribution” (Kosar, 2020, p. 139), which has resulted in ongoing internal debate around whether the organization should maintain “objectivity” versus “neutrality” in its reports and materials (Aftergood, 2006; Kosar, 2020). Despite these challenges, CRS has been recognized in research examining U.S. federal policymaking as a key actor supporting members of Congress throughout the stages of the policy process (Dreyfus, 1976; Fagan & McGee, 2020; Gude, 1985; Kosar, 2020; Rothstein, 1990; Weiss, 1989; Wolanin, 1976).
Nevertheless, few empirical studies have directly examined the organization. This gap in the literature can be attributed to long-standing restrictions and burdensome barriers in accessing documents and contacting CRS staff for individuals unaffiliated with Congress (Mulhollan, 2007). In turn, private citizens and researchers were left at the behest of complicated and costly processes to gain access to information produced by CRS (Kosar, 2015, 2020; Williamson, 2007). However, this limitation was removed with the passing of the Consolidated Appropriations Act of 2018, which “directed the Library [of Congress] to make CRS reports publicly available online” (Hayden, 2018). 1
The public-facing CRS website debuted in September 2018 and immediately began publishing reports released during the 115th U.S. Congress (January 3, 2017–January 3, 2019), with the plan of releasing all future reports and retroactively releasing former reports online. This change in access to CRS reports creates a novel opportunity to gain insights into a critical and trusted source of information available to congressional members and their staff, while also limiting future potential modifications to the writing or production of documents resulting from their new public audience. Accordingly, we argue in this study that CRS serves as an intermediary organization that operates as an information source and trusted partner positioned between the research community and policymakers in the U.S. Congress. Moreover, considering the 115th Congress had an extensive list of education-related priorities (Shepherd, 2017; Ujifusa, 2017), CRS reports from this congressional session may reflect a large and wide-ranging body of reports that can offer a broad understanding of the information sources considered in the federal education policy process.
Examining CRS’s education-related reports made available to Congress offers an opportunity to understand a key source of information for legislators that may contribute to shaping federal policy conversations and decisions. Three research questions guide our study:
What content areas are addressed in education-related CRS reports released during the 115th U.S. Congress?
What sources of information do education-related CRS reports cite, and to what extent do these sources provide research evidence versus other types of information?
How are sources of information utilized within education-related CRS reports?
We consider these questions by examining 55 CRS reports designated by the organization as related to education through qualitative content and citation analysis.
Literature Review
Calls to improve research-informed policy and policy-relevant research often underscore the long-standing gap between policy actors—those who are directly involved in the production and implementation of policy—and academic researchers (Galey, 2015; Hillman et al., 2015; Honig & Coburn, 2008). Previous research notes that policy actors and researchers have distinct language, norms, and goals that prevent the two sides from communicating effectively, resulting in a “two-communities” divide (Birnbaum, 2000; Caplan, 1979; Natow, 2020b; Snow, 1963; Tseng, 2012; Tseng & Nutley, 2014; Wolanin, 1976). For example, researchers tend to emphasize theory and reliability with the aim of informing future research, whereas policymakers focus on experience, common sense, and tangible connections to specific and current issues (Birnbaum, 2000; Daly & Finnigan, 2014; Tseng & Nutley, 2014; Wolanin, 1976). These differences can inhibit the extent to which empirical research and information are considered in the policy process, with some suggesting the two communities “frequently talk past rather than to each other” (Wolanin, 1976, p. 94) and referring to the use of research in policy as “trees without fruit” (Keller, 1985) or “shipyards in the desert” (Weiner, 1986).
In order to better understand mechanisms to bridge this chasm, researchers have considered a process referred to as “knowledge brokering” to examine the information exchange processes that connect the two communities (Daly et al., 2014; Lomas, 2007; Malin & Brown, 2020; Meyer, 2010; Neal et al., 2015; Neal et al., 2019; Weber & Yanovitzky, 2021). Lomas (2007) suggests that successful knowledge brokering requires a “trusted and credible” actor that “understands the cultures of both the research and decision-making environments” and is “able to find and assess relevant research in a variety of formats” (p. 130), ultimately with the goal of facilitating the dissemination of research to guide policy decision-making. Moreover, Meyer (2010) points out that brokering knowledge “means far more than simply moving knowledge—it also means transforming knowledge” (p. 120). Therefore, knowledge brokers often provide context and meaning to the information they provide and do not simply serve as couriers of evidence.
One group of actors often discussed in the knowledge-brokering process are intermediary organizations. These organizations do not decide on the adoption or implementation of policy directly but, instead, they serve primarily in a supporting role, such as through the provision of information or expert advice (Barnes et al., 2014; Cooper & Shewchuk, 2015; Hammond et al., 2022; Lubienski et al., 2014; Malin et al., 2020; Neal et al., 2019; Ness, 2010; Nutley et al., 2007; Shewchuk & Farley-Ripple, 2022; Tseng & Nutley, 2014). Similar to research on knowledge brokering, authors underscore the role of trust by policymakers when considering whether an intermediary is viewed as a valued source of research and information (Barnes et al., 2014; Daly et al., 2014; Finnigan & Daly, 2014; Malin et al., 2020; Ness, 2010; Shewchuk & Farley-Ripple, 2022). For example, DeBray et al. (2014) investigate the use and misuse of research by intermediary organizations in New Orleans regarding charter school effectiveness. The authors find that “policymakers appear to be embracing evidence that is concise, slickly produced, timely, and accessible—and aligned with their policy positions or from sources they trust” (DeBray et al., 2014, p. 177). They suggest that specialty intermediary organizations exist to confirm beliefs and serve partisan efforts, such as school choice, voucher programs, and charter schools, without empirical evidence of their effectiveness. Ultimately, their analysis demonstrates the power of intermediary organizations, regardless of whether their research-based decisions are sound or not.
Research highlights the influence of intermediary organizations in state-level policymaking in higher education, which is the primary level of policy decision-making for the sector. For instance, studies examining the development and proliferation of higher education performance-based funding models underscore the role of intermediaries in the facilitation and transmission of information and support for this policy solution (Gándara et al., 2017; Hammond et al., 2022; Miller & Morphew, 2017). This research also emphasizes, however, that intermediaries often maintain certain agendas due to either political partisanship, support from foundations and philanthropies, or broader missions, which, in turn, influence the perspectives that intermediary organizations offer (Hammond et al., 2022). K–12 education research similarly discusses the influence of external entities and organizational characteristics on intermediary organization decision-making and information provision (Conaway et al., 2015; DeBray et al., 2014; Goldie et al., 2014; Lubienski et al., 2014), suggesting that these factors can be a concern when gauging the reliability of these entities. Goldie et al. (2014) also note, however, the existence of an echo chamber of information across intermediaries, where the same studies and citations are amplified by organizations in support of similar policy goals despite being unrepresentative of the existing literature as a whole.
A recent typology notes the potential diversity among intermediary organizations in education policy and suggests that these groups often do not remain entirely neutral between the policymaker and researcher communities. Instead, intermediary organizations can be classified into one of four categories—researcher-based intermediaries, researcher-leaning intermediaries, policymaker-leaning intermediaries, and policymaker-based intermediaries—based on their association with researchers and policymakers (Ness et al., 2018). For example, researchers or academics tend to lead researcher-based intermediaries, which are often housed within academic institutions or large research entities. On the other hand, policymaker-based intermediaries include professional associations of policymakers as well as partisan and nonpartisan organizations that offer information that may not be empirically rigorous and is often framed for specific policymaking audiences. Researcher-leaning and policymaker-leaning intermediaries fall between these extremes and will often analyze, disseminate, and translate more empirically robust information, with the former producing its own research while the latter relies on analyses by other entities.
For purposes of the current study, we argue that CRS can be classified as an intermediary organization given their primary role is to support Congress and expand the legislative branch’s capacity around information use and consumption (Kosar, 2020), despite serving little to no direct role in the crafting or implementation of policy. According to the Ness et al. (2018) typology, CRS aligns most closely with researcher-leaning intermediaries, given their ability to conduct in-house analysis and produce new research; however, CRS’s primary role of supporting Congress, coupled with the previous restrictions to accessing its reports by the general public, suggests a closer association with the policymaker community than is traditionally outlined among researcher-oriented intermediary organizations. Consequently, in considering CRS as an intermediary organization, the current study offers an opportunity to expand on this typology and deepen the understanding of intermediaries by considering an entity uniquely positioned between the two communities. Further, research examining the influence of intermediary organizations in education policymaking tends to focus at the local (DeBray et al., 2014; Honig et al., 2017; Penuel et al., 2017) and state levels (Conaway et al., 2015; Gándara et al., 2017; Massell et al., 2012; Miller & Morphew, 2017; Ness & Gándara, 2014), with fewer studies considering similar organizations at the federal level (Kosar, 2020; McDonnell & Weatherford, 2013). Research by McDonnell and Weatherford (2013), for instance, is representative of several studies that traced the roles of organizations around the development and adoption of the Common Core at the federal and state levels. Nevertheless, given the key role of federal education policy in guiding and setting parameters for localized decisions, as well as continued calls for increased partnerships and collaboration between the research and policymaking communities by the National Academies around critical education-related topics (Gamoran & Dibner, 2022), more insight into influential entities throughout this process is essential. Therefore, we contend that better understanding a widely referenced intermediary organization influencing national policymaking, such as CRS, can offer such insights into an often under-considered policy arena.
Conceptual Framework
Guiding this study is the concept of research utilization, which aims to understand the ways policymakers reference, consider, and apply research-based information in the policy process. Given the disconnect and differences that emerge due to the two-communities divide, understanding the consideration and application of research in the policy process can be complex (Birnbaum, 2000; Daly & Finnigan, 2014; Tseng & Nutley, 2014; Wolanin, 1976). Weiss (1979) developed the foundational work on this concept, which culminated in a seven-category typology detailing possible ways research is used. Subsequent studies established a more widely recognized and utilized abridged version that focuses on three broader categories of research use: instrumental use, conceptual use, and political use (Amara et al., 2004; Beyer, 1997; Deshpande & Zaltman, 1983; Dunn, 1983; Lindblom & Cohen, 1979; Natow, 2020a; Nutley et al., 2007; Tseng, 2012).
Instrumental use of research focuses on the direct application or discussion of research in the policy process (Dunn, 1983; Tseng, 2012; Weiss, 1979). Put differently, instrumental use of research occurs when policymakers’ decisions are based exclusively on what has been demonstrated or shown through research (Weiss, 1979). Given the myriad of information sources and influential stakeholders influencing public policies, explicit instrumental use is more common in the private sector than in the public or government sectors (Deshpande & Zaltman, 1983; Lindblom & Cohen, 1979; Nutley et al., 2007).
Conceptual use of research, which is also referred to as the “enlightenment model,” focuses on longer-term consideration of research, suggesting that the information supports a policymaker’s broader understanding on a given topic rather than having a direct influence on a specific instance of decision-making (Dunn, 1983; Tseng, 2012;Weiss, 1979). Dunn (1983) further delineates this type of research use by suggesting that instrumental use results in observable changes in behavior of the policy actor, whereas conceptual use changes the ways in which actors think about a policy or problem.
The third type of research utilization is political use, which focuses on the tactical use of information to support predetermined positions on policies or issues (Amara et al., 2004; Beyer, 1997; Natow, 2020a; Tseng, 2012; Weiss, 1979). Notably, Weiss (1979) suggests that political use is a valid function for research, as long as findings are not misrepresented.
A recent study by Natow (2020a) examining the federal higher education rulemaking process notes an important limitation of the extant literature on research use—namely that these categories are not necessarily wholly independent and that overlapping or blended forms of research use are possible. Citing research by Sunesson and Nilsson (1988) and Klemperer et al. (2001), Natow (2020a) explains that policymakers often utilize different forms of research use concurrently. Moreover, given the political nature of policymaking, Natow (2020a) underscores that all research use is inherently political as well as instrumental, conceptual, or political, based on the Weiss (1979) typology. She argues for an expansion of the research-use typology by explaining that, within the policymaking process, “politics and the political use of research are so dominant that they fuse with other forms of research use to produce new categories of use that do not fit neatly into the traditional typology” (Natow, 2020a, p. 23). The analysis for this study will extend these suggestions and gauge whether an expanded typology would be effective in this case.
Data and Method
CRS released 1,842 reports during the 115th U.S. Congress that were publicly available through their website at the time of data collection in March 2019. Fifty-five of these reports (2.98%) were indexed by CRS as related to “education” and were included in this study. 2 These reports include subjects that cover preK–12 through higher education as well as other topics deemed education-related by the organization, such as education support for veterans and food assistance programs. As these reports can be requested by any member of Congress or congressional committee (Relyea, 2010) and are written by one of CRS’s five divisions (American Law; Domestic Social Policy; Foreign Affairs, Defense and Trade; Government and Finance; Resources, Science and Industry), any tangentially related report that has education policy–related consequences may be indexed as “education.” A subcategory of documents analyzed includes five “In Focus” reports, which are unique in that they are brief, three-page documents that address issues such as programs supporting minority-serving institutions and Head Start. Although the research team analyzed the “In Focus” reports in the same manner as all the others, the unique nature and framing of these documents were notable. The 55 reports reviewed included 1,568 total pages, with a range from 3–128 pages and an average of 28.51 pages.
We empirically analyzed these reports using qualitative content and citation analyses (Merriam, 2009; Shih et al., 2008). Considering the limited research available on CRS publications, we utilized inductive approaches to identify an initial list of common themes across the 55 reports. Through this process, we developed a three-level analysis for the content of CRS reports that included the report’s purpose (Report Type), the report’s setting (Report Frame), and the report’s general issue area (Report Topic) (Saldaña, 2016). Table 1 provides an overview of these three components of our analysis, including definitions and examples from our sample. We discuss these levels of analysis further in our findings section.
Three-Level Analysis of CRS Report Content
While considering the content of the reports, we also analyzed the references in each report to gain a better understanding of what sources CRS used to inform their reporting. Using a similar inductive approach (Saldaña, 2016), the research team compiled an initial list of types of references, including academic articles and chapters, congressional reports, court cases, CRS reports, current and proposed legislation, and popular media. Ultimately, we identified 13 categories of references from this review, presented in Table 2.
Reference Typology and Definitions
For references coded as “other,” the research team discussed if there was a more appropriate classification for these citations, leaving only unique and obscure references with this code (e.g., a website for a school district, an official press release from a state-level agency, or reports released from federal student loan servicers outlining their duties). In addition to classifying citations, we also recorded the way in which reports explicitly framed and applied the references for additional analysis (e.g., an excerpt within a report, a connection to a law or a court case, or a suggestion for a reader seeking additional information) and consideration within the research utilization typology.
The research team took several steps in order to maximize validity and reliability for this study. Multiple members of the research team reviewed the same report during early stages of the analysis to improve intercoder reliability (Merriam, 2009). We also held regular team meetings to facilitate investigator triangulation and discuss any questions or obscure references, in order to ensure internal validity of the study. These meetings also offered opportunities to develop and later refine the analytic framework to code the remaining documents (Saldaña, 2016). As we finalized our framework and coding structure, research team members returned to earlier coded documents and reevaluated them based on changes and updates to our analysis schema.
Findings
Our findings indicate that CRS education-related reports fall into two classifications: those aimed at providing general information and those focusing on specific legislation. Within both classifications of reports, we find that a variety of sources are mentioned, though there was a reliance on references developed and/or released by federal government agencies and officials. Additionally, CRS’s mission and position underlie the ways in which references are utilized based on the abridged Weiss (1979) typology. Table 3 provides a comprehensive list of the reports included in this study, including their emergent classifications based on our analysis and the number of citations made per report, which are all discussed further in this section.
Reports Included in Analysis
Note: Reports are organized based on publication date. Report Type includes General Information (GI) or Legislation Focused (LF); Report Frame includes PreK–12 Education (K12) or Postsecondary Education/Higher Education (HED); Report Topic includes one of twelve emergent categories (Basic Needs; Education Assessment; Elementary and Secondary Education Act (ESEA); Federal Appropriations and Budgeting for Education; Financial Aid/Student Debt; Higher Education Act (HEA); Higher Education Institutions; Military/Veterans; Student Support; Students with Disabilities; Tax Benefits; Other).
Content of Reports: What Do CRS Reports Examine?
The first area of analysis focused on the content of the reports. From this examination, two distinct groups of reports emerged based on their overall goal, or what we are classifying as Report Type: general information reports and legislation-focused reports. General information reports took a descriptive approach to a broader topic, including financial aid and student loans or K–12 educational assessment, whereas legislation-focused reports focused on specific proposed or enacted legislation, such as the House of Representatives’ PROSPER Act or the Bipartisan Budget Act of 2018. There was a focus on the role of the federal government across all reports, which aligns with the purpose and mission of CRS, but the format and specificity of the reports varied, based on the report’s goal.
Of the 55 total education-related CRS reports, 33 reports (60%) fell into the general information category. These reports cover a broad array of subject areas, including opportunities for American Indians in elementary and secondary education, concepts and considerations around educational assessment, and basic needs assistance programs. These general information reports offer summaries of applicable legislation and provide nuanced history, context, and perspectives on the subject matter. For instance, the CRS report titled Indian Elementary-Secondary Education: Programs, Background, and Issues includes a historical overview of the federal government’s role in educating the country’s Native population, a description of programs, an indication of which students these programs serve, and a discussion of current issues and opportunities around the topic. In the following excerpt, the report details an area of educational opportunity improvement for the Native population and mentions what the federal government is currently doing toward that end:
There is continued interest in increasing the role of Indian tribes in an effort to increase student achievement and cultural relevance of education. Increasing the role of tribes in public schools may confront sovereignty, accountability, collective bargaining, and property ownership issues and will impact non-Indian students in public schools. . . . Programs funded under ESEA Title VI might be viewed as incremental efforts to increase cultural relevance and tribal influence. (Dortch, 2017b, p. 41)
The primary goal of this report is to provide an overview of the most important topics related to American Indian education at the elementary and secondary levels, while also offering insights into the benefits of funding such areas.
In comparison, the remaining 22 reports (40%) were categorized as legislation focused. These reports were more methodical in their description of components of the legislation, providing an overview of each subsection and discussing how the legislation would change or interact with current policy. For example, the CRS report on the PROSPER Act includes the following excerpt:
The changes that would be made by H.R. 4508 reflect several key themes: (1) simplifying the federal approach to providing student aid; (2) modifying federal student aid rules; (3) eliminating or winding down programs; (4) revising the educational quality and financial accountability requirements applicable to IHEs; (5) revising public accountability, transparency, and consumer information requirements; and (6) establishing specified limitations to the Secretary of Education’s authority. (Hegji et al., 2018, pp. 1–2)
The underlying goal of this report centered on informing the reader about technical components of the PROSPER Act, as well as offering some historical background and clarifying the broad goals of the legislation, rather than discussing potential implications of the passage of the legislation or indicating whether CRS suggests it should be supported.
An additional finding regarding Report Type relates to CRS’s role as a knowledge broker. In particular, while the general information reports aligned with the expectations of the brokering process and the provision of context and meaning to the information provided (Meyer, 2010), similar framing was less common among legislation-focused reports. This difference may be partially attributed to CRS’s responsibility to be responsive in a timely manner to congressional requests in the production of informational reports (Fagan & McGee, 2020; Kosar, 2020; Mazanec, 2019), which are likely even more time sensitive among reports related to legislation. We include Report Type as an additional point of analysis in the remaining findings as a mechanism to further understand this distinction.
Besides Report Type, we also examined CRS reports based on Report Frame, or whether the report was positioned within the preK–12 education setting or postsecondary/higher education setting, and Report Topic, or the general issue area of the report. Through our inductive analysis, 12 issue areas emerged as Report Topic options: Basic Needs, Education Assessment, Elementary and Secondary Education Act (ESEA), Federal Appropriations and Budgeting for Education, Financial Aid/Student Debt, Higher Education Act (HEA), Higher Education Institutions, Military/Veterans, Student Support, Students with Disabilities, Tax Benefits, and Other. In considering Report Frame, 34 reports (61.81%) were related to postsecondary/higher education overall, though this does include several reports that were also associated with preK–12 education (e.g., the Pregnancy Assistance Fund program that includes both high schools and institutions of higher education as potential recipients of the associated funds). Similarly, higher education–related issues emerged as the most common Report Type, including nine reports examining Financial Aid/Student Debt. These commonalities could be attributed to both chambers of Congress noting that reauthorizing the HEA was a legislative priority during the 115th Congress (Shepherd, 2017; Ujifusa, 2017). In contrast, however, the second most frequent Report Topic, which was covered across eight of the reports (14.54%), focused on aspects of the Elementary and Secondary Education Act (ESEA). Notably, ESEA was most recently reauthorized through the Every Student Succeeds Act (ESSA) of 2015 and was not due for its next reauthorization until after the 2020–21 school year. However, during the 115th Congress, there were congressional hearings and processes in place for the Department of Education to approve state ESSA plans. Therefore, although timeliness of information may be central to CRS reports, there are also certain topics that remain important for federal education policy following policy implementation and, accordingly, may influence the documents written by CRS.
Report References: What Do CRS Reports Cite?
Our second area of analysis examined the references used by CRS in their reports to understand the types of sources informing the organization. A total of 3,740 references across the 55 reports were analyzed, which included in-text citations, footnotes, and endnotes, as well as source information for data in tables and figures. The distribution in use of references was wide across the sample. For example, the 44-page report titled Bankruptcy and Student Loans included the most references with 320, and the 128-page report titled Federal Student Loan Forgiveness and Loan Repayment Programs utilized the second most references with 223. At the other extreme, two of the three-page “In Focus” reports (covering Head Start: Overview and Current Issues and The Pregnancy Assistance Fund) used only three references each. Removing the shorter “In Focus” reports from the sample, the 10-page report titled FY2016 State Grants Under Title I-A of ESEA included only seven references. On average, there were 68 references per report and approximately 2.4 references per page.
In addition to the overall number of references included in each report, there were notable trends regarding the frequency of reference type utilized as displayed in Table 4.
Frequency of Citations by Reference Type
The most frequently cited source of information was current legislation (33.66%), followed by federal agency reports (26.52%) and other CRS reports (15.48%). The CRS report, Educational Assessment and ESEA, includes many references to current legislation including a reference to P.L. 114-95, which is the Every Student Succeeds Act (ESSA), when discussing the assessments that states are required to administer as part of their accountability systems in order to be eligible for Title I funding. The CRS report, Federal Pell Grant Program of the Higher Education Act: Primer, references a federal agency report by citing the U.S. Department of Education’s AY 2015-2016 Pell Grant End-of-Year Report when discussing the federal Pell grant recipients from academic year 2011–2012 to academic year 2015–2016. Current legislation, federal agency reports, and CRS reports were the most common reference types, accounting for more than 75% of the references cited. These sources of information could all be more broadly classified as related to existing policies and government agencies, suggesting greater reliance on more internally proximate information rather than external expert research.
Accounting for a substantially smaller share across our sample, references coded as academic journal articles or book chapters were cited only 141 times (3.77%). For example, the CRS report, History of the ESEA Title I-A Formulas, cites several academic publications, including Orfield’s (2015) chapter titled, “Lyndon Johnson and American Education,” as well as Gamson, McDermott, and Reed’s (2015) article titled, “The Elementary and Secondary Education Act at Fifty: Aspirations, Effects, and Limitations.” When reviewing specific details of these academic references, we found a wide range of types of sources, including peer-reviewed articles and working papers as well as book chapters and full books. Notably, however, there were more references from books, book chapters, and more readily accessible research articles (e.g., working papers or reports released in partnership with research organizations) than journal articles that are traditionally behind paywalls. Nevertheless, our sample encompassed a wide range of academic journals representing a variety of disciplines, including education (e.g., Educational Researcher, American Educational Research Journal, Review of Educational Research, Educational Evaluation and Policy Analysis, The Journal of Higher Education), law (e.g., Santa Clara Law Review, Vanderbilt Law Review, Suffolk University Law Review), business and economics (e.g., American Economic Review, Journal of Economic Literature, The Review of Financial Studies), health (e.g., Journal of Women’s Health, Journal of Adolescent Health), and broader social science (e.g., American Behavioral Scientist, The Russell Sage Foundation Journal of the Social Sciences). Another interesting trend was that the academic references were found in only 19 reports and included several repeat citations. For instance, the CRS report, Indian Elementary-Secondary Education: Programs, Background, and Issues, included seven references to Prucha’s (1986) book The Great Father: The United States Government and the American Indians, often in conjunction with other academic references.
In addition to academic research, reports from research organizations (1.39%) and member organizations (3.07%) were referenced at significantly lower frequencies. Numerous entities were included in these codes, though some of the most widely referenced groups were East Coast and Washington DC–area organizations, such as Mathematica, New America, and the Urban Institute, as well as professional organizations (e.g., California Medical Association, National Association of Secondary School Principals), institutional-level groups (e.g., The Council for Higher Education Accreditation), and advocacy groups (e.g., Young Invincibles). Given the broad categorization of research and member organizations in this study, it is notable that the authors of these reports included politically partisan and nonpartisan groups as well as entities of various sizes, though we did not record organizational characteristics during our analysis to provide further insight into how these may impact the frequency of references. Nevertheless, it is notable that these reference sources along with academic research, which tend to be independent and offer the greatest research capacity among those included in our analysis, are among the less cited across reports. This finding may suggest that CRS does not necessarily rely on empirically founded research in their reports as much as more general information sources, which may also influence how these references are utilized.
Of note, the general trends in utilization of reference types was also similar across reports regardless of whether they were classified as general information or legislation focused, though some variation did exist. Specifically, general information reports referenced agency reports and current legislation substantially more than any other reference type, with CRS reports and court cases ranking closely as third and fourth. On the other hand, legislation-focused reports overwhelmingly considered current legislation, with CRS reports and agency reports ranking a distant second and third. Of note, the fourth most widely cited reference type overall, court cases, was heavily influenced by its use in general information reports and, more specifically, the report, Bankruptcy and Student Loans, which accounted for 227 of the 235 references found. This example was an extreme outlier across our sample but was notable. For legislation-focused reports, congressional reports were the fourth most widely type of reference cited, though there was not a similar single report that accounted for the majority of the count. One possible inference that could be made from these differences, however, is that the timeliness of legislation-focused reports may necessitate CRS to rely on more easily accessible reference types that are produced by government entities, which could explain why congressional reports remain more heavily cited in legislation-focused reports than in general information reports.
Research Use: How Do CRS Reports Use References?
Our final area of analysis sought to understand how references are utilized in context within CRS reports, specifically as aligned to the Weiss (1979) research-use typology (instrumental use, conceptual use, and political use).Although some research suggests the use of research to persuade readers or decision makers by adding validity to claims (e.g., Penuel et al., 2017), given the limited research on CRS and their reports as well as the focus on content analysis in the current study, our coding scheme and analysis focused on the explicit purpose of references rather than attempting to make assumptions regarding potential strategic motives. To this end, although all three uses of research were evident, there was a noticeably unequal distribution. Table 5 presents the distribution of the Weiss (1979) typology across the 3,740 references cited in the 55 CRS reports analyzed, as well as the breakdown between general information and legislation-focused reports.
Distribution of Weissian Research-Use Typology Across Citations
The distribution across citations was consistent based on Report Type as well as across the overall data set. Therefore, the following will present representative examples across the three types of research use in the context of CRS reports overall rather than focusing explicitly on differences between general information and legislation-focused reports.
The majority of references (n = 3,311; 88.53% of citations) were classified in the category of instrumental use. These references were used to support statements in the report through the provision of specific explanations, an approach that is akin to the use of references in academic writing. For example, in the CRS report, The Closure of Institutions of Higher Education: Student Opinions, Borrower Relief, and Implications, HEA is cited in various components of the report, such as defining a teach-out plan for current students when an institution closes or noting existing requirements and limitations to borrowers seeking to discharge their student financial aid loans when an institution closes (Hegji, 2018). Notably, all 141 citations for academic journal articles and book chapters were utilized in this manner, providing support for various statements and perspectives included in the reports. A second common instance of instrumental use came from citations for sourcing data represented in tables and figures. For example, in the CRS report, The Individuals with Disabilities Education Act (IDEA) Funding: A Primer, there were citations for related CRS reports, a U.S. Department of Education budget report, and separate analyses conducted by CRS to support their tables and figures (Dragoo, 2018). As instrumental use was the most common context for references overall, these examples were indicative of a significant portion of citations examined.
References were classified as conceptual use at a far lower rate (n = 393; 10.51% of citations). The most common example of this application of references was when reports directed the reader to other sources for additional information. Among legislation-focused reports, these citations directed the reader to alternative outlets to provide background information and additional insights to supplement the limited overview provided. For example, in Bipartisan Budget Act of 2018 (P.L. 115-123): CHIP, Public Health, Home Visiting, and Medicaid Provisions in Division E, authors direct readers to other sources to access an overview or more direct discussion of the underlying policies impacted by this legislation, such as provisions for foster care and child welfare or CHIP requirements for maintenance of effort (Mitchell et al., 2018). Interestingly, the resources highlighted in the majority of these instances are other CRS reports that focus on these specific topics, with reports from other federal agencies as the occasional alternative, rather than empirical analyses or other resources.
Evidence of political use was rare (n = 36; 0.96% of citations), occurring in only 13 reports analyzed. Generally, these references most often cited direct quotes from popular media, political speeches, or the Congressional Record or were used as references to highlight a contrarian perspective or unique position that did not necessarily further the overarching argument or narrative. For example, the CRS report titled Bankruptcy and Student Loans references a quote from Representative John Erlenborn (R-IL13) in the Congressional Record when discussing why student loans are not dischargeable when declaring bankruptcy under Section 523(a)(8) of the modern Bankruptcy Code. Specifically, the report states, “At least one of the Members who supported Section 523(a)(8) therefore believed that declaring bankruptcy immediately after graduation—and thereby ‘making the taxpayers pick up the tab’ for the debtor’s student loans—would be ‘tantamount to fraud’” (Lewis, 2018, p. 5). Another example draws from the report, GI Bill Legislation Enacted in the 114th Congress, where Representatives Jeff Miller (R-FL1) and Mark Takano (D-CA41) are cited regarding a provision to the GI Bill enacted following the U.S. Department of Education’s decision to withdraw its recognition of the Accrediting Council for Independent Colleges and Schools (ACICS) in 2016. Specifically, Representatives Miller and Takano supported a requirement that the Department of Veterans Affairs is to “notify GI Bill participants of the approval status of the program of education . . . intended to ensure that GI Bill participants attending ACICS-accredited institutions would not immediately lose GI Bill benefits” (Dortch, 2017a, p. 4) following the change in recognition of ACICS as an approved accreditor. Notably, the references coded as political use in these reports may not serve as exemplary representatives of the classification as defined by the typology (Weiss, 1979), specifically due to limited evidence of their tactical nature, but include the instances that most closely aligned when analyzing the documents.
Discussion
The current study affirms CRS’s alignment as an information-providing organization based on its public-facing reports. In particular, the reports reviewed serve primarily as sources on specific legislation and general policy topics and, although all were indexed as related to education, they cover a range of topics that are directly and indirectly related to preK–12 and postsecondary/higher education. To this end, one important interpretation of the content of CRS reports is that the organization serves as a generalist regarding provision of information, which may also be suggestive of the expertise of staff members and analysts in charge of education-related areas. This interpretation aligns with studies that have noted CRS’s broad role in supporting and expanding the capacity of Congress to interpret and consume information (Kosar, 2020), while maintaining limited specific expertise areas like other federal legislative branch agencies such as the Congressional Budget Office or General Accounting Office.
Given the generalist role of CRS, considering the sources of information it draws upon for its reports can offer insights into who or what is presented to federal policymakers. Overall, CRS primarily cites language from legislation as well as publications, reports, and analyses conducted by government and government-related entities, maintaining less engagement with independent and nongovernmental researchers and organizations. The reality that CRS does not focus as much on rigorous and independent studies suggests that the organization includes less empirical research in its reports, instead focusing on more general sources. This aligns with previous perspectives emphasizing the “two-communities” divide between policy actors and researchers (Birnbaum, 2000; Caplan, 1979; Snow, 1963; Tseng, 2012; Wolanin, 1976), suggesting more limited consideration of academic work by policymakers and related entities. This could, however, also be the result of the difficulties in accessing peer-reviewed research outside of academia as well as the speed at which CRS must produce its reports. Notably, the majority of these references occurred in reports categorized as general information, which may also be a function of the research community’s focus on informing future decision-making rather than specific current issues or legislation (Birnbaum, 2000; Daly & Finnigan, 2014; Tseng & Nutley, 2014; Wolanin, 1976). This context potentially weakens CRS’s initially proposed position as a researcher-leaning intermediary organization per the Ness et al. (2018) typology and aligns it more closely with a policymaker-leaning intermediary. Specifically, although CRS references in-house analyses and its own documents in its reports, the limited engagement with empirical research and reliance on more broadly available information sources, raises questions regarding whether CRS truly functions as a research intermediary or rather as a consolidator and presenter of information. As hypothesized, CRS’s nuanced positioning may bring further complexity to this established typology as this organization functionally serves as an in-house intermediary organization in the U.S. Congress while also remaining external to the actual crafting and implementation of policy.
Our final area of inquiry focused on CRS’s utilization of references in their reports. Drawing on Weiss’s (1979) research-use typology, the underlying context of these publications as well as the use of reference material could be characterized primarily as instrumental use, with the goal of drawing on citations to provide support and/or demonstrate what is known about a given topic. Although in several instances, citations were classified as having conceptual or political use, these types of citations occurred substantially less often than those classified as instrumental use. Further, whereas a few citations of political speeches and other sources appeared most closely aligned with political use, it is possible that these examples were being employed in both a political and an instrumental use concurrently, as described by Natow (2020a). The application of the Weiss (1979) typology for references may also question the appropriateness of classifying CRS as a knowledge broker. Returning to Meyer (2010), who emphasized research brokers’ roles in transforming knowledge, it is unclear if applying extant knowledge and research primarily through instrumental use can be classified as interpretation or strictly moving knowledge. Nevertheless, given that previous research has highlighted CRS’s influential role in the federal policymaking process (Dreyfus, 1976; Fagan & McGee, 2020; Gude, 1985; Kosar, 2020; Rothstein, 1990; Weiss, 1989; Wolanin, 1976), it is ultimately unclear whether its publicly facing reports offer a complete portrayal of the organization and its research capacity or role as a knowledge broker.
Limitations
The current study provides initial empirical insights into CRS and the reports it produces, but there are several limitations to this study. First, because CRS is involved in other areas of information provision to congressional members and staff, such as data analysis and creation of confidential reports (Dreyfus, 1976; Fagan & McGee, 2020; Gude, 1985; Kosar, 2020; Rothstein, 1990; Weiss, 1989; Wolanin, 1976), examining its publicly available reports does not fully encapsulate everything this organization does to support and inform Congress. As such, our study should not be viewed as a complete evaluation of CRS as an organization or information provider nor can its findings be attributed outside of an analysis of its publicly available reports. To this end, while the study provides insights into the use of academic research and other sources of information, the takeaways offered are reflected only in this specific area of CRS’s influence. Additionally, because of the limited empirical research on CRS, our consideration and operationalization of the Weiss (1979) typology was limited to the explicit use of references rather than gauging potential strategic decision-making (Penuel et al., 2017). Moreover, given this study’s focus on education-indexed reports, it is possible that other areas of federal policy are discussed and written in a different manner than the reports examined. Finally, given the continued limited access to the authors creating these publications as well as the individuals using them, insights can focus only on the information available to policymakers through CRS reports rather than the construction and/or utilization of these documents.
Conclusion
Despite CRS’s role as a key informant for federal policymakers, there exists limited empirical research examining this organization and its work. The passing of the Consolidated Appropriations Act of 2018 provided new opportunities for researchers and other interested persons to access previously unreleased reports and gain a better understanding of the information provided by CRS. Based on an evaluation of 55 education-related reports released during the 115th U.S. Congress, this study provides insights into the content and information sources that CRS references when developing its reports and, by consequence, the types of information that are available to federal policymakers. Notably, our analysis highlights an array of report topics that tend to be framed within either the preK–12 or postsecondary/higher education focus. More broadly, however, these reports either provide more general information about a topic or are framed as an overview and discussion around specific legislation. Despite this range in content, when considering what is referenced in these reports, there are some commonalities. Specifically, although CRS draws on a multitude of different sources to support their narratives, its reports overwhelmingly turn to current and proposed legislation and government-associated entities, including other CRS reports and analyses, as its primary information sources. In contrast, academic research and reports provided by independent research organizations tend to be considered far less often, which may suggest a more limited consideration of empirical evidence.
Given the limited research on CRS as an organization, several additional areas warrant greater examination. For example, CRS’s other responsibilities, such as meeting with congressional members and committees and conducting confidential analyses, could provide additional insights into the impact of the organization and how it compares to other intermediary organizations internal and external to the federal government. Further research on CRS and its various roles could also extend the current study by providing additional insights into the organization’s implicit goals and decision-making processes around research utilization. Relatedly, little is known about the employees of this organization, which may also provide greater understanding of the research capacity of CRS. As more information on research use in federal policymaking becomes available through CRS and other sources, future research could uncover how members of Congress set their agendas, decide on their requests for additional research on various topics, and determine the utility of in-house versus external research sources. As evidenced by other research areas, information provision is only one step in the policy process as the recipients and users of this information play an important role in how CRS’s work translates into informing decision-making (Tseng, 2012). Consequently, future research examining federal policymaking should consider the extent to which CRS is referenced by legislators and others in the policy process.
Footnotes
Notes
Authors
PAUL G. RUBIN is an assistant professor in the University of Utah’s Department of Educational Leadership and Policy. His research focuses on the intersections of higher education policy, governance of the postsecondary sector, and the use of empirical research in the policy process.
DR. JENNIFER B. AYSCUE is an assistant professor in the Department of Educational Leadership, Policy, and Human Development at North Carolina State University. Her research focuses on school integration in K–12 schools and federal education policy. Prior to joining the faculty at NCSU, Dr. Ayscue served as an American Educational Research Association Congressional Fellow in the United States Senate.
ELIZABETH M. UZZELL is a postdoctoral research associate in the University of Virginia Partnership for Leaders in Education. She has a PhD in educational evaluation and policy analysis from North Carolina State University, where she served as a graduate research assistant. Her research focuses on program implementation, state and federal education policy, and systemic inequities in P–12 education.