Abstract
Although gender parity has been achieved in some STEM fields, gender disparities persist in computing, one of the fastest-growing and highest-earning career fields. In this systematic literature review, we expand upon academic momentum theory to categorize computing interventions intended to make computing environments more inclusive to girls and women and consider how those characteristics vary by the success of the intervention. Particular attention is given to the efficacy of broadening participation and success for women in computer science, information technology, and related fields. After scrutinizing 168 relevant studies, 48 met the inclusion criteria and were included. We introduce a framework for gender equity in computing, expanding on existing research on academic and STEM momentum to encompass new domains representing social and structural momentum. Our analysis reveals the complex roles of intervention domains, strategies, goals, levels, and duration in shaping their efficacy. Implications for theory, research, and practice are discussed.
Keywords
Computing 1 is one of the fastest-growing career fields in the United States and globally; yet, the computing workforce may not keep up with the rising demand (Sax et al., 2017). It is vital to include women in these careers—not only to meet the labor demands of the 21st century but to close wage disparities between women and men (Abbiss, 2008; Fox, 2010). Despite the pervasive daily involvement in technology among individuals from all genders, the representation of girls and women in computing fields continues to be alarmingly low (OECD Directorate for Science, Technology and Innovation, 2018).
Gender inequities in computing participation risk harm to individuals within the field (e.g., students, faculty, staff, workers) and to the field itself, which cannot realize its possibility without first better representing its users and the society it reflects (Litts et al., 2020; Lynn et al., 2003; Pinkard et al., 2017). Girls and women typically experience unwelcoming climates in computing spaces from schooling to the workforce (DuBow & Kaminsky, 2019; Margolis & Fisher, 2002; Scott et al., 2017). Hall and Sandler (1982, 1984) noted girls and women in these male-dominated spaces experienced unfounded questions of ability and belonging from peers, mentors, and instructors. These behaviors and the reproduction of stereotyped beliefs surrounding computing ability and belonging persist today (Britton, 2016; Jensen & Deemer, 2019).
Increasing women’s participation in computing education continues to be of pressing importance. Indeed, this journal has published systematic literature reviews on gender equity in computing fields: one on primary and secondary information technology education (Volman & Van Eck, 2001) and the other on computer-related postsecondary majors (Singh et al., 2007). While each investigated contemporary research on girls’ and women’s experiences in these fields, neither focused on interventions and the application of research to diminish the gender disparities they reported. While gender inequity remains a problem, there has been an explosion of interest and research in gender equity in computing for decades 2 that addresses computing education across the educational life course and global efforts to achieve greater gender equity for women across social backgrounds and contexts. Systematic literature reviews published elsewhere tend not to focus on the most recent literature (e.g., Gürer & Camp, 2002), on gender (e.g., Kafai & Burke, 2015), or on interventions (e.g., Pantic & Clarke-Midura, 2019).
Yet, computing interventions focused on improving gender equity have been developed at each level of education and occupation. In general, each seeks to cultivate a gender-inclusive computing environment. Although individual strategies have been identified to increase girls’ and women’s representation in computing, it is unclear which strategies are reliable and potentially scalable. By investigating educational interventions with the potential for cultural change, this systematic literature review addresses the following research questions: (1) What are the characteristics of interventions that create inclusive spaces for girls and women in computing? (2) What are the goals of these interventions? (3) To what extent are these interventions associated with achieving their intended aims toward greater gender equity? 3 We evaluated the quality of existing evidence on interventions by synthesizing interdisciplinary peer-reviewed literature and the outcomes of each study. We close with a more expansive framework for academic momentum with potential use beyond research on gender equity in computing.
The Gender Gap in Computing
Problem Statement: While Malleable, Gender Inequality Persists in Computing
Women continue to be severely underrepresented in computing majors in postsecondary education, and employment rates among women in computing are even lower (Bonham & Stefan, 2017; Cheryan & Plaut, 2010; Dryburgh, 2000; Main & Schimpf, 2017). Women can be systematically pushed out of pathways to computing careers through lack of opportunity, persistent stereotyping, and unwelcome climates (Beyer, 2014; Heilbronner, 2012). Evidence suggests pathways toward computing careers for women have become less accessible over time. From 1982–1992, the proportion of women enrolled in computer science majors declined by 7% (Dryburgh, 2000). According to the U.S. Department of Education, since 1995, women’s completion of bachelor’s degrees in computer science declined from 28% to 18% (Perry, 2018). Black, Latina, and Indigenous women continue to be especially underrepresented in computer science majors and occupations (Charleston et al., 2014; Estrada et al., 2016; Myers, 2018).
Meanwhile, participation in science, technology, engineering, and mathematics (STEM) is a key avenue through which we might narrow gender gaps in economic opportunities (Abbiss, 2008; Fox, 2010). This is especially true for high-earning and high-demand fields such as computing (Corbett, 2015). Problematically, these fields miss a critical perspective through the exclusion of women and people of color. This oversight is detrimental to the progression of the field and of the larger society (Fox, 2010). Some science and technology disciplinary fields are more sex-segregated than others, as some have moved gradually toward and even achieved gender parity while others have not (Sax et al., 2017). Compared to other STEM fields, computing has a uniquely dramatic variation in the computing gender gap over time, with cycles of declining female participation following periods of rising gender equity (Lehman et al., 2017). As of 2019, women comprise 48.3% of the labor force in biological sciences compared to only 25.8% of computer and mathematical occupations (National Science Board, 2021). The question remains why this particular—and more recent—field is so volatile for girls and women.
Critical theorists argue we must redefine the problem by broadening the scope of what constitutes computing and challenge the field’s masculine-dominated culture (Abbiss, 2008). Notably, the literature’s definition of computing is currently contested and ambiguous. Some scholars define the pursuit of a computing career as seeking a computer science major or information technology major exclusively (Dryburgh, 2000), but other scholars include a broader range of majors in their definitions (Lent et al., 2008). The Association for Computing Machinery (ACM) defines computing disciplines as including the following majors: computer engineering, computer science, information systems, information technology, software engineering, bioinformatics, computational science, gaming and animation, and medical informatics (ACM, 2006). We adopt this last definition in this manuscript.
Persistent gender disparities in computing exist across the globe, with some variation. Researchers find that the more gender-egalitarian nations are (on measures such as tertiary educational attainment), the larger the gender gap in STEM and mathematics fields (Charles & Bradley, 2009; McDaniel, 2016). Several key studies establishing this pattern draw on the limited cross-national and cross-sectional survey data available, finding similar patterns: highly industrialized economies tend to have more rigid gender-typing of STEM career fields as evidenced, for example, in girls’ and women’s attitudes and performance (Charles, 2017). Recent research in nations with considerably less wealth suggests the potential for shrinking gender gaps in STEM areas including computing fields specifically (Perez-Felkner et al., 2020). Studies examining demographic differences within non–U.S. countries suggest the importance of broadening participation in all countries from the less resourced members of society, including First Nation and Aboriginal distance education learners (Gibb, 2006; Voyageur, 2001).
Explaining the Gap: Person Inputs and Intersectionality
Noted in the synthesis of women in computer-related majors research by Singh et al. (2007), scholarship on the gender gap in computing fields has too often attended to “a unitary notion of woman” (p. 517; original italics). Using a gender lens alone would obscure the intersecting challenges met by girls and women who may experience compounding forms of marginalization associated with economic disadvantage, disabilities, gender identity, sexual identity and orientation, race/ethnicity, and/or other identity statuses and lived experiences (Gaston Gayles & Smith, 2019; Ireland et al., 2018). Intersectionality is a theoretical and methodological tool used to assess structural, political, and representational axes by which women from marginalized groups have experienced discrimination and oppression (Cho et al., 2013; Collins, 2015; Crenshaw, 1991). These intersecting axes of inequality have been conceptualized as a double bind (Malcom & Malcom, 2011) that can hinder women’s success in computing, with consequences for future gender equity in the field.
Intersectionality was developed from Black feminist scholarship and can be used to investigate gender and race in tandem in academic STEM (Charleston et al., 2014; Prescod-Weinstein, 2020) and specifically in computing fields (Yamaguchi & Burge, 2019). Structural intersectionality addresses how environments can obfuscate the specific needs of girls and women of color, who may feel pressured politically to prioritize one identity (e.g., gender) over another (e.g., race) and can be tokenized when their representation is scarce—as it often is in computing classrooms, faculties, and in the workforce (Haynes et al., 2020). Recent research on intersectionality attends to additional forms of marginality, including sexual identity (Miller et al., 2020) and groups underrepresented in computing fields such as Latinas (Rodriguez et al., 2020). Aggregating social identity categories to examine gender alone can allow blind spots; more nuanced studies of gender disparities in computing should consider gender equity.
To address entrenched gender inequality, it is essential to understand the mechanisms behind why social identity continues to affect girls’ and women’s representation in STEM fields overall (Kim et al., 2018) and computing specifically (Margolis et al., 2000; Rodriguez & Lehman, 2017). Structural changes that can increase girls’ and women’s equity may be especially effective for those with intersecting gender and racial/ethnic identities (Ireland et al., 2018) and those from less economically advantaged families (Frank et al., 2004).
Current Review: Computing Interventions in Education for Gender Equity
Computing interventions focused on gender equity in computing vary widely in scope, duration, and goals. It remains unclear which are effective and scalable. Further evaluation is needed to determine whether these interventions can be implemented with larger and more diverse populations. Therefore, we sought to review the quality of existing gender equity interventions by synthesizing interdisciplinary peer-reviewed evidence.
Methodology
Conceptual framework: Momentum for equitable pathways through STEM
In this manuscript, we assess computing interventions to enhance gender equity and their relative success in breaking down structural barriers that affect girls and women across intersecting identities. We propose a framework to analyze elements of computing entry and persistence and interventions that improve them. This framework informs our classification criteria in response to research questions 2 and 3 and extends from prior research on momentum with respect to academic opportunity generally in STEM specifically.
Academic and STEM Momentum
With empirical data from nationally representative longitudinal cohort studies, Adelman (1999, 2006) developed a widely used framework for academic momentum, arguing for the importance of cumulative experiences with mathematics course-taking to enhance the likelihood of (all) students’ success in college and preparedness for mathematics-intensive degree fields. Wang (2017) conceptualized the teaching and learning domain of STEM momentum as students’ mastery of subject matter and strategies they may employ to “regulate, adjust, adapt, and assess one’s own learning processes” (p. 284). The curricular domain of momentum referred to students’ enrollment and the likelihood of persistence to degree through coursework. She also expanded on the motivational domain of momentum, including students’ aspirations, mindsets, persistence, and agency (i.e., efficacy in seeking knowledge and resources). Wang (2013) also discussed factors beyond postsecondary institutional and precollege demographic factors that may facilitate or inhibit STEM momentum. Altogether, the momentum framework identifies potential mechanisms for closing persistent disparities that especially disenfranchise populations underrepresented in postsecondary education and STEM careers. This framework thus aligns with that of our systematic review, intended to identify potential evidence to enhance gender equity in computing, and informs our classification.
We leverage the momentum framework given its attention to potentially either enhance or reduce educational opportunities. We reach across the educational life course to assess educational interventions aimed at expanding gender equity in computing fields, including curricular and extracurricular learning opportunities in computing that extend beyond secondary and postsecondary schooling. Wang’s (2017) study found that counter-momentum friction (e.g., financial and other barriers) and carry-over momentum supports (i.e., various personal and other precollege factors) may factor into individuals’ academic momentum pathways. To assess interventions’ efficacy in enhancing gender equity, with attention to transparency and scalability, we explicitly identify and add domains to the model, as described later.
Proposed Classification Framework
Drawing on the literature described previously, we expand upon these conceptualizations of momentum and propose a framework for gender equity in computing (see Figure 1). We include teaching and learning, curricular, and motivational domains. We also expanded the theory to attend more directly to social and structural momentum domains, which could bolster and sustain computing ambitions. Figure 1 shows personal inputs that may shape entry to computing pathways; intersecting relationships between motivational, curricular, and teaching and learning domains present in the earlier momentum models; “social” domain that may function to support individuals within a complex system; and a “structure” that anchors this model, representing schools, community organizations, universities, or other potential sites for intervention.
Momentum framework for gender equity in computing.
We consider contextual supports and barriers a distinct momentum domain, as they are necessary for systemic, sustained gender equity in computing entry and persistence. This structural domain of momentum is one of our two key additions to Wang’s (2017) framework. Broadened opportunity appears necessary to expand gender equity in who is prepared for and who enrolls in computing coursework, with the potential to progress into computing occupations. Opportunities to engage with computers and the computing field are not distributed equitably (Corbett & Hill, 2015; Elliott, 2017). Thus, to examine momentum, it is necessary to assess the degree to which various structures allow for its development.
Moreover, social support is particularly important for girls and women in computing to foster an inclusive environment (Hanks et al., 2011). A commonly cited barrier to women’s entry and persistence in computing is a chilly climate (Hall & Sandler, 1982), whereby peers, computing instructors, and potential advisors and mentees fail to attract and retain women in the field through unwelcoming and noninclusive behavior. More recent studies have demonstrated both subtle and direct hostility shown toward women in computing (e.g., Stout et al., 2017), including but not limited to the denigration of their skills, intellect, and merit (DuBow & Kaminsky, 2019). Our second addition to the momentum framework—the social domain of momentum—evaluates this component of computing entry and persistence.
Accordingly, we leverage our expanded momentum framework to offer a robust understanding of how academic momentum is created and maintained throughout the educational life course. This model may serve as a framework for future studies of momentum in myriad academic fields beyond computing. We apply this framework to the computing field, with a focus on building scalable interventions aimed at improving gender equity. The following section explains these domains that inform our systematic classification of the literature reviewed.
Teaching and Learning Domain
Scholars have examined how to improve students’ computing engagement through fostering learning and inclusive environments. Women were found to complete more courses when there is a balance of skill, interest, and challenge in their college-level computing coursework (Milesi et al., 2017). Skill development can enhance girls’ computing major selection and career aspirations (Mau & Li, 2018). This research suggests inclusive teaching and learning interventions have the potential to reduce gender disparities.
Curricular Domain
Exposure to computer science coursework is a key predictor of girls’ interest and preparedness for computing in postsecondary education and beyond (Fisher & Margolis, 2003; Tillberg & Cohoon, 2005). Limited access to courses can impede girls’ and women’s access to computing-related postsecondary programs and careers. Therefore, the curricular domain of computing momentum appears dependent on the structural domain. Curricular offerings have increased in the past decade, as have policies and programs at district, state, and federal levels to increase the capacity of the computing education labor force (Alba, 2016; Goode, 2007; Moller & Crick, 2018). Offering on-ramps to computing majors
Motivational Domain
Women are aware that computing fields are male-dominated, and this awareness can be detrimental to their participation (Blickenstaff, 2005; Carnevale et al., 2011; Chen et al., 2023; Hill et al., 2010). Scholars note stereotypical notions of gender function as a deterrent against pursuing computing (Cheryan et al., 2009, 2012; Fisher & Margolis, 2003; Kekelis et al., 2005). Undergraduate women show a “stereotypical” role model (e.g., likes playing video games, watching anime) expressed a significant decrease in computing interest 2 weeks after exposure (Cheryan et al. 2012). In a longitudinal study of boys and girls entering high school, Sáinz and Eccles (2012) found girls’ perceived self-efficacy in computer science declined during high school, despite similar levels of achievement among girls and boys. Given findings that girls’ selection into computing and other STEM-related coursework can be shaped by their social contexts (e.g., Frank et al., 2008), we turn to social and structural domains to assess their potential roles in the development of computing momentum.
Explaining the Gap: Social and Structural Domains
Pathways toward gender equity in computing—for individuals and collectives—may be stalled by insufficient social and structural support. These domains are especially important given evidence of a chilly climate in male-dominated STEM fields, which can hinder girls’ and women’s sense of belonging in classrooms, disciplinary cultures, and career fields (Hall & Sandler, 1984; Lee & McCabe, 2021; Morris & Daniel, 2008; Simon et al., 2017). Computing classrooms, extracurricular clubs, and disciplinary spaces have typically been constructed as masculine spaces, whereby women are excluded from the “clubhouse” (Margolis & Fisher, 2002). Cheryan and colleagues demonstrated how stereotypes and limited portrayals of girls and women in computing and related fields could result in their inhibited or “precluded interest” (Cheryan et al., 2009, 2015). Given myriad interventions to broaden interest and participation in computing (see, e.g., Kay, 2006; Scott & White, 2013), social contexts may be especially malleable and well-positioned for gender-equity-focused computing interventions.
Both formal and informal contexts can influence students’ opportunities to participate in computing fields and the potential for meaningful social interactions within them (Blau, 1977; Cheryan et al., 2009, 2015; Margolis & Fisher, 2002). From technological toys and games in childhood to computing major selections in adulthood, students’ socialization experiences can shape their pathways toward or away from these fields. Girls and women may also value professions associated with social value or impact; computer science and programming have not typically been framed to align with these interests and values (Fong et al., 2016). Thus, women may expect to find less value in computing careers (Beyer, 2014; Weinberger, 2004). Interventions might aim to be more inclusive of students’ interests and values, irrespective of gender identity (see Cheryan et al., 2009).
Social Domain
Access to peer and adult mentors in computing may also shape computing interest and persistence (Main & Schimpf, 2017). In turn, we evaluate the social domain of momentum. Peers influence students’ interest in scientific careers and majors, especially among girls (Frank et al., 2008; Robnett, 2016). Mentors play a key role in fostering girls’ interest in computing, potentially challenging stereotypes and relaying key information about major selection and career opportunities (Dryburgh, 2000; Main & Schimpf, 2017). Indeed, negative interactions can and do occur in computing environments, including microaggressions and other forms of hostility and harm (Erete et al., 2021; Michell et al., 2017). Exposure to such negative signals is associated with declines in girls’ interest in computing fields (Master et al., 2016). These findings highlight necessary structural changes to enhance opportunities to make computing more engaging and accessible throughout the life course.
Structural Domain
School structures are key sites for socialization into and away from computing, but these sites are rarely examined as the base upon which academic momentum is built. What we named the “structural domain” serves as the foundation for introduction and retention in computing. From early childhood through postsecondary education, interventions seeking to broaden girls’ participation tend to be based in schools or other community structures. These may include curricular, extracurricular, or summer interventions but are too often based in sites with already high investment in computing and technology, such as with well-resourced schools and/or near computing hubs such as Seattle and Silicon Valley (e.g., Friend, 2015). Computing access is not evenly distributed.
Higher education institutions also vary widely as sites for students to explore, study, socialize, and gain access to computing fields. As in the K–12 literature, broadening participation interventions are often located in more resourced institutions and those with strong computing programs. Interventions may be especially impactful at institutions that broaden the computing field across gender, race, and socioeconomic status. Given the range of educational pathways women may take through schooling to the computing workforce, it is important to also consider developing and investigating interventions at minority-serving institutions, community colleges, regional comprehensive universities, and other less-studied institutional types to uncover potential mechanisms to increase the full representation of women in computing.
Databases, discipline-specific journals, and search parameters
We employed inclusion and quality criteria to objectively constrain the vast interdisciplinary literature on this topic. These criteria were used systematically to select the appropriateness of studies in answering our research question, assess the quality of the studies, and synthesize the evidence on our topic (Collins & Fauser, 2005). Systematic literature reviews are not prone to the bias that can arise in literature reviews that do not use a scientific approach to data collection (Petticrew & Roberts, 2006). Because of the wide variation in methodology and disciplinary norms in reporting methods and effect sizes, this intentionally inclusive and interdisciplinary assessment of the literature on computing interventions was especially well-suited for a systematic review.
As a potential limitation, we included only studies published in English. Our research team’s diversity allowed us to initially explore the inclusion of manuscripts in other languages but doing so presented potential challenges for analytic consistency. Instead, authors robustly reviewed and coded the publications based on the journal readership rather than cultural and linguistic identities of the authors.
We searched across four different databases and 12 discipline-specific journals, using a total of nine search terms to identify a large population of articles that might be included in our sample. Specifically, databases included EBSCO, ERIC, Google Scholar, and WorldCat databases; education journals including Economics of Education Review, Educational Researcher, Review of Educational Research, and all other related AERA journals; and social-science journals that typically publish research on gendered inequality in STEM, including Sociology of Education, Gender and Society, Sex Roles, American Sociological Review, and American Journal of Sociology. These searches generated disciplinary journals and proceedings in computing fields, as evidenced in supplemental Appendix Table A1 and our reference list. Search terms included “gender and computer science,” “gender and computing,” “gender and information technology,” “women and computer science,” “women and information technology,” “women and computing,” and “women in information technology.”
The search identified a total of 168 publications including academic journals, conference proceedings, dissertations, scientific magazines, and books discussing computing interventions. The final sample was narrowed down by a meticulous selection and screening process described and displayed—along with their accompanying rationale—in Table 1. We evaluated all 168 collected publications, judging them against inclusion and quality criteria (see Figure 2).
Screening process and analytic sample refinement
Inclusion and quality criteria.
Inclusion Criteria
We included publications in our sample if they explicitly discuss a computing intervention, focused on gender, and were published in English between 2000–2020. Publications were excluded if they did not meet these criteria—for example, if no intervention was specified (e.g., Sullivan & Bers, 2013).
Computing Interventions
We did not constrain the characteristics of the implemented interventions. Instead, interventions included examined a variety of interventions across a wide range of institutions. The initial inclusion screening narrowed our sample to 87 publications.
Outlet Type
We excluded books and dissertations from our analyses (n = 41), given their distinct scope and because they typically do not undergo traditional peer review. We retained peer-reviewed conference proceedings and scientific magazines for several reasons. First, traditional journal outlets are not the exclusive domain for peer-reviewed, widely cited, and impactful scholarly research in computing disciplines, given the typical publishing lag time and rate of software and technological change. Instead, the research includes a broader landscape responsive to rapid technological advances while still being perceived as robust; journals, proceedings, and scientific magazines are treated similarly as sources of research evidence (Luo et al., 2018; Meho & Rogers, 2008). Given the frequency and use of these outlets as influential vehicles for research on computer science, computer and electrical engineering, and information science advances, excluding conference proceedings would ignore a sizable proportion of the research being done in this area—including broadening participation interventions. Conference proceedings and scientific magazines undergo peer review but are unlikely to be subsequently published in academic journals; they are also disseminated widely to faculty, academic leaders, and other practitioners in the field. These publications are thus likely to be used in the development and implementation of computer science interventions worldwide.
Year
We restricted our search to a 20-year period to avoid duplicating the work of another systematic review that effectively reviewed this literature up until the year 2000 (Volman & Van Eck, 2001). This inclusion screening by year removed another 40 publications.
Quality Criteria
We systematically reviewed the quality of the publications selected from the initial inclusion/exclusion review. We adopted a modified version of the quality criteria presented in Garcia and colleagues’ (2019) systematic review of Hispanic-serving institutions. The criteria questions allowed us to identify and assess the rigor and scientific quality of each publication (see Figure 2). Few intersectional and international studies cleared the criteria, despite our inclusion criteria being designed to be open to a wide array of research outlets. This was disappointing but suggested the need to expand pathways for scholars from and countries underrepresented in research journals.
We assessed the quality of computing intervention publications using 10 criterion measures. In this review process, we examined the objective of the interventions, conceptual framework, methodology and data employed, presentation of findings, and conclusions drawn for each publication. For example, we could not include a published conference paper or scientific magazine article that presented results from a series of findings with insufficient methodological detail to support how they arrived at those results and interpretations.
Within our sample, we initially coded for two types of intervention studies, identified as either strict or inclusive interventions. Strict interventions were defined as initiatives formed to address disparities in participation and persistence among girls/women in computer science; these could include afterschool or summer programs, curriculum development and/or expanded offerings, extracurriculars, professional development, or other efforts. Our more inclusive definition of interventions was “an initiative formed to address disparities in participation and persistence among girls/women in computing, including legislation and policy changes, and broad attempts to change the culture of computer science spaces or a publication discussing results of potential intervention. Most interventions in our sample of studies with evaluations met the “strict” definition (n = 48), and a minority met the inclusive definition (n = 18).
We only included interventions that fit our strict definition of computing intervention, as most inclusive interventions (n = 18) were a proposal for an intervention program that had not been evaluated (e.g., Barker et al., 2014). Studies that did not meet the quality criteria were excluded from the final review (n = 21): those with inadequately or insufficiently described methods or findings (n = 14) or that were missing the theoretical framework or context for the study (n = 7). Publications were only included in the final sample if all standards were met. This left a final total of 48 publications, comprised of 34 journal articles, 13 conference proceedings/scientific magazine articles, and one substantial peer-reviewed book chapter that met our quality criteria and were included in this systematic review.
Development of Coding Structure
Our six-member analysis team employed a multistage analysis process. First, we used a grounded-theory approach to classify an initial randomized subsample of 20 manuscripts to allow themes to emerge naturally. Next, we independently applied an emergent classification scheme, developing additional memos justifying the application. Discrepancies were collaboratively adjudicated to establish agreement among the team about the parameters of each category and each study’s alignment with a given category. This iterative process was used in two additional cycles to test our initial coding structure on a randomized subsample of five articles and then another. We discussed and further refined our coding structure over a series of meetings, generating a coding manual. Subsequently, we identified, reviewed, and recorded additional descriptive information about each intervention. Coding distinctions were discussed as a team until reaching a consensus. This process was developed and maintained over multiple months to ensure reliability and consistency between the authors. This process resulted in the final coding framework used.
Table 2 provides the complete list of codes and their descriptions. Our final codes addressed both the goals of the interventions and their outcomes. We conceptualized the goals and outcomes of the interventions separately, as these interventions tend to have broader aims than tangible, measured outcomes of the intervention. While interventions may seek to improve social connections, access to opportunities, and more-relevant coursework, they may only successfully foster social connections. Coding thus captures alignment between the stated goals of the interventions and their measured outcomes.
Code list and descriptions
The intervention programs had an array of intended goals we categorized into the following codes: (1) improving computing content, (2) increasing computing access, and (3) improving social support in computing spaces. The range of specific outcomes of these intervention programs was categorized into the following codes that addressed domains of improving gender equity in computing: (1) teaching/learning domain, (2) motivational domain, (3) curricular domain (identifying persistence as a distinct proportion of this domain), (4) social domain, and (5) structural domain. These codes were assigned to interventions that specifically and explicitly measured outcomes in each category. Three of these domains aligned with our framework for computing persistence. Although persistence was a part of the curricular domain, the volume of studies focused on this outcome specifically encouraged us to capture it as its own outcome category in the results.
Next, we logged key information about the type and scope of the interventions. First, we indicated whether the intervention occurred at the elementary, middle school, high school, postsecondary, or workforce levels, or whether it was a longitudinal study. We indicated the number of institutions, organizations, schools, or workplaces involved with the intervention and the target population of the program and primary intervention strategy. We also documented the duration of each intervention and how the intervention was evaluated. Finally, we indicated whether the goals of the intervention were met. This last step was key to identifying characteristics of successful interventions.
Analytic Strategy and Final Coding
In the next phase of analysis, the remaining studies were randomly assigned to the authors and coded independently using the conceptual framework explained previously. Our conceptualization of effective computer science interventions involved a process of examining each intervention to discern types, goals, methodologies for implementation and evaluation, and outcomes. Specifically, interventions were examined for location and reach of implementation, target population, duration, and primary activity involved. This vital information captured contextual variation in the size and types of interventions implemented. We examined in detail the outcomes of interventions and whether claims made by the authors about the implications of these interventions were warranted. This required us to attend closely to the methodology of evaluation to ensure they could make accurate claims about the intervention’s impact. Interventions were coded as successful if outcomes measured in the publication matched at least one of their stated goals. Once interventions were coded, we evaluated the statistical significance of nonrandom associative relationships among the intervention studies using Fisher’s exact tests in STATA 16. This measure generates a precise p-value directly, without a separate test statistic; it is more conservative than a similar chi-squared test of association and is importantly more appropriate for the analysis of small samples (Bind & Rubin, 2020; Upton, 1992).
Methodological Limitations
There are a few limitations of this systematic review. As noted earlier, the literature from our search tends to focus on girls and women in a binary gender framework; future systematic reviews may uncover more recent data collections and research on the experiences of transgender or nonbinary computing students. In addition, we note while the initial collection of publications was exhaustive in design, we were limited to databases the authors have access through institutional library databases and open-access sites. There may be computing interventions published in outlets with limited circulation. To mitigate this limitation, we included Google Scholar and WorldCat databases for broader reach.
Results
RQ1: Characterizing Interventions for Gender Equity in Computing
Level of Interventions
Table 3 presents characteristics of interventions studied in this systematic analysis, by level, scope, and duration. Computing interventions took place prior to high school (n = 11, with one elementary and 10 middle school studies; 22.9%), upper secondary or high school (n = 14; 29.2%), or at the college/university level (n = 18; 37.5%). The remaining interventions were longitudinal interventions spanning from elementary education to the workforce level (n = 5; 10.4%). The level of the intervention audience was not significantly related to other characteristics of the intervention, from the number of sites where the intervention was deployed to the duration of the intervention. Despite variation in the levels at which interventions were deployed, the goals of these interventions are also not significantly different. Notwithstanding descriptive variation, these interventions’ goals and outcomes did not vary significantly by level.
Characteristics of interventions studied
Scope of Interventions
Interventions studied ranged between 1 and 10 sites (e.g., school, university, summer camp). Most were single-site interventions (n = 31; 64.6%). Just under 20% of the interventions occurred at 2–5 sites (n = 9; 18.8%), 10.4% were deployed at 6–10 sites (n = 5), and 6.3% were deployed at more than 10 sites (n = 3). The scope was mostly unrelated to intervention goals being met or types of goals and outcomes. However, there were notable exceptions. First, interventions at a single site were more likely to express a goal of improving content than those at multiple sites (p < 0.01). Second, single-site interventions were less successful with curricular aims (i.e., 3 met vs. 19 unmet), when evaluating intervention curricular outcomes by scope (p < 0.02), perhaps because they were earlier stage interventions not yet ready for scale.
Duration of Interventions
The largest proportion of interventions lasted either for 1 academic year (n = 10; 20.8%) or between 1–3 years (n = 9; 18.8%). Another 23% were briefer: 1 semester (n = 7; 14.6%) or just 1 day (n = 5; 10.4%). Interventions lasting between 1 semester and 3 years were most likely to report content improvement goals (p < 0.03), compared to shorter interventions lasting days and longer interventions lasting 3 or more years. For instance, seven interventions lasting 1 year met their content goals, as compared to zero that did not. Otherwise, duration is not significantly associated with intervention goals or outcomes. Shorter interventions still have the capacity to be successful and may have a wider reach. Short interventions, like a 1- to 2-day workshop often require less institutional commitment, which may increase their scalability.
Intervention Evaluations
We determined whether goals were met in each intervention using the evaluation of the intervention presented in each study. Because we excluded studies without clearly articulated and robust discussions of their methodology from our final sample, the design allows us to independently assess the reported results to identify whether an outcome was achieved. Intervention evaluation methodologies were primarily quantitative (n = 28; 58.3%), but 14.6% were evaluated qualitatively (n = 7), and 27.1% employed both quantitative and qualitative methodology (n = 13). Quantitative evaluations typically took the form of pre- and post-tests and surveys, while qualitative evaluations involved interviews. Mixed methods evaluations used both survey and interview methods.
RQ2: Goals of Computer Interventions for Gender Equity
Improve Computing Content
Recall that Table 2 describes analytic codes, with attention to intervention goals and outcomes, and number of studies that fell in this category, the proportion of studies measured, and the proportion of studies that were successful in realizing their intended aims. With respect to the goal of improving computing content, interventions with this goal focused on new course designs, course content revisions, improvement of teaching practices, and/or content of extracurricular activities. These interventions sought to make changes to the presentation or content of computing material to make computing spaces more alluring or inclusive to girls and women by increasing teaching and learning and curricular momentum. Most interventions sampled had this goal (n = 34; 70.8%). The overwhelming majority of studies with the goal of improving content achieved it (n = 24; 70.6%). Exactly how each of these goals was achieved is discussed in the following sections on outcomes.
Improve Social Support in Computing
We assigned the improving social support goal code to studies that focused on fostering a community for girls and women to become interested in or persist in computing spaces, improving the social domain. Interventions assigned this code provided mentors, formed cohorts, created social groups (e.g., online or in person) for girls or women, and/or facilitated extracurricular activities or learning communities (e.g., peer/pair programming or collaborative activities to produce nonstereotypical images of women in computing). More than 1/3 of the interventions assessed had social support as a goal (n = 18; 37.5%), and 72.2% (n = 13) achieved it.
Increase Structural Access to Computing
Interventions with the goal of increasing access to computing sought to create opportunities for girls and women to learn more about and engage with the computing field, thus improving the structural domain. Those receiving this code provided financial support, scholarships, grants, single-day informational activities, organizational change, school-industry partnerships, and leadership or professional development activities, or changed the structure of advising. Of these interventions, 47.9% reported this goal (n = 23), achieved by 73.9% (n = 17).
RQ3: Goal Alignment: Did These Interventions Achieve Their Intended Outcomes?
Intervention outcome goals coded by the research team aligned closely with our momentum framework for gender equity in computing. However, outcomes evaluated in these interventions often also aligned with momentum components not explicitly stated as intervention goals. For example, interventions only focused on improving motivation may have spilled over into other domains. This was not uncommon. Here, we identify (a) how the goals of computing interventions compare to their outcomes and (b) the characteristics of successful and unsuccessful programs. Tables 4 and 5 summarize these patterns.
Most common characteristics of successful interventions by outcome achieved
Most common characteristics of unsuccessful interventions by outcome achieved
Increasing Teaching and Learning Momentum
Interventions were assigned the teaching and learning momentum outcome code if they focused on achieving gender parity in computing performance. Over 45% of interventions in our sample measured teaching and learning outcomes (n = 22; 45.8%), and of these, 63.6% achieved them (n = 14). Fifteen successful interventions had the stated goals of improving computing content, six sought to increase access to computing, and five to improve social support in computing. 4 These interventions often include modifying course design or revising the content of a computing instruction method to be more relevant to girls’ interests (n = 10). In Denner et al.’s (2012) intervention, girls developed their own computer games in a middle-school introductory computing course.
The largest proportion of successful interventions took place at the college level (n = 6) for 1 year (n = 5). At the college level, Butler et al. (2017) used “pencil puzzles” like sudoku in their teaching of abstract programming concepts. Medel and Pournaghshband (2017) included more examples of women in course materials, in part by using “they” pronouns rather than “he” or “she.” Course-content revisions attending to gender engaged other forms of inclusivity. Python-based EarSketch programming used music remixing in high school and college computer science courses as a culturally relevant entry point for students across racial/ethnic and gender backgrounds to engage in coding (Magerko et al., 2016). Culturally responsive pedagogy was also employed in COMPUGIRLS, an intervention focused on girls of color (Scott & White, 2013).
Some interventions associated with teaching and learning had multiple goals with varying results. Of successful interventions, 37.5% involved computing courses (n = 6) and 25.0% involved game-making activities (n = 4). Five of six course-based interventions had the goal of improving computing content, but four also had the goal of increasing computing access, and two of six hoped to improve social support in computing. These aims could align. For example, all game-based gender equity interventions sought to make computing content more inclusive, and one had the additional goal of improving social support in computing. Of the six interventions that were unsuccessful in improving teaching and learning momentum, two were course-based interventions, two were extracurricular activities, one involved pair programming, and one was a recruitment program. Unsuccessful interventions were evenly distributed across middle school, high school, and college (n = 2 each) and usually took place for one semester.
Improving Motivational Momentum
Interventions improving mindsets about computing received the motivational momentum outcome code. Interventions coded this way measured computing attitudes (e.g., confidence, interest) or perceptions of computing (e.g., perceived likelihood of success, attitudes toward the computing field, sense of belonging, or perceptions of computing as a career choice). In our sample, 66.7% of interventions measured motivational momentum (n = 32; 66.7%), and of these, 71.9% improved computing mindsets (n = 23). Of the course-based interventions that improved computing attitudes, half had the goal of improving content and half had the goal of improving access. None of the course-based interventions that improved motivational momentum stated the goal of improving social support. Most successful interventions were deployed at just one site (n = 17) for 1 day (n = 5) or 1–3 years (n = 5).
The intervention strategy that successfully increased motivational momentum most was extracurricular activities (29.6%; n = 8); for example, participation in Ireland’s CodePlus afterschool program increased girls’ perceived ability to program (Sullivan et al., 2015). Other coding clubs focused on girls from racially minoritized backgrounds (e.g., Scott & White, 2013). For instance, Nesiba et al. (2015) studied the Young Women in Computing outreach initiative in New Mexico. The program aimed to enhance computing interest among Hispanic girls from elementary school through college. Strategies included afterschool clubs and summer camps, high schoolers’ engagement with text-based app programming, and increased interest in CS careers following shorter “roadshows.”
Additional intervention strategies found success. These included courses designed to be more relevant to girls’ and women’s interests and more inclusive of girls and women in in-text examples (22.2%; n = 6) and collaborative game design (11.1%; n = 3). Conferences and events also found success, such as Grace Hopper conference attendance among first-year Harvey Mudd College students. In their study of the multiday mentoring grad cohort program for Ph.D. students in computing, Alvarado and Judson (2014) found that interest in computer science majors increased following attendance at this convening. Women participants reported stronger interest in becoming well-known in their field, greater interest in giving back to their community, and stronger computing identity than nonparticipant women. Grad cohort participants reported identifying with computing to the same degree as propensity-score matched nonparticipant men.
Most extracurricular interventions aimed to improve content (n = 7) and increase access to computing (n = 5; e.g., Witherspoon et al., 2016), but only two had the goal of improving social support. Only five interventions measuring motivational outcomes did not successfully improve them. Of these, two were course-based, one extracurricular, one game-based, and one involved pair programming. These few unsuccessful interventions occurred the most in middle school (n = 2), at just one site (n = 3), for 1 semester (n = 2) or 1 year (n = 2).
Increasing Curricular Momentum
The increasing curricular momentum outcome code was assigned to interventions measuring enrollments and representation in computing spaces. Less than a third of the studies in our sample measured curricular momentum (n = 12; 25.0%), and 75.0% (n = 10) of these interventions were successful. Among interventions that successfully improved curricular momentum, 44.4% of them involved an introductory computing course (n = 4; e.g., Salminen-Karlsson, 2002) and 22.2% involved extracurricular activities (n = 2). One extracurricular activity integrated fashion, design, and computing to teach middle school girls about wearable computing in a pilot summer course (Lau et al., 2009). Among interventions successfully improving curricular momentum, increasing access was the most common goal (n = 10; p < 0.01). These interventions were most often deployed at the college level (n = 5) at one site (n = 5) for 1–3 years (n = 4).
Interventions that were unsuccessful in increasing curricular momentum (n = 3) involved one competition, one extracurricular activity, or one game-design activity. A robotics competition intended to increase structural access to computing and involved 2–5 sites over the course of a day (Witherspoon et al., 2016). The extracurricular activity sought to both improve content and structural access and engaged 6–10 sites for 1–2 months. The game design activity aimed to improve content involved 13 Scottish schools for more than a year, falling short of their aims to increase enthusiasm for computing and reduce gender gaps following the game authoring intervention, perhaps because of a high baseline affinity for gaming (Robertson, 2013).
Improving Social Momentum
The findings described below align with research highlighting the importance of peer networks, community, and mentors in developing computing interest (Blum & Frieze, 2005; Cozza, 2011; Ehrlinger et al., 2018). We assigned improving social momentum outcome code to interventions that evaluated the effectiveness of interventions in fostering social connections in computing spaces. Less than half of the studies in our sample evaluated social domain outcomes (n = 17; 35.4%), and of these, 70.6% (n = 12) were successful.
Among successful social domain interventions, 21.4% involved extracurricular activities (n = 3; e.g., Pivkina et al., 2009, studied the young women in computing program in New Mexico, which was studied later by Nesiba et al., 2015). Support communities outside of the computing classroom comprised another set of social domain–focused interventions (n = 4). These interventions fostered social connections through paired activities, like pair programming, or through the development of networks, such as a campus student organization where Canadian undergraduate and graduate student women could converse openly about their computer science experiences (Tsoukalas & Wu, 2009). Four interventions focused on curricular changes that sometimes also included pair programming and collaboration, such as in the study of computer science college students’ experiences with a 3D virtual reality programming intervention (Ortega et al., 2017). Altogether, the most common goals of successful social domain interventions were to improve computing content (n = 11) and social support (n = 10; p < 0.05).
Interventions that improved social momentum were most deployed at postsecondary (n = 5) and secondary levels (n = 4), at one site (n = 10), for 1–3 years (n = 5) or 1 semester (n = 4). Interventions that were unsuccessful in this domain (n = 3) involved course design (n = 2) and an extracurricular activity (n = 1). Improving content was the most common goal of interventions that did not meet their goals (n = 3). Unsuccessful course-based interventions were deployed for 1 year at the high school (n = 1) and college (n = 1) levels, respectively. The unsuccessful extracurricular intervention was deployed at the middle school level, at one site, for 2–5 days. Several postsecondary institution-based interventions—such as collaborative learning communities—were developed at a series of institutions, departments, and programs over time. These models were designed with collaborative activities, such as group and pair programming, and integrated social elements, such as designing wearables (wearable music players, sensory reactive clothing for babies servicing as a parenting aid, responsive purses, wearable games) in StitchFest—a hardware hackathon collaborative project (Richard et al., 2015).
Increasing Structural Momentum
This code was assigned to interventions evaluating whether programs were successful in expanding structural access. These interventions focused on increasing tangible opportunities in computing and access to information about the computing field. About one-third of interventions were evaluated on this outcome (n = 14; 29.2%), and 11 were successful. These interventions most commonly took place at the college (n = 4) or high school (n = 3) levels, at one site (n = 7), for 1–3 years (n = 4) or for more than 3 years (n = 2). These interventions often took the form of new or revised courses or extracurricular activities which provided spaces for girls and women to expand their interest and skills in computing and learn more about computing career paths. The most common goal of interventions that successfully increased access to computing was to increase access (n = 8), but many also had the goal of improving computing content (n = 6) and improving social support (n = 5). The most common strategies for increasing structural access in computing were courses (36.4%; n = 4) and extracurricular activities (18.2%; n = 2).
Structural approaches could be applied to these strategies. For example, in a school-wide curricular intervention at a private all-girls middle school in Silicon Valley, the annual mandatory computer science courses were taught by computer science–trained women teachers, mentor networks were available, learning was project-based, and all students received laptops (Friend, 2015). Correlational findings suggest the intervention met its goals by reducing structural barriers to computer science. Lynn et al. (2003) developed and evaluated an experiment aiming to enhance computing interest and enjoyment among public school girls aged 9–12 via a website prototype integrating prior approaches to reducing gender bias and stereotyping in computing, with attention to making computing fun, playful, and accessible at home; computing interest and enjoyment increased among girls in the treatment across demographic subgroups.
Of the three interventions that evaluated structural outcome but did not achieve it, one involved course design, one was an extracurricular intervention, and one involved pair programming. The course-based intervention sought to improve computing content and social support through the development of an introductory computing course designed to bring women into the computing field. The extracurricular intervention sought to improve computing content only; the pair-programming intervention aimed to improve computing content, access to computing, and social support. The interventions increased structural access by explicitly recruiting girls and women to participate in these activities and bringing these interventions to locations with less access to computing. Both the pair-programming and extracurricular interventions took place at the middle school level, while the course-based intervention took place at the high school level. The course-based intervention took place at 6–10 sites for 1 year, while the extracurricular intervention took place across 2–5 sites for 1 day, and the pair-programming intervention involved just one site for 1 full semester.
Improving Persistence
The improving persistence outcome code was assigned to studies where the evaluation measured whether the intervention improved retention in the workforce or in curriculum, improved workforce advancement or promotion, successfully encouraged continued or long-term participation in computing, increased transfers from community colleges to 4-year institutions (i.e., staying in this field), or improved graduation and/or credential attainment (i.e., certificate or degree). While aligning with the curricular domain of computing momentum, a quarter of our sample evaluated persistence outcomes (n = 12; e.g., Sullivan et al., 2015). Most of these interventions (e.g., Lagesen, 2007) successfully improved persistence (n = 9).
The most common goals of interventions that successfully improved persistence were improving computing content and improving social support in computing. Two of these successful interventions were course-based interventions, two were award-based, one was mentorship-based, one involved pair programming, and one involved a recruitment event. For instance, more than a year after participation in the grad cohort mentoring program, the women computing students were still pursuing computing PhDs (Stout et al., 2017). Two course-based interventions identified the goals of improving computing content; one of these course interventions also stated aims to increase access to computing and improve social support. Five out of seven interventions improving persistence focused on the college level (e.g., Lagesen, 2007; Staehr et al., 2001). Of the five interventions unsuccessful in improving computing persistence, three (60%) involved extracurricular activities at the high school level at one site for between 1 day and 3 years (Pivkina et al., 2009; Sullivan et al., 2015).
Diverse Contexts for Computing Interventions
One-quarter of the interventions in our sample took place outside the United States, several of which were discussed previously (n = 12). However, most were based in Western nations with similar gender equity challenges in computing, including Australia (n = 3), Canada (n = 2), Norway (n = 1), Scotland (n = 1), Sweden (n = 1), British Columbia (n = 1), New Zealand (n = 1), and Ireland (n = 1). One intervention studied took place in Hong Kong. The most common intervention strategy involved course design (n = 3), extracurricular activities (n = 2), and game design (n = 2). These interventions most commonly aimed to improve computing content to be more inviting to girls and women (n = 5), and they were most successful in improving motivational momentum (n = 5). Nine of the 12 interventions abroad successfully achieved their stated goals. Successful interventions abroad were most common at the university level (n = 5) but tended to last for a shorter period than U.S.-based interventions (2–5 days, n = 4).
Discussion
In the wake of the COVID-19 pandemic that disproportionately affected women (Croda & Grossbard, 2021), it is particularly important to attend to aspects of computing interventions that are successful in improving computing gender equity (see McDonald et al., 2006, on implementation and scale-up of educational interventions). This investigation is particularly timely following the adaptation of novel in-person and expanded digital interventions in education and during economic shifts in the academic and technology workforce and rising interest in artificial intelligence (AI).
Characteristics of computing interventions are reported in Tables 3–5. We found that most successful interventions were deployed at the college level and for at least 1 year. Meanwhile, most unsuccessful interventions were deployed at the K–12 level between 1 day and 1 year. These findings suggest it takes time to shift access to and the climate of computing; interventions may benefit from longer implementation such as a year or more. Irrespective of whether they succeeded, most computing interventions were not (at least not yet) robust, multisite interventions. Those studies that were more comprehensive also risked being less successful due to more diffuse goals, which can hinder efforts to identify scalable computing interventions. To identify generalizable aspects of computing interventions that improve gender equity, we closely examined the goals and outcomes of programs in our sample and explored how they aligned with our momentum for gender equity framework.
Interventions and Opportunities for Gender Equity in Computing by Momentum Domain
Teaching and Learning, Curricular, and Motivational Domains
Most interventions sought to improve the teaching and learning domain of computing momentum by altering computing content. These interventions mostly involved a course design intervention to make computing courses more accessible and inclusive to girls and women. These interventions often succeeded at this goal, measured by each aspect of computing momentum, including the teaching and learning, curricular, motivational, and social domains. However, interventions with the goal of improving teaching and learning momentum were unlikely to significantly alter the structural domain. Instead, interventions with this goal are most likely to successfully achieve persistence, performance, attitudes, and social outcomes. This finding highlights the classroom as a vital component in increasing gender equity in computing but suggests a more holistic approach is needed to change the landscape of computing.
Teaching and learning interventions can enhance gendered attitudes and perceptions toward computing (Beyer, 2014), connect previous experience with intellectual preferences (Rankin et al., 2019), and strengthen computing knowledge and skills, which may foster the outcomes discussed above (Code.org, CSTA, & ECEP Alliance, 2022). This aligns with previous research that identifies removing obstacles at introductory or entry-level courses as a strategy to maximize the effectiveness of interventions and build a solid ground for long-term involvement with computing (Butler et al., 2017).
These courses revamp teaching activities to bolster students’ interests, reinforce knowledge about computing, and improve confidence. While improving computing content to make it more interesting and inclusive to girls and women is a key avenue to building skills and shaping attitudes toward computing, it does not fundamentally shift the environment of computing to attract them to the field or to be more equitable to girls and women. The quality of women’s interactions with computing course material and departmental cultures can shape their commitment and sense of belonging (Pantic & Clarke-Midura, 2019).
Indeed, research indicates chilly climates persist in computing courses (Vitores & Gil-Juárez, 2016) and suggests structural and cultural changes are needed (Cheryan et al., 2015). There was considerable attention to teaching and learning among interventions in general and those successful at meeting their aims specifically (e.g., Robertson, 2013; Rover et al., 2016); fewer interventions attended to curriculum and motivation. Among interventions aimed at motivational change, we found that those that were most successful were based on extracurricular activities and often had connections to structural and/or social change, even if not explicitly stated as a goal of the intervention (e.g., Akkuş Çakır et al., 2017; Staehr et al., 2001).
Social Domain
The smallest share of interventions sought to provide social support for girls and women in computing spaces. While fewer interventions had this goal, they successfully improved the social domain over 70% of the time, in addition to positively affecting curricular outcomes and persistence. Improving social support in computing spaces is a crucial step toward warming the chilly climate for girls and women in computing. Computing environments are an ongoing challenge for girls’ and women’s participation and persistence in the field (Frieze & Quesenberry, 2013). Moreover, Berenson et al. (2004) asserted that socially isolated work in computing discourages persistence among girls and women.
Collaboration and Peers
Collaborative working and learning environments may invite diverse perspectives, provide emotional support and validation, and increase access and retention to the coding and programming course activities for girls and women, who tend to be more heavily influenced by the social element of STEM fields (Bergsieker et al., 2020; Pantic & Clarke-Midura, 2023). This highlights the need to address the social domain of momentum in future research. Pantic and Clarke-Midura (2023) found that peer support can enhance women’s academic success in computer science via a greater sense of belonging, respect, and validation; in turn, this can enhance girls’ engagement and persistence in the field (Kim et al., 2018). Comparatively limited interest in computing fields continues to be a primary explanatory factor for women’s underrepresentation in computing fields and careers (Akkuş Çakır et al., 2017; Beyer, 2014; Kerger et al., 2011; Margolis et al., 2000). When girls are formally and informally excluded from K–12 classroom activities and positive reinforcements, their interest in STEM fields (including computing) declines (Leedy et al., 2003). Accordingly, structural opportunities appear critical to enhancing gender equity in computing.
Institutional Contexts
Mentoring networks with successful women in computing as role models, guest speakers, mentors, and alumni could strengthen girls’ computing beliefs, attitudes, and interests, and their intentions to major in computing and persist in the computing occupational field. Mentorship models are widely applied by institutions, ranging from K–12 schools, undergraduates, and graduate groups, such as the Grad Cohort Workshop (Stout et al., 2017). Explicitly identifying social support as a goal could encourage those implementing gender equity interventions to evaluate and center this aim to shift the culture of computing spaces and more adequately measure social outcomes.
Structural Domain
Less than half of the intervention studies aimed to alter the structural domain of computing momentum by increasing structural access to computing for girls and/or women. In turn, they were successful in about half of their attempts. Given disparities in access to computing opportunities, a need remains for educational interventions in settings that are intentionally diverse in their enrollments, such as rural and lower-income schools and communities, vocational and community colleges, and postsecondary institutions that serve underrepresented students (see Singh et al., 2007; Yamaguchi & Burge, 2019).
Indeed, structural access is vital to improving gender parity in computing. Girls and women tend to have less hands-on experience with computing during their schooling and fewer opportunities to gain that experience (Margolis & Fisher, 2002). Pinkard et al. (2017) concluded that for urban middle school girls who consume and participate in but are typically marginalized in digital media, multiple and visible pathways to see their identities represented are important. Put simply, we cannot improve gender parity in computing if opportunities to engage with the field are not equitable (Michie & Nelson, 2006). Computing interventions we reviewed tended not to focus on structural change to improve gender equity for girls and women in computing. Importantly, institutional and organizational investment in structural and organizational change is vital to the success of these interventions (Rosser, 2018).
Limitations and Future Directions: Intersectionality and More Diverse Contexts
In a prior RER-published systematic review focused on women and gender equity in computing fields, Singh et al. (2007) lamented the too-often monolithic approach to gender in this line of research. Our systematic literature review of interventions to enhance gender equity in computing focused on a later time window: 2000 through 2020. Despite our inclusive, international, and interdisciplinary search parameters, we too found few studies that looked beyond gender equity alone compared to gender in relation to additional and potentially compounding marginalized identities (see, e.g., Nesiba et al., 2015; Scott & White, 2013; Scott et al., 2017). Fewer considered gender beyond binary notions of male/female, despite mounting evidence of the theoretical and empirical limitations for binary, mutually exclusive definitions of gender (Butler, 2011; Hyde et al., 2019).
Our initial collection of articles intentionally searched for and included intersectional studies and those from non-U.S. locations, but the conventions of a systematic literature review curtailed some structurally innovative pieces—those not explicit about their methodology or their attention to gender and not published in English or in peer-reviewed outlets. Despite our efforts to cast a wide and inclusive net in the literature search, the structure of systematic reviews may silence voices of and studies about gender equity beyond binary gender differences (see also Perez-Felkner, et al., 2024). We encourage readers to engage and publish intersectional research on interventions that include evaluation in their designs to enhance our knowledge base on gender equity in computing and other fields with a history of exclusion.
Conclusion
Computing interventions report meaningful goals and commitments to close the gender gap in the field. However, the effectiveness of interventions is arduous to evaluate, and discerning the degree to which interventions are scalable is unknown since programs vary widely in scope, duration, and population focus. We sought to identify and review the quality of existing interventions for broadening participation and gender equity in computing fields by synthesizing interdisciplinary peer-reviewed evidence and the outcomes of each study. We applied a momentum framework to categorize gender equity interventions. While some interventions aimed to address multiple domains of computing momentum, none successfully improved them all, and in some cases, the breadth of the programs’ aims hindered their success. Successful interventions involved institutional aims and commitments, curricula tailored for girls, and networks of social support. To fundamentally shift the culture of computing education to be more inclusive to girls and women, interventions matter.
Footnotes
Appendix
Citation information for included computing intervention studies for gender equity (N = 48)
| Authors | Title | Publication outlet |
|---|---|---|
| Staehr et al., 2001* | An Intervention Programme for Women in Computing Courses: Does it Make a Difference? | Computers and Networks in the Age of Globalization |
| Gabbert & Meeker, 2002* | Support Communities for Women in Computing | ACM SIGCSE Bulletin |
| Salminen-Karlsson, 2002* | Gender-Inclusive Computer Engineering Education: Two Attempts at Curriculum Change | International Journal of Engineering Education |
| Crews & Butterfield, 2003 | Gender Differences in Beginning Programming: An Empirical Study on Improving Performance Parity | Campus-Wide Information Systems |
| Grant, 2003 | A Study on Critical Thinking, Cognitive Learning Style, and Gender in Various Information Science Programming Classes | Proceedings of the 4th Conference on Information Technology Curriculum |
| Jenson et al., 2003 | “Girl Talk”: Gender, Equity, and Identity Discourses in a School-Based Computer Culture | Women’s Studies International Forum |
| Lynn et al., 2003* | Bridging the Gender Gap in Computing: An Integrative Approach to Content Design for Girls | Journal of Educational Computing Research |
| Werner et al., 2004 | Pair-Programming Helps Female Computer Science Students | Journal on Educational Resources in Computing |
| Denner et al., 2005 | The Girls Creating Games Program: Strategies for Engaging Middle-School Girls in Information Technology | Frontiers - A Journal of Women's Studies |
| Doerschuk et al., 2007 | Pilot Summer Camps in Computing for Middle School Girls: From Organization Through Assessment | ACM SIGCSE Bulletin |
| Lagesen, 2007* | The Strength of Numbers: Strategies to Include Women into Computer Science | Social Studies of Science |
| Logan, 2007 | Should Computing Be Taught in Single-Sex Environments? An Analysis of the Computing Learning Environment of Upper Secondary Students | Educational Studies |
| Mento et al., 2008 | Encouraging Women and Minorities to Attain Degrees in Computing and Related Fields | Information Systems Education Journal |
| Lau et al., 2009* | Learning Programming Through Fashion and Design: A Pilot Summer Course in Wearable Computing for Middle School Students | ACM SIGCSE Bulletin |
| Pivkina et al., 2009* | Young Women in Computing: Lessons Learned from An Educational & Outreach Program | ACM SIGCSE Bulletin |
| Tsoukalas & Wu, 2009* | WICS @ SFU: Assessing the Impact and Outcomes of a Women in Computing Science Student Group at the College Level | Proceedings of the 14th Western Canadian Conference on Computing Education |
| Werner & Denning, 2009 | Pair Programming in Middle School: What Does It Look Like? | Journal of Research on Technology in Education |
| Carbonaro et al., 2010 | Computer-Game Construction: A Gender-Neutral Attractor to Computing Science | Computers & Education |
| Craig & Coldwell, 2010 | An Initiative to Address the Gender Imbalance in Tertiary IT Studies | Interdisciplinary Journal of Information, Knowledge, and Management |
| Andujar et al., 2012 | Attracting High School Students to Computing: A Case Study with Drag-Drop Interfaces | EdMedia + Innovate Learning Conference |
| Denner et al., 2012* | Computer Games Created by Middle School Girls: Can They Be Used to Measure Understanding of Computer Science Concepts? | Computers & Education |
| Cohoon. et al., 2013 | Educating Diverse Computing Students at the University of Virginia | IEEE: Computer |
| Robertson, 2013* | The Influence of a Game-Making Project on Male and Female Learners’ Attitudes to Computing | Computer Science Education |
| Scott & White, 2013* | COMPUGIRLS' Standpoint: Culturally Responsive Computing and Its Effect on Girls of Color | Urban Education |
| Alvarado & Judson, 2014* | Using Targeted Conferences to Recruit Women into Computer Science | Association for Computing Machinery |
| Freeman et al., 2014 | Engaging Underrepresented Groups in High School Introductory Computing Through Computational Remixing with EarSketch | Proceedings of the 45th ACM Technical Symposium on Computer Science Education |
| Outlay et al., 2014 | Getting IT Together: Linking Computing Intervention Camps with Computing Careers | Proceedings of the 52nd ACM Conference on Computers and People Research |
| Scott. et al., 2015 | Culturally Responsive Computing: A Theory Revisited | Learning, Media, and Technology |
| Friend, 2015* | Middle School Girls' Envisioned Future in Computing | Computer Science Education |
| Lang, 2015 | Outreach Programmes to Attract Girls into Computing: How the Best Laid Plans Can Sometimes Fail | Computer Science Education |
| Nesiba et al., 2015* | Young Women in Computing: Creating a Successful and Sustainable Pipeline | 2015 IEEE Frontiers in Education Conference |
| Richard et al., 2015* | StitchFest: Diversifying a College Hackathon to Broaden Participation and Perceptions in Computing | Proceedings of the 46th ACM Technical Symposium on Computer Science Education |
| Sullivan et al., 2015* | CodePlus–Designing an After-School Computing Programme for Girls | 2015 IEEE Frontiers in Education Conference |
| Magerko et al., 2016* | EarSketch: A STEAM-Based Approach for Underrepresented Populations in High School Computer Science Education | ACM Transactions on Computing Education |
| Robinson et al., 2016 | African American Middle School Girls: Influences on Attitudes Toward Computer Science | Computing in Science and Engineering |
| Rover et al., 2016* | Evidence-Based Planning to Broaden the Participation of Women in Electrical and Computer Engineering | 2016 IEEE Frontiers in Education Conference (FIE) |
| Townsend & Sloan, 2016 | Pre- to Post-Conference Differences: Celebrations of Women in Computing | 2016 IEEE Frontiers in Education Conference |
| Witherspoon et al., 2016* | Gender, Interest, and Prior Experience Shape Opportunities to Learn Programming in Robotics Competitions | International Journal of STEM Education |
| Akkuş Çakır et al., 2017* | Development of a Game-Design Workshop to Promote Young Girls' Interest Towards Computing Through Identity | Computers & Education |
| Brady et al., 2017 | All Roads Lead to Computing: Making, Participatory Simulations, and Social Computing as Pathways to Computer Science | IEEE Transactions on Education |
| Butler et al., 2017* | Pencil Puzzles for Introductory Computer Science: An Experience- and Gender-Neutral Context | Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education |
| Khan & Wei, 2017 | Free Talk Zone: Inclusive Pedagogy to Encourage Women in Computer Science | 2017 International Conference on Computational Science and Computational Intelligence |
| Medel & Pournaghshband, 2017* | Eliminating Gender Bias in Computer Science Education Materials | Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education |
| Ortega et al., 2017* | Towards a 3D Virtual Programming Language to Increase the Number of Women in Computer Science Education | 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual & Augmented Reality (KELVAR) |
| Scott et al., 2017* | Broadening Participation in Computing: Examining Experiences of Girls of Color | Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education |
| Stout et al., 2017* | The Grad Cohort Workshop: Evaluating an Intervention to Retain Women Graduate Students in Computing | Frontiers in Psychology |
| Rheingans et al., 2018 | A Model for Increasing Gender Diversity in Technology | Proceedings of the 49th ACM Technical Symposium on Computer Science Education |
| Hollman et al., 2019* | Information Technology Pathways in Education: Interventions with Middle School Students | Computers & Education |
Notes
Authors
LARA PEREZ-FELKNER is a professor of higher education and sociology at Florida State University, P.O. Box 3064452, Tallahassee, FL 32306-4452; e-mail:
KRISTEN ERICHSEN is a senior data scientist at Knowli Data Science, 3551 Blairstone Road Suite 105-102, Tallahassee, FL 32301; e-mail:
YANG LI is an academic assessment professional, focusing on evaluating student engagement and post-graduation outcomes, at Florida State University; e-mail:
JINJUSHANG CHEN is a postdoctoral faculty member at Beijing Normal University, Beijing, China; e-mail:
SHOUPING HU is the Louis W. and Elizabeth N. Bender Endowed Professor of Higher Education at Florida State University; e-mail:
LADANYA RAMIREZ SURMEIER is an assistant teaching professor in the Public Health Program at Florida State University; e-mail:
CHELSEA SHORE is an independent researcher working with the SAFE Project; e-mail: