Abstract
Home Language Surveys (HLS) are widely used by states as an initial screening tool to determine whether students should receive English as a Second Language (ESL) services. Parents/guardians are asked to complete the surveys when enrolling a student into a school. We collected surveys from the 50 states and the District of Columbia. We completed them as if we were the parent/guardian of six use-case students. Research questions were (1) How do HLS vary from state to state, if at all? What kinds of questions do they ask? (2) How do HLS across the 50 states and the District of Columbia identify, or not, six fictitious students for further ESL screening? We found that states and U.S. Department of Education–approved HLS questions identified students differently due to unclear questions, such as asking bilingual families to name one dominant language. We recommend additional validation measures be taken with survey questions.
Keywords
Home Language Surveys (HLS) are used in all 50 states and the District of Columbia as an initial gateway to K–12 English as a Second Language (ESL) services. Though each state may issue guidelines for how surveys are administered, procedures generally include that parents/guardians of new students complete the survey, and school officials then decide, based on responses, whether to further screen students. This decision is pivotal for students and families, placing students either on pathways toward or away from language testing that could officially designate them as English Learners (ELs). This decision likely has eventual ramifications for most aspects of students’ school experiences, including the types of classrooms where students are placed, the curriculum they learn, the teachers they are assigned, the peers they interact with, and the assessments they undergo. Yet HLS have long been criticized for their inconsistency across states and for misalignment with principles of quality survey design (Bailey & Kelly, 2013) and multilingualism (García & Kleifgen, 2018).
As language teacher educators, we have often heard of teachers’ concerns about HLS. Over the years, they have told us about students they believed to have been misidentified due to unclear HLS questions. In this exploratory study, we set out to examine how HLS across states identify (or not) students in various bi/multilingual contexts. Our purpose is to reveal the extent of differences across states’ HLS in identifying (or not) hypothetical use-case students for further ESL screening, with an eye toward considering validity aspects of HLS, as the initial screening tool for ESL services. Based on our experiences as teacher educators and our conversations with teachers over the years, we created six fictitious students (see cases in Method). We gathered HLS from all 50 states, plus the District of Columbia, and answered them as if we were family members of our case students in examining our research questions:
Literature Review
Diversity of Multilingual Students
Undergirding our work is the idea that multilingual students in the United States are incredibly diverse. These students bring to U.S. schools a richness of linguistic wealth. Students who officially receive ESL services are a small subgroup of the many students with bi/multilingual resources. We call students who are identified for ESL services ELs because federal policy uses that term (Every Student Succeeds Act [ESSA], 2015). We recognize the label has problems in naming students in relation only to English and perpetually casting them as learners. Wright (2019) discussed the tremendous diversity among students identified for ESL in that they “vary widely by race, ethnicity, home language, level of schooling, socioeconomic status, parents’ level of education, parents’ proficiency in English, proficiency and literacy in their home language, and proficiency in English” (p. 2). Scholars have warned of dangers of viewing ELs as a monolithic group. Lacelle-Peterson and Rivera (1994) pointed out that ESL-identified students include children from Native American, Amish, Franco-American, Chicano, and immigrant communities: Indeed, while language represents an important, educationally significant variable . . ., it is only one of many educationally relevant characteristics of any individual English language learner, whose whole identity, including cultural heritage, ethnic group affiliation, gender, and individual learning differences, must be taken into consideration in educational decisions. (pp. 59–60)
More recently, Umansky and Porter (2020) discussed the diversity of EL-classified students and implications that policies must be flexible to meet various subgroups’ needs. We recognize that given such diversity among bi/multilingual students and their wide-ranging educational priorities, crafting an HLS that more precisely identifies students who would benefit from ESL services is no small task.
Previous HLS Research
HLS criticism extends back some time, with researchers arguing that HLS questions, administration, and results are inconsistent across and within states. Twenty years ago, Littlejohn (1998) questioned Office for Civil Rights (OCR) policies requiring that any mention of a language besides English on the HLS meant a student would be tested. Littlejohn gave the specific example that a student who mostly spoke English but who had a Spanish-speaking relative stay in the home (similar to our student Raul, see below) would be tested. Additionally, Abedi (2008) casts doubt on HLS by comparing questionnaire responses from 1,031 eighth graders to students’ official designations and finding discrepancies. Goldenberg and Quach (2010) found that Arizona’s then use of a single HLS question resulted in underidentification of ELs.
More recently, Bailey and colleagues have published multiple HLS studies. Bailey and Carroll (2015) criticized U.S. ESL assessment practices, including the initial HLS. They described HLS administration specifically as a “ubiquitous yet nonmandated and widely varying practice” (p. 254). They identified two HLS purposes: to focus on students most likely needing ESL services and to yield information about students’ language backgrounds to help schools programmatically and instructionally. They concluded, however, that the HLS is a poor substitute for other instruments, such as interviews, observations, assessments, or universal screening.
Bailey and Carroll (2015) called for research on how tight an initial identification net HLS should cast. A too-wide net means students will be overidentified for testing, but with a too-narrow net, students needing support might be missed (see also Bailey & Kelly, 2013). Bailey and Carroll (2015) suggested administrators err on the side of overidentifying students because subsequent testing should prevent misidentification of students not needing services. Bailey and Kelly (2013) explained that HLS determine whether various student populations would be tested. These populations include students who know varying degrees of English (ranging from none to a lot), students who are balanced bilingual speakers of English and another language (though see Shin, 2018, for criticism of the mythical ideal of balanced bilingualism), and students who speak only English but have parents or relatives who speak other languages. Poor construction of HLS items, they noted, can lead to inaccurate identification; poor constructions might include ambiguous wording or HLS with too few items not focused on a student’s “current language dominance and degree of English exposure” (p. 792) and that overlook factors, such as English preschool attendance. Ultimately, Bailey and Kelly (2013) recommended the federal government provide greater guidance to states and increase transparency in operationalizing HLS. And they suggested that states provide clear guidance to divisions and adopt validation plans.
Linquanti and Bailey (2014) presented results from Council of Chief State School Officers representatives concerning HLS. The group suggested renaming HLS to Language Use Survey because the federal definition of language use in relation to identifying ELs extends beyond the home. We agree with this renaming, but we use HLS because it is most commonly used by states. Linquanti and Bailey (2014) considered students’ current language(s) and frequency of use and exposure to English as essential constructs for HLS, while languages spoken among adults in the home and students’ first language (L1) spoken were considered less important (Linquanti et al., 2016; Lopez et al., 2016). They, further, raised several issues about HLS. For example, they argued that focusing questions on a student’s exposure to and use of English should avoid overidentification of bilinguals. Asking questions about language used with the child, rather than among others in the home, would provide more accurate information. And emphasizing questions about current languages use, rather than first languages, might prevent parents’ from inaccurately reporting an L1 out of cultural pride. Linquanti and Bailey (2014) further recommended use of hypothetical student profiles to evaluate HLS. Additionally, Haas et al. (2015)—one of few studies of a state HLS and how students eventually perform on the English proficiency test—found asking families about all of a student’s languages, rather than focusing on a dominant language, helped in not overidentifying fluent bi/multilinguals.
Other sources also commented on accuracy of parents’ responses. Kim et al. (2018) found that the top reason educators gave for misidentification of ELs was families’ inaccurate HLS responses. The researchers suggested better communication with families and improving HLS. Byers-Heinlein et al. (2019), describing an interview protocol for asking parents about infants’ language exposure, argued that questions must be carefully worded. Parents, for example, can have difficulty answering questions about percentages of a child’s language exposure because they might not think in terms of percentages, they might think differently than researchers, or their responses might be biased. Bailey (2016), after conducting focus groups, reported that parents saw limitations in how HLS focused on language in the home, rather than including digital language exposure and language use in children’s lives outside their homes. Families also raised questions about HLS expecting parents to use the same language with each other as with children. García and Kleifgen (2018) argued that questions worded with a “monolingual bias” are difficult for multilingual families (p. 13). Additionally, Abedi (2008) reported that parents might give inconsistent information on HLS for various reasons beyond question comprehension, including concerns about citizenship or opportunities for their children.
Some families—though it is difficult to know how many—have reported providing misinformation on HLS to avoid EL labels they view as harmful for their children (Taxin, 2014), related to a stigma of being viewed as not proficient in English (Monzó & Rueda, 2009). As with other labels, EL designation might have both positive and negative outcomes for students (Umansky & Dumont, 2019), and some have argued that EL labeling is more permanent in practice than policy might intend (Okhremtchouk et al., 2018).
HLS Policy
Use of some sort of HLS extends back 80 (now almost 90) years, according to Bailey and Kelley (2013), who detail HLS history from those early beginnings with the Hoffman Schedule of Bilingual Background (Hoffman, 1934) through No Child Left Behind. We seek not to repeat their historical work but to provide updates under ESSA.
The U.S. Department of Education (USDOE; USDOE & Office of English Language Acquisition, 2017) issued an “English Learner Toolkit” to guide states in providing ESL services. The toolkit defined the HLS as: a questionnaire given to parents or guardians that helps schools and [local education agencies] LEAs identify which students are potential ELs and who will require assessment of their English language proficiency (ELP) to determine whether they are eligible for language assistance services. (USDOE & Office of English Language Acquisition, 2017, p. 1)
Though ESSA (2015) did not specifically require HLS, it stipulated that states must have consistent procedures to identify and assess potential ELs within 30 days of enrollment. Some states allow divisions to make changes to state HLS questions, and others do not (see findings below). The toolkit is a companion piece to the U.S. Department of Education Office for Civil Rights & U.S. Department of Justice Civil Rights Division’s (USDOEOCR & DOJCRD, 2015) joint guidance, which specifies that schools must identify primary or home language other than English (PHLOTE) students for further screening through either “an HLS or other means” (p. 10). Though the term PHLOTE is used in federal guidance, we view it as problematic in many ways, including that it presents students as having only one primary/home language; describes that language as other; and includes primary/home descriptors, which can be difficult to define and not universally applicable.
The toolkit listed three items on a quality checklist of HLS content:
Is the purpose and use of the HLS clearly communicated to both families and those who administer the survey?
Does the HLS elicit information about the student’s current English abilities?
Are the questions clear and understandable to those who administer the HLS? (USDOEOCR & DOJCRD, 2015, p. 2)
The toolkit suggested three OCR- and DOJ-approved questions:
What is the primary language used in the home, regardless of the language spoken by the student?
What is the language most often spoken by the student?
What is the language that the student first acquired? (USDOEOCR & DOJCRD, 2015, p. 4)
The toolkit explained that asking these questions and testing a student whose family mentions a language other than English on any of the three questions would be minimally compliant with federal law. The toolkit suggested that HLS cast a wide net so potential ELs are not missed, and the USDOEOCR and DOJCRD (2015) letter suggested that one example of noncompliance occurs when schools “use a method of identification, such as an inadequate HLS, that fails to identify significant numbers of potential EL students” (p. 11). The letter argues that screening and identifying students is a critical step in serving ELs, delineating that divisions must have an accurate and timely process for deciding which students should be tested. Our aim is to help examine how states’ HLS might be adequate, or not, in screening students whose complex language backgrounds might be difficult to capture through HLS.
Method
Researchers’ Positionalities
We are language teacher educators at institutions in different U.S. regions. April manages an ESL teacher education program at a large Southern public university, while Elena manages a similar program at a metropolitan, Midwestern public university. We previously taught ELs, and we consider ourselves multilinguals. We also recently completed HLS for our own children as they entered public schools, so it was impossible to approach this work without at times seeing it as parents. At the time of analysis, April’s daughter was in a first-grade dual-language English-Spanish program, and Elena’s son was in a kindergarten English-only classroom. As we read all 50 states’ HLS and applied questions to our fictitious case-study students, we could not help but notice that our children would have been tested in some states but not in others because of differences in question wording. Though we believe in the importance of community voices in schools, we viewed this difference as inconsistent for people moving across state lines.
This project grew from a larger study in which we as practitioner-researchers (Cochran-Smith & Lytle, 2009) analyzed how teachers in April’s ESL assessment course discussed services they would provide to families responding to HLS. As we systematically analyzed discussions from seven course iterations, we noticed the stories teachers told about students whose families could not easily answer HLS. We grew increasingly interested in how HLS questions might or might not fit diverse bi/multilingual students. With this in mind, we developed our use cases. We first drafted four cases based loosely on teachers’ comments and our knowledge of bi/multilingual students who might be difficult to classify. Our intent was to stretch HLS and to reveal inconsistencies in identification for these cases. In contrast, we added two cases for whom we thought HLS questions might more easily fit and for whom we expected HLS would yield more consistent results.
Gathering HLS
To gather HLS questions, we searched the internet for HLS published by each state’s department of education during spring 2020. 1 It is possible that some states have updated HLS since then. We acknowledge that some states allow individual divisions to develop their own HLS questions; however, we decided to gather state-level HLS to paint an overall picture of what HLS look like across the United States. It is difficult in a national survey to capture the specificity of division-level differences. When states allowed individual divisions to add survey items, we used only state-required questions. Furthermore, a limitation of our study is that we examined only online published materials when available. States not keeping their websites updated might have had more recent HLS. Keeping an updated log of HLS as states change policies is beyond our scope; instead, this study used the HLS available at the time of survey collection. One state—Oregon—was undergoing HLS revision and had two published versions; we coded both. When HLS were unavailable online from the state department of education, we used the state’s HLS questions published by WIDA, a consortium of 40 states, territories, and agencies that provides assessments for schools to identify and serve ELs. In the few cases where we could not find HLS through state or WIDA websites, we contacted state departments of education and requested copies (i.e., Iowa and New Hampshire). For one state—South Carolina—we verified HLS questions by contacting division-level personnel.
Once we gathered HLS, we placed questions by state in a spreadsheet. We stored original copies of HLS questions and, when available, guidance issued from states on how schools should administer and interpret HLS. Importantly, we examined questions only, not how they are administered, though we believe the issue of how surveys are administered can also affect survey results. Some states require that staff assist families in completing HLS, while others include it in a larger registration packet that families complete on their own.
Responding to HLS
Our coding involved responding to all HLS as if we were our case students’ parents/guardians. HLS sometimes included questions schools use to garner additional information from families (e.g., Arkansas: “In what language do you prefer to receive written communication from the school?”). However, we did not code questions that states said did not count for screening purposes or questions that did not ask about language when mention of a language other than English was the determining factor for screening. If no mention was made of whether a question was used for screening and it asked about language, we assumed it was and coded it. Additionally, when states offered HLS translated into various languages, we only coded English versions. Although we view the availability and accuracy of translation as critical for HLS, studying translated versions was outside our scope and is work we leave to future researchers.
Before responding to questions, we first checked which states were using the toolkit’s three recommended questions (USDOE, 2017). We marked each state in our spreadsheet as (1) using the USDOE questions, (2) using a reworded version of the USDOE questions, or (3) using different questions. Our “rewording” category included states that changed words in ways we thought altered questions’ meanings and consequently might affect students’ screening outcomes (e.g., Reworded Version: deleting the phrase “regardless . . .” from the question “What is the primary language used in the home, regardless of the language spoken by the student?” as in the District of Columbia.)
If a state changed wording of USDOE questions in cosmetic ways that we did not expect to affect a question’s outcome, we categorized the state as using the USDOE’s questions (e.g., changing the USDOE question “What is the language most often spoken by the student?” to “What is the language most often spoken by your child?” [italics added]). For states that used the USDOE questions, we only coded questions once. For remaining states, we coded each HLS. We did all coding together in live, synchronous meetings where we discussed each question in relation to the cases we were coding, ensuring that we agreed on each response. In this way, we achieved interrater agreement, or “the absolute consensus in scores furnished by multiple judges for one or more targets” (LeBreton & Senter, 2008, p. 816). For each case in each state, once we arrived at a decision of whether the student would be identified for screening, we moved on to the next HLS. If we answered all questions and based on those responses in relation to the state guidance were unable to ascertain whether the child would be screened, we marked the case as “unclear” for that state. For example, in a state where the screening threshold was a mention of a language other than English, a case would be marked “unclear” if we could not determine whether a family would answer all questions in English or would mention an additional language. That uncertainty was a signal for us that family members could reasonably answer the questions either way. To double-check our coding, we independently coded five randomly selected states. We attained 96% interrater reliability, and we resolved the one disagreement as an error, rather than an actual disagreement.
As we began coding answers to states’ HLS questions, we continuously refined and clarified our cases and checked back with previous codings to ensure alignment with updated cases. We did this because we wanted to ensure that questions we answered as “uncertain” were uncertain not because our cases were underdeveloped (or not as fully developed as a parent’s understanding of a real child would be) but because questions themselves were unclear in the context of the child’s language use. We also ensured that our responses across states were consistent by adding information to the cases whenever required.
Use Cases
Through this iterative process, we developed our first four use cases:
Stefan
Stefan is registering for kindergarten. He was born and has lived his entire life in the United States. One of his parents was born in the United States and speaks English as an L1. Another parent, who was born in Bulgaria and immigrated to the United States 10 years ago, speaks Bulgarian as an L1 and English as an L2. Each parent has spoken to Stefan in their L1 since Stefan’s birth, and Stefan speaks to each parent in that parent’s L1. Stefan has attended 2 years of English-speaking day care and 3 years of English-speaking private preschool. Stefan has no siblings.
Raul
Raul is also registering for kindergarten. Both Raul and his parents were born and have lived their entire lives in the United States. Raul’s parents have relatives in Colombia and consider Spanish to be their heritage language. Though they grew up speaking English, they learned Spanish in school and from using Spanish with relatives. They only speak in English to Raul, and they consider Raul to be an only-English speaker. Raul attended 2 years of private English-speaking preschool. An uncle of one of Raul’s parents visited the family from Colombia and stayed for 8 months. Raul’s parents completed the HLS during this time and cannot say which language is most frequent in the home, though they continued to use English with Raul. Raul learned a few Spanish phrases that he used to greet his great-uncle and/or to ask for household objects. Raul has no siblings.
Sarah
Sarah is registering for eighth grade after her family moved to a new state. Sarah speaks English at home. She was adopted from China at age 1. She was learning Mandarin until her adoption. Both her adoptive parents speak only English. They have enrolled her in a Chinese community weekend school since age 5 to give her connections to Chinese culture and language, and she learned Mandarin through the program. Sarah has not received ESL services previously. She has an adopted brother from Korea who has attended Korean-language school; they speak English to each other.
Petr
Petr is a high school exchange student from Germany, registering for ninth grade, where he studied English at school. He speaks German with his family, though his parents also speak English. He speaks English with his U.S. host parents. Petr speaks German to his biological siblings. In the United States, he has a host brother who is learning German at school. The boys speak English and German to each other. Though it is difficult for individuals to estimate the exact proportion of each language used, English is the dominant language in the host family. Petr’s exchange agency maintains that he passed an English proficiency exam before exchange-program admission.
We subsequently added two cases who we thought might be able to more consistently answer HLS questions:
Eva
Eva is registering for kindergarten. She was born in the United States and has lived her whole life in the United States. Both her parents speak Spanish and are from Mexico. They speak Spanish exclusively at home. Eva has two older siblings who speak English at school and often to each other and to Eva. Eva speaks English with her siblings and mostly Spanish the rest of time. Spanish is the clear primary language in the home and is Eva’s dominant language and L1. Eva has not attended preschool or day care.
Valentin
Valentin is registering for eighth grade. He was born in Honduras and attended school in Spanish through seventh grade. His dominant language and L1 is Spanish. He took English classes at his school in Honduras. His parents and siblings at home speak Spanish to each other and to Valentin.
Findings
In this section, we first present our results generally about HLS across states, and then we provide case-by-case findings.
HLS Overview
It is important to remember that next steps after the HLS vary by state, due to a number of differences across states (see Table 1). In all states, the HLS is an initial—not a sole—step toward EL identification. For most states, the next step is English proficiency testing, but 14 states specified at least one additional required intermediary step, such as a parent interview or academic records review, before testing (see Table 2). Additional states, such as Alaska and Ohio, described in their guidance to divisions, intermediary steps that were not required. HLS results are only final for students not identified for further screening, as they will not be identified as ELs. Some states did, however, provide guidelines for teachers to recommend further screening students not previously identified or to override parent/guardian’s HLS responses if they believed answers were inaccurate (e.g., Maine, South Dakota, and Wyoming).
Information About State HLS
Note. HLS = Home Language Surveys; USDOE = U.S. Department of Education; LOTE = language other than English.
We only counted the questions used to screen students for ESL services. bMississippi has three different sets of recommended questions. We used the first set published in their guidance. cOregon Bridge has inconsistency in whether it is a mention of a LOTE or the absence of mentioning English that moves the student toward screening. We used the mention of LOTE in our coding because at least one question said to do so in two places. But we suspect that HLS administrators might be confused and apply different screening indicators.
States That Specified Mandatory Intermediary Steps Before Testing/Assessment
Note. ESL = English as a second language; HLS = Home Language Surveys; EL = English learner.
Of the 52 HLS, we found that six states—Alaska, Connecticut, Oklahoma, Rhode Island, Virginia, and West Virginia—used questions either exactly as the USDOE (2017) suggested or with minor rewording that would not affect our results (e.g., substituting “your child” for “your student”; see Table 1). We found that 12 states (including the District of Columbia and one of Oregon’s two HLS) had questions that appeared based on the USDOE-recommended questions but with rewordings we thought could change outcomes. In addition to the rewording mentioned above (i.e., asking about the language spoken in the home but omitting the phrase “regardless of the language spoken by the student”), we found one other reoccurring rewording: adding -(s) after “language.” For example, some states added -(s) to language/language(s) as an optional plural (e.g., South Carolina changed “What is the language [italics added] most often spoken by the student?” to “What language(s) [italics added] is most often spoken by the student?”). Notably, although these questions attempted to include bi/multilingual students, the grammatical structure often remained unchanged so that the plural “languages” did not agree with the verb “is.”
Most HLS (n = 34, including Oregon’s other HLS version) asked questions substantially different from USDOE’s questions. These surveys ranged in length from two questions (i.e., Illinois, Michigan, and Texas) to nine questions (i.e., Iowa and Wisconsin). HLS used widely varying question formats:
Yes/No questions (e.g., Alabama/Mississippi: “Is a language other than English spoken at home?”)
Agree/Disagree statements (e.g., Nevada: “The primary language spoken in the home of the pupil is not English”)
Multiple-choice questions (e.g., Minnesota: “My student understands: ○ language(s) other than English. ○ English and language(s) other than English ○ only English.”)
Open-ended or listing questions (e.g., North Dakota: “List other language(s) that your child has used with a grandparent or caretaker: ___________.”)
Colorado included alternative question versions within the same HLS (e.g., Divisions could ask either “What is/was the student’s first language?” or “What is the native language of the student?”). Some states asked about specific linguistic aspects. For example, New York and New Mexico asked about students’ proficiency in different language skills (e.g., New York: “What language(s) does your child understand? . . . speak? . . . read? . . . write?”). And Delaware asked about languages and dialects (e.g., “What language did your child first learn? Language:__________ Dialect:___________”).
Though detailed analysis of the language of questions was not our primary goal, we conducted one round of exploratory coding to consider questions’ predominant themes. We did this by grouping all HLS questions into inductive, emergent themes and adding themes to our coding list as we found new ones (Miles et al., 2020). Overwhelmingly, HLS asked about students’ first or native language(s), the presence of language(s) in the home, and the language(s) the child uses (see Table 3 for a full list of themes). Yet wording of these questions varied considerably (see Table 4 for a sampling of questions about a student’s L1).
Question Themes
Note. HLS = Home Language Surveys.
Sampling of Questions About Students’ L1s
Note. USDOE = U.S. Department of Education; L1 = first language.
We found that subtle wording differences can matter considerably. Our findings revealed that HLS results varied state-to-state for each of the four initial use case students, though more pronouncedly for Stefan and Raul than Sarah and Petr. Stefan and Raul each had 27 states (more than half) with unclear results, meaning that it was unclear how families would answer questions and therefore it was unclear whether the student would be screened further. This uncertainty happened because of questions that we believed reasonable adults would have difficulty answering, in relation to student use cases whose situations are complex and are not easily captured by the questions. For example, we deemed it impossible for the parent/guardian of a bi/multilingual child to identify which language the child spoke most often. Stefan’s parents, when asked to name one primary language, might list either English or Bulgarian. Or Raul’s parents might be unable to say which language Raul hears most often in the home while their uncle is visiting. For Sarah, it is unclear how strongly she and her adoptive parents identify with her initial language and culture before her adoption. And for Petr, it is unclear whether the “home” in the HLS refers to that of his U.S. host family or his German biological family, a difficulty that might be even further complexified if Petr were older and eligible to complete the HLS for himself.
These unclear findings suggest that within states real students who might be similar to our case students in terms of language exposure will have different HLS outcomes. Some will be identified for screening and others will not, based not on actual data about their language experiences and/or skills but on how parents/guardians answer unclear questions.
Additionally, Stefan and Petr had no states where they were clearly not identified for further screening. And Raul and Sarah were identified by more states than not. We attribute this largely to guidance states issued to educators in interpreting HLS. Overwhelmingly, states stipulated that any mention of a language other than English would require further screening (see Table 1). The four clear exceptions we found were as follows:
Maryland, which required mention of a language other than English on at least two questions
New Jersey and Wisconsin, which used a tree-like format (i.e., If you answered yes to Question 2, go on to Question 5)
Idaho, which used a matrix to decide based on responses to various questions whether a student is tested, receives a phone call home, or is not identified
One additional special case was in Oregon. On one of Oregon’s two versions (called the Oregon Bridge), there was inconsistency on how to interpret HLS responses. On one half of the page for all three questions, guidance stipulated students would be screened if any language other than English was mentioned. But on the other half of the page for two of the questions, guidance instead stated students would be screened if English was not mentioned. For bi/multilingual students, this wording is critical. Under one interpretation, Stefan would be identified, for example, because his parents listed Bulgarian, but under the other interpretation, Stefan would not be identified because his parents also listed English. We believe such inconsistency could be confusing for educators interpreting the HLS. (In our analysis, because the third Oregon Bridge question stated in both places that the threshold would be any mention of a language other than English, we determined whether to screen our cases based solely on this question.)
Finally, Sarah and Petr were both identified for screening by most states (45 for Sarah and 48 for Petr). We attribute this to the ubiquitous presence (n = 46) of questions about a student’s L1. When HLS asked about a student’s L1 and used any mention of a language other than English as the criterion to identify students, then Sarah (who was adopted internationally as an infant after beginning to learn Mandarin) and Petr (who was born in Germany and learned German as his L1) will be identified. Stefan was also similarly identified in states where questions allowed families to list multiple languages as L1s, but Stefan’s screening status was unclear in states where families had to choose one language.
Case-by-Case Discussion
In this section, we present findings from our four initial cases. First, however, we report results for Eva and Valentin. For these students, as expected, there was consistency in identification, and we were pleased to find that all 52 HLS identified Eva and Valentin for screening. Answering questions was straightforward because both students clearly had Spanish as their L1 and as the clearly dominant language in both their homes. This finding suggests that for students like Eva and Valentin, HLS questions yield consistent results.
Stefan
A map of results for Stefan reveals that in many states (n = 27) it was unclear about whether Stefan would be identified for further screening (see Figure 1). Stefan might be thought of as a “fluent bilingual,” although we reject the mythical idea that bilingualism should or can be “balanced” (Shin, 2018). Instead, we recognize that bi/multilinguals use their different languages, sometimes separately and other times together, for varied purposes; they develop proficiencies across languages in complex ways, related to both content and context (Grosjean & Byers-Heinlein, 2018). We also recognize that bilingualism does not fully capture many students’ multilingual repertoires. We attribute these unclear results to the “monolingual bias” that García and Kleifgen (2018, p. 13) warned against, related to how HLS questions often focus on only one language.
Map of results for Stefan.
Notably, all three USDOE questions were coded as “unclear” because each asked respondents to pick one language, rather than allowing Stefan’s family to share information about both his languages. Many states’ questions similarly required monolingual responses (e.g., Georgia: “Which language does your child best understand and speak?”). Questions that ask only about one language do not enable Stefan’s family to present a full picture of his bilingual resources, including his English proficiency. As mentioned above, several states’ HLS added -(s) to allow for multiple languages but did not alter questions’ verbs so plural “languages” would make sense grammatically (see Fowler, 2014, for more on the importance of question wording). Implicit assumptions within these questions included that families should be able to describe one primary language for themselves and their children. Such expectations simply do not fit the multilingual richness of many U.S. homes. Linquanti and Bailey (2014) argued that bilinguals would not be overidentified if questions focused on students’ English exposure and use. We agree, and we add that if the threshold for screening is the mention of any language other than English, then bi/multilinguals will by definition always be screened if they mention their additional language(s). Stefan was identified for further screening by 25 HLS. Given the overwhelming presence of guidance that any mention of a language other than English leads to screening (see Table 1), Stefan would have been identified in many of the states where his identification was unclear if questions had allowed parents/guardians to list multiple languages.
Raul
As with Stefan, the map of Raul’s results shows considerable variation in outcomes (see Figure 2). Raul was identified by 17 HLS, not identified by 8, and 27 were unclear. Raul might be regarded as a heritage learner, in that Spanish is important in his family culture (Valdés, 2005). As is the case between Raul and his parents, heritage learners can have a wide range of proficiencies in the heritage language (Montrul, 2010). For Raul’s family, questions asking them to quantify or pronounce more dominant the use of one language over another in the home, particularly during their relative’s visit, were uncertain (Byers-Heinlein et al., 2019).
Map of results for Raul.
For example, the different ways that HLS asked about language in Raul’s home was important for Raul. Some HLS asked Raul’s parents to pick one language as the “primary” language (e.g., USDOE: What is the primary language used in the home, regardless of the language spoken by your child?). While coding, we wondered: how might families interpret the word primary? We thought of several possibilities: that it might be the most dominant or privileged, the most culturally important, the first language, or as some HLS asked, the one most frequently used (e.g., Wyoming: What language is used by you and your family most frequently at home?). For Raul’s parents, these questions involve decision making about whether to include their uncle’s stay as part of the language culture of their home. This question is fraught with considerations of the importance of extended family, whether Raul’s parents consider their uncle a member of their home. We found this question to be not only practically impossible to answer but also possibly stressful, given English’s global privilege (Crystal, 2003) and the value families generally see in all their languages.
Sarah
Sarah was identified by 45 HLS, not identified by 4, and 3 were unclear (see Figure 3). As mentioned above, Sarah’s international adoption in infancy meant that she began learning English after first learning Mandarin. Research suggests school-aged internationally adopted children score at appropriate levels on standardized language tests but demonstrate some weaknesses in comparison to nonadopted children matched by age, gender, and socioeconomic status (Delcenserie, 2016). Researchers point out that comparison groups for internationally adopted children are often high-performing because the children are often adopted by economically and educationally privileged families (Delcenserie et al., 2013). For Sarah, we found questions asking about the student’s first language to clearly point to Mandarin as a response. However, the use of different terms sometimes made it difficult to know how Sarah’s parents would interpret questions. For example, we were uncertain whether Sarah’s family would list English and/or Mandarin for New Jersey’s question, “List home language(s) spoken” or whether they would choose yes or no for Michigan’s question: “Is your child’s native tongue a language other than English? Yes No. What is that language?”
Map of results for Sarah.
Similarly, when answering the following question from Colorado, we were uncertain whether they would consider Mandarin a language she learned at her weekend school or a language learned prior to adoption: “Does the student speak a language(s) other than English? (Do not include languages learned in school.)” This question pointed to the need of defining specifically what learning a language in school entails: Does it include weekend school, dual-language programs, language-immersive preschools, or other multilingual school settings? We did not code HLS for a student who learned an additional language solely at a school-based program, but considering such a case remains an important area for future research.
In states where Sarah was not identified, it was because Texas and Georgia HLS asked questions only about current, rather than historic, language use; Maryland required two responses to mention a language other than English; and Wisconsin used a tree format that effectively meant a language other than English had to be mentioned more than once.
Petr
Petr was identified by 48 HLS. Results for 4 HLS were unclear, and there were no states where Petr was not identified (see Figure 4). Though much literature exists on language development and study-abroad experiences, we found surprisingly less literature on placing K–12 exchange students in ESL services. Walker et al. (2004) reported that rural-area teachers who then had few or no immigrant students might have had their first encounters with linguistically diverse students through exchange programs. They contended that teachers’ attitudes might be more positive toward exchange students than toward immigrant students because exchange students often already have moderate English proficiency, are strong students, and come from European backgrounds. Interestingly, we found one state, North Dakota, which allowed an exception for not testing a student identified by the HLS if there was “overwhelming evidence of academic success at the time of registration.” While we see benefits of not requiring English-proficient students to complete testing every time they move to a new school, we are wary of how requirements, such as “overwhelming evidence of academic success,” might be interpreted inequitably for students, particularly given disparities in attitudes against immigrant students, such as those found by Walker et al. (2004).
Map of results for Petr.
Additionally, Petr was identified in so many states, due to the clarity of at least one question eliciting information about the presence of another language. However, although overall results were generally clear that Petr was identified, there were many unclear questions nested within HLS. These questions asked us to determine which of Petr’s homes (his U.S. host-family’s home or his German home) these questions were asking about. We found some states (e.g., Hawaii, Indiana, Virginia) that specified that exchange students should complete HLS, but we did not find guidance on whether host or home families should be respondents or the focus of questions. For example, we found ourselves uncertain how to answer questions like New Jersey’s: “At home, does the student hear or use a language other than English more than half of the time?” We also wondered about implications of questions asking about students’ homes for many EL subpopulations who might consider themselves as having multiple homes, including transnational and migrant students, students from split-family homes, or students who spend significant time with other adult caregivers.
Discussion
We set out to examine differences across states’ HLS in identifying (or not) hypothetical use-case students for further ESL screening. Our results reveal that while Eva and Valentin were consistently identified across states, inconsistencies exist in HLS for our four initial use cases. While these inconsistencies did not affect our use-case students who clearly spoke a language other than English as their L1 and as the dominant language in their homes, these results did affect students with more complex language situations. Findings suggest that the same student, due to both unclear questions within a state’s HLS and inconsistent questions across states, might be identified or not in what can seem like haphazard ways. Having a federally mandated HLS would certainly help with consistency in identification; however, we believe that having a federal tool would not help if questions are not carefully worded, given the diversity of families who respond to these surveys. Bailey and Kelly (2013) asked states to adopt HLS validation plans. We echo that call and propose that developing and using case students, such as those in our project, to test questions might be one method for HLS validation. Teachers and parents, who Bailey and Kelly (2013) rightly mention as partners in validation plans, can help state agencies develop case students based on children they know and work with, aligned with recommendations for use of hypothetical student profiles from Linquanti and Bailey (2014). Different stakeholders can then pilot and redesign questions with these students in mind.
Through this validation process, questions must be carefully examined to identify and eliminate instances of monolingual bias (García & Kleifgen, 2018). An immediate simple step in question revision can be made in pluralizing languages, so that questions allow families to answer HLS inclusively of all the languages they and/or their children use. This revision should not involve mere addition of -(s) on the word “language” but should minimally include revision of questions’ grammatical structures, so that they work either with plural or singular language(s). Additionally, educators should work with families in closely reexamining questions that might inadvertently ask families to describe their language repertoires through a monolingual lens, for example, by having to prioritize languages, naming them as “primary” or “dominant.” Questions asking parents to place percentages on how frequently family members speak a particular language should be avoided (Byers-Heinlein et al., 2019). And as Bailey and Kelly (2013) suggested, questions can focus on the student’s current, rather than historical, language use.
It is important also to consider that use of validation cases alone, however, might not be enough. States and divisions, and the many educators who work within them, as well as families and students themselves, might have differing opinions about whether Stefan, Raul, Sarah, and Petr should be identified for screening. Pluralizing “language(s),” for example, could still lead to overidentification of bilinguals if the threshold of mentioning a language other than English remains the trigger for screening. These disagreement areas call to attention the difficulties of identifying complex individuals through a simple, standardized list of questions.
We recognize also that states’ HLS development is informed by federal policies and guidelines. We found monolingual bias in all three of the USDOE-suggested questions. We recommend that the above-described revisions should begin with the USDOE questions. Additionally, use of the acronym PHLOTE in the USDOEOCR and DOJCRD’s (2015) guidance asks schools to identify students who use a “primary or home language other than English.” A large and growing body of scholarship has problematized labels in general, specifically those that position bi/multilingual students solely in relationship to English or suggesting that one of their languages is more important than the other(s) (García & Kleifgen, 2018; Valdés et al., 2014). In this context, we find the PHLOTE label emphasizes the monolingual idea that students have one “primary or home” language that is identified as other to English and perhaps affects the questions that are ultimately included in HLS. We recommend that language of policies, guidelines, and HLS questions be carefully examined to promote inclusivity of a wide diversity of bi/multilinguals (Lacelle-Peterson & Rivera, 1994; Wright, 2019).
We did not specifically examine HLS administration, but we saw promise in state guidelines that required an intermediary step between HLS and testing. For example, states conducting family interviews or implementing the idea of a “watch” category (Linquanti & Bailey, 2014) before testing might provide an opportunity to clarify “unclear” questions before assessment. It is important to note that divisions and division personnel carrying out these intermediary steps would need additional support, guidance, and resources, for example, in clarifying questions to ask in an interview or information to find from school records. We recognize that policymakers can support having an intermediary step by allowing flexibility in the 30-day time limit, with parent/guardian approval, for schools seeking to include additional data gathering in the initial identification process.
Additionally, we ask humbly for cooperation among states in developing, sharing, and implementing HLS questions. We ask policymakers, consortia, and state leaders to consider ways states can collaborate to share students’ previous assessment data, so the identification process does not have to restart every time a student moves between states. Sharing of information, even across state lines, can also provide a student’s new teachers with valuable assessment and instruction information that can speed up students’ access to quality services in their new schools (Law & Eckes, 2007).
We ask researchers to share their expertise about good survey design with partners in education who are working on HLS question development. And we ask that researchers examine these areas for future scholarship:
How translated versions of questions affect HLS outcomes
How HLS are administered
How students who have participated in multilingual school programs answer HLS
How families experience HLS and the greater identification process
How the size of states’ EL populations and other demographics covary (or not) with characteristics of HLS
How bias might affect educators’ interpretation of HLS
How many students are under- or overidentified by various HLS questions and what their long-term educational outcomes might look like
We recognize that the work toward HLS improvement is not easy. Given the diversity of multilingual students in U.S. schools (Wright, 2019), designing questions that are flexible enough to solicit useful and consistent responses is particularly challenging. Revising questions involves detailed validation processes with many partners working together. We also recognize that the HLS is one piece of the overall ESL assessment process, and we echo Bailey and Carroll’s (2015) call for reevaluating the entire system.
Footnotes
1.
Given our data collection’s timing in spring 2020, some states shared guidelines for HLS administration during pandemic-related closures; this guidance generally included instructions for administering HLS while social distancing. Though we recognize that how a survey is administered is important (Fowler, 2014), we found that pandemic-related instructions were irrelevant, given our focus on HLS questions.
Authors
APRIL S. SALERNO is an assistant professor of ESL teacher education and curriculum and instruction program coordinator at the University of Virginia. Her research interests include understanding how to better prepare and support preservice and in-service teachers to work with multilingual learners.
ELENA ANDREI is an assistant professor of TESOL and TESOL program coordinator at Cleveland State University. Her research interests include second language literacy and multiliteracies, teacher education, and “nonnative” English speaking teachers.