Abstract
Safe and secure housing is a pressing concern for survivors of domestic violence (DV). Domestic Violence Housing First (DVHF) is a policy that serves DV survivors’ needs by removing barriers to housing stability via flexible financial assistance, mobile advocacy, trauma-informed practice, and partnerships with community housing providers. In this paper, we outline a study protocol developed to evaluate the efficacy of the Wisconsin Domestic Violence Housing First Pilot Program (WI-DVHF). Funding was provided to 9 social service programs across the state and implemented in novel ways. We develop a multi-site, multi-level, longitudinal analysis of WI-DVHF at the individual, site, and community levels. WI-DVHF includes 4 elements differentiating it from previous studies, including analysis of DVHF in rural and urban locations, unique implementation across different sites, and access to a WI rental education program. We also include data from comparable non-WI-DVHF sites and clients served during the study period. This will address concerns about comparing survivors within a site as DVHF funding may free up funds that can benefit other survivors at that site and to better understand the general challenges and trends these organizations face during the study period. The US housing crisis emphasizes the importance of identifying unique and flexible ways to meet the housing needs of DV survivors. This study highlights the potential for DVHF to improve the lives of survivors, and its adaptability to previously understudied cultural groups and across the rural-urban divide. Further, WI-DVHF funds may positively benefit other programs and participants within a site by freeing up valuable space in DV shelters and other resources.
Housing insecurity is a major issue for survivors of domestic and intimate partner violence, but resources are limited.
We describe a study protocol for implementation of the Housing First philosophy in 9 Wisconsin domestic violence shelters/service providers.
This work will provide insight into how flexible financial assistance and survivor-focused housing programing can best serve survivors of domestic violence.
Background
In Wisconsin, as in most of the United States, the threat of homelessness remains a significant barrier for families attempting to escape domestic violence (DV). i While emergency shelters, a common and historically important service mechanism, are located throughout the state, they are often at capacity and spaces are extremely limited—a phenomenon that has been exacerbated by movement restrictions and increased violence associated with the COVID-19 pandemic.1-5 Between October 2019 and September 2020, the Wisconsin Department of Children and Families (DCF) documented that 20 682 adults requested shelter accommodation but were unable to be helped due to shelters being full. 6 Even if emergency housing were able to accommodate all immediate survivor needs, safe and stable permanent housing would still be a pressing need for survivors. Indeed, housing instability is a potent predictor of poor health outcomes among DV survivors. 7 Furthermore, for many DV survivors, financial coercion is a key component of abuse and leaves them without a stable income, emergency savings, or decent credit rating. 8 Consequently, survivors often lack funds for a security deposit, rent arrears, or utility bills, making safe housing for themselves and their children unobtainable. In 2020, it was estimated that 88% of unmet DV survivor needs in Wisconsin related specifically to the attainment of housing and shelter. 9 Homelessness and access to housing, therefore, continues to be a critical challenge for families leaving violent and abusive situations in Wisconsin.
One model of service provision that attempts to address the housing insecurity faced by DV survivors is Housing First (HF). Originally developed to address the need for safe and stable housing among homeless community members, 10 service providers adopting HF for DV survivors (DVHF) use trauma-informed care practices to enhance emotional safety, mobile advocacy approaches to meet survivors where they are, community engagement to increase partnerships with housing providers, and flexible financial assistance (FFA). 11 In the context of DVHF, FFA refers to financial support designed to reduce any barriers faced by the survivor in acquiring safe and stable housing including payment of rent or rental arrears, moving expenses, security deposits, childcare, utility and medical bills, transportation costs, and home safety systems, among others. The DVHF philosophy guides FFA to be “no strings attached” funding, creating a unique opportunity for flexibility and creativity in how DVHF funds are tailored to each client’s specific needs and each organization’s guiding philosophies and strategies. Ultimately, the goal of providing FFA in conjunction with trauma-informed care and mobile advocacy is to assist survivors in securing safe and stable housing, which, in turn, is a prerequisite for addressing other mental and physical health, employment, and family needs. 11
Critically, previous studies have demonstrated some of the benefits of the DVHF model. For instance, in California and Washington, 93% of DV survivors who received DVHF funding reported that they had learned more ways to plan for their safety and 96% of survivors reported feeling more hopeful about the future. 12 Additionally, a recent analysis of 345 homeless or unstably housed DV survivors found that DVHF resulted in greater housing stability and decreased economic abuse by partners over 6 months compared to DV survivors who received typical services. 13 In studies of homeless populations with significant psychiatric and substance-use comorbidity, those who received HF had higher rates of permanent housing, were better able to maintain permanent, independent housing, and had lower rates of substance abuse compared to those who did not receive HF.14,15 Therefore, these findings suggest that HF and DVHF may influence well-being in ways that extend beyond housing stability.
As part of the American Rescue Plan, Wisconsin’s Department of Children and Families (DCF) received funding to implement a pilot program of HF for DV survivors throughout Wisconsin—the Wisconsin Domestic Violence Housing First Pilot Program (WI-DVHF). To determine the effectiveness of WI-DVHF, our study evaluates whether DV survivors who receive DVHF funding have improved safety, financial, educational, health, familial, and social outcomes. To do this, we will collect original quantitative and qualitative data from DV survivors across 9 Wisconsin DV organizations participating in the WI-DVHF pilot program. We also will collect site-level data quarterly to understand differences in how housing is addressed across sites as well as how the presence of housing funding is associated with reductions in shelter stays, how funds are used for DV survivors (including spillovers to non-DVHF programs and clients), ii site operations and structure including effects on site staff (eg, turnover, burnout, and staff morale), and relationships with landlords and housing associations within the community. The evaluation team will also attend the program’s monthly cohort technical assistance calls, where sites discuss challenges of implementation and share strategies. To our knowledge, this is the first multi-site, multi-level, longitudinal study of the implementation of DVHF as well as the first to include comparison sites.
In this paper, we describe the program evaluation protocol developed to assess the effectiveness of the WI-DVHF. We outline challenges posed by this task, both in terms of the population of study and unique to the WI-DVHF program. We also identify key innovations of the WI-DVHF program that will offer new insights into the challenges faced by DV survivors and new solutions created by DV programs.
Research Questions
The WI-DVHF evaluation was developed to address several important research questions relating to DV and survivor outcomes. First and fundamentally, does DVHF assistance help DV survivors secure and/or maintain stable and adequate housing? Second, does DVHF funding lead to better financial, employment, health, educational, familial, and social outcomes for recipients over time? Third, how do these outcomes compare to DV survivors with housing needs who did not have access to DVHF funding? In each instance and throughout our evaluation, we hypothesize that DV survivors who received DVHF assistance will not only have improved outcomes over time but also in relation to survivors who did not receive DVHF assistance.
In addition to these individual-level questions, our study is also uniquely positioned to explore how DVHF is administered across sites throughout the state. In particular, we are interested in how sites use DVHF funds to assist clients and what this means for available resources and organizational climate and efficiency at each site. For example, DVHF may help organizations to shorten (or avoid altogether) shelter stays, which, in turn, increases bed availability without physically increasing shelter capacity. Additionally, through quantitative surveys and qualitative interviews with sites over time, we will explore changes in the organization’s capacity to work with landlords and rental companies as well as with other local resources (eg, household repair personnel, car mechanics, movers) with potential to improve client safety.
WI-DVHF Sample Frame
The initial call for site applications for the WI-DVHF program was released August 24, 2021, with expressions of interest due October 5, 2021. From 21 applications, DCF initially awarded WI-DVHF funding to 4 sites, which was then expanded to 9 sites in December 2021, when additional funding became available. These 9 sites are scattered across Wisconsin and include more traditional DV organizations offering shelter alongside a variety of supportive services, non-shelter DV programs, and larger social support programs that are expanding programing for DV survivors, such as culturally specific mutual assistance organizations or youth assistance programs. Sites selected to participate in the WI-DVHF program received site grants to be used in the implementation of DVHF. Table 1 provides descriptive statistics for site characteristics (including control sites, discussed later in this paper). The WI-DVHF program also includes funding for technical assistance and support, which is provided by End Domestic Abuse Wisconsin, and funding for an evaluation of the program.
Descriptive Statistics of Control and DVHF Site Characteristics.
Sites report different services areas across data reports. We report the largest stated number of counties served.
Data were pulled from the DCF Annual report for 2018 to 2019. This value is the sum of total adults and children (shelter and non-shelter) served by that site in that year.
Data were pulled from the DCF Annual report for 2018 to 2019. This value contains data on total shelter nights (total nights of stay across all clients served in shelter) for the year 2018 to 2019.
This measure indicates the number of shelter beds available at the site as of 2020. “Unknown” indicates that the site did not respond to that request for information. Data collected by Sims (2021).
Our evaluation team was approached by DCF to develop and implement a program evaluation protocol for WI-DVHF in December 2021. Consequently, participant sites had already been selected, notified, and in some cases, had their funding disbursed. This meant we were not able to randomize which applicant sites received funding nor how the funds were to be utilized. Given the selection process coordinated by DCF, which involved scoring each organization’s application for DVHF funding, we also are cognizant that the selected sites could have better functioning and resources relative to other sites, and thus our findings may not generalize to other organizations. We therefore designed our evaluation to the specific details of the sites selected and their proposed implementations of DVHF. We also were conscious of the unique characteristics of Wisconsin compared to former DVHF locations (eg, California and Washington) when developing our evaluation and adapted our approach to be relevant for specific cultural groups in our communities and to focus on important contextual characteristics of sites (rural/urban, etc.).
Sites varied in their proposed use of funds, with some requesting funding to hire new staff (n = 7), top off existing FFA funds (n = 7), create FFA funds (n = 2), increase mobile advocacy efforts (n = 7), increase the technical skillset and financial education of clients (n = 4), provide clients with safety kits (n = 1), and facilitate culturally responsive systems of reimbursement (n = 1). There were also some consistencies in how sites proposed using funds; for example, all 9 sites indicated they would rely on existing staff to take on at least some DVHF responsibilities, including transitioning staff to housing-specific positions. Finally, there are several unique elements about our WI-DVHF sample and its implementation:
Rural-urban divide
While housing instability and homelessness can affect all DV survivors, we anticipate that the challenges of finding suitable housing will differ along rural and urban lines. Wisconsin is experiencing a housing crisis, 16 but the nature of this crisis varies, with some areas having rental units priced well above both the market rate and Section 8 housing voucher limits, while other areas lack units that meet basic safety and sanitation standards.17-19 The housing market in some regions—for example, urban Milwaukee—are characterized by a few major landlords, while others—like rural northern Wisconsin—more frequently have “mom-and-pop” landlords.
Conveniently, the WI-DVHF sample includes sites located in both urban and rural communities. Two sites are located in Wisconsin’s major metropolitan centers, Milwaukee and Madison, while 2 other sites serve smaller urban centers with the rest located in primarily rural areas. In addition, while the majority of organizations only service one county each, 3 sites serve 2 or more counties (and a mix of urban and rural counties), and one is the only organization offering bilingual and bicultural services for DV survivors in an 8-county area. The sites in the sample therefore serve clients who are often located in very remote and sparsely populated areas.
Additionally, if Wisconsin is divided into compass quadrants, the distribution of sites is somewhat skewed. Four sites are located wholly within the southeast, primarily because of the presence of both Madison and Milwaukee in the sample. One site is solely in the southwest, and 2 are located in the northwest. The remaining 2 sites—one in the north, and one in the south—are split across the east and west. The northeast of Wisconsin is, therefore, underrepresented in WI-DVHF sample by geographic area, although not by population dispersion. Despite this limitation, we believe the sample provides a holistic perspective on the challenges of providing DV and housing services in both rural and urban areas.
Unique implementation and programing
Second, because of the open nature of the call for proposals from DCF, organizations have been able to propose a number of innovative and novel implementations of the DVHF model. For example, one site has operated a small FFA/HF program since 2017 with great success and in addition, to using WI-DVHF funds to support survivors, the organization also chose to create a Housing Engagement Coordinator position. Rather than providing direct advocacy services, the Housing Engagement Coordinator would be responsible for “county-specific community engagement efforts to increase housing partnerships and resources for survivors,” which would include initiatives such as a “Landlord Liaison Program” designed to educate landlords about the housing barriers experienced by DV survivors, assuage their hesitancies in renting to survivors, and outlining the organization’s supportive role. The coordinator would also be responsible for identifying and training sites’ own client advocates (existing staff and/or newly hired employees to assist with housing) on county-specific resources such as car mechanics, home security providers, and home repair services that could be engaged to fill “non-traditional, housing related resource gaps.” Thus, while providing immediate services is a priority for this organization, it is also using the WI-DVHF program to create and consolidate community linkages necessary for ensuring stable and safe housing for survivors after the pilot has concluded.
Meanwhile, other organizations have utilized the open nature of WI-DVHF to implement unique and much need culturally specific programing. For example, one culturally specific organization focused on Wisconsin’s Hmong population determined that their clients disproportionately preferred to “double up” or move in with others such as siblings, cousins, or friends, while making a final decision about their abusive relationship. The organization, therefore, proposed to use FFA funds to compensate family members and friends to help defray the costs of their stay. The organization also argued that funding this reimbursement scheme would also help survivors to feel as though they were contributing and maintain a degree of personal autonomy, particularly in a situation where they can often feel powerless. To our knowledge, this is the first systematic use of FFA to fund semi-permanent, familial home stays and has arisen because of specific preference amongst the Hmong community.
Likewise, another site–although not itself a culturally specific organization–has proposed partnering with such programs in their area, including organizations that serve Spanish- and Arabic-speaking communities. The organization proposed these partnerships in recognition of the lack of trust in traditional support services amongst marginalized, non-White, non-English speaking communities due to prior substandard treatment and services. These considerations are particularly important given the impacts of racism and intergenerational poverty on both the likelihood of experiencing DV and the challenges in overcoming housing instability. This organization recognized their ability to reach survivors in these communities would be limited despite the incredible need and demand for DV and housing assistance. Thus, while the site will still be responsible for dispersing WI-DVHF funds, it will do so in conjunction with organizational partners who are trusted and familiar with their clients’ personal and community needs.
WI Rent Smart
The final unique element of WI-DVHF is the presence of UW Extension and its outreach program Rent Smart. UW Extension’s Rent Smart program is a free web-based course that focuses on providing individuals with the knowledge and skills necessary for a successful renting experience. This is particularly important as many DV survivors have limited experience with tenancy or less positive experiences with renting and tenancy, and this means that DV organizations often also adopt the role of tenant advocate and act as a “go between” by helping to manage tenant and landlord expectations in addition to regular DV advocacy. The Rent Smart program helps participants to know and understand their rights and responsibilities as a tenant as well as knowing and understanding the rights and responsibilities of landlords, which in turn (potentially) minimizes the burden upon DV organizations.
To assess the impact of Rent Smart, we will include questions regarding Rent Smart participation, knowledge generation, and tenancy issues in our individual survivor and organizational surveys (discussed in more detail below). By including these questions, we hope to investigate not only rates of participation amongst DVHF recipients but also whether Rent Smart helps increase DV survivors’ knowledge of their tenancy rights and responsibilities. Additionally, we will be able to provide valuable feedback to UW Extension on the effectiveness of the Rent Smart program specifically for DV survivors, which often require specialized assistance and programing.
Study Design
To assess the effectiveness of WI-DVHF, we intend to track outcomes over the course of a year at 3 distinct levels: (1) DV survivors, (2) DV sites, and (3) the broader community. Here, we describe the study design for each level, as well as our control sites.
DV survivor analysis
First, at the microlevel, we are interested to observe whether survivors who receive WI-DVHF funding not only see an improvement in their housing situations but also improvements in other aspects of their lives. To determine the impact of WI-DVHF, we will ask each DV study site to obtain a release of information (ROI) from potential study participants (ie, clients who are eligible for housing assistance) and transmit their first name, preferred contact information, non-English language or accommodations needed to complete the survey, services accessed and dates of services, WI-DVHF funding amounts and uses to the evaluation team. This client information will be transmitted securely to the study team via encrypted link once per month. The ROI script that advocates and case managers will use makes clear to potential research participants that should they choose not to release their contact information to the study team, it will not affect their access to DVHF funding. Not all individuals served by an agency need DVHF assistance, each agency has its own criteria to determine eligibility for DVHF assistance, and the amount of DVHF funding provided to each agency is unlikely to be enough that all individuals who are in need and eligible for DVHF could receive assistance at each site. Sites also will track and submit aggregated information to the study team about the number of clients who declined to release their contact information and the total amount of funding received each month by those who declined to contextualize the scope of the evaluation.
Using this contact information, the study team will then begin collecting longitudinal data on participation via HIPAA-compliant Qualtrics survey with a link unique to each participant. The survey will be protected via a password to help prevent others, particularly (ex)abusers, from gaining access, after which participants will be provided with a consent form followed by the survey itself. iii The survey will cover 8 distinct areas including demographics, housing stability, employment, health, family (for those with children), personal wellbeing, DV exposure (historical and current), and experiences of the WI-DVHF program and Rent Smart. In order to enhance participation and retention, after completing the survey, participants will be eligible to receive a gift card after completing each survey. In order to collect longitudinal data, participants will receive a survey invitation every 3 months over the course of a year. Additionally, we plan to link administrative data provided by sites, including what DVHF funds were used to pay for and how much DVHF funding they received, to the self-report data from participants to examine differences in wellbeing based on type and amount of financial assistance.
Because several sites are providing culturally specific programing, in particular to the Hmong community, we are also planning to make several changes to the collection of individual-level data with these participants. First, we will engage a consultant who works with the Hmong community to ensure the survey is culturally sensitive and appropriate to the needs of those potential respondents. Once that is complete, we will translate the survey into Hmong, Spanish, and Arabic to maximize participation and comfort for those whose first language is not English. We are also anticipating that a small number of participants will not be literate—either in their first language or English. To reach these individuals, we will conduct interviews by Zoom, phone, or in-person, which will provide a comprehensive picture on the impact of WI-DVHF.
Finally, if grant funding permits, we would like to complement our survey data with longer form interviews and/or focus groups—beyond those conducted as a written survey substitute—with a small number of WI-DVHF participants from across sites. Longer interviews and focus groups will allow us to collect more in-depth data about participants’ experiences of WI-DVHF including unanticipated findings that arise from the survey methodology.
DV site analysis
In addition to the effects on individual recipients, we anticipate that WI-DVHF will have a substantial impact on DV sites themselves including their operation and provision of services. Information for this site-level assessment will be collected through a variety of avenues. At the outset of the pilot, we intend to interview relevant staff (ie, site leadership and staff directly involved in DVHF implementation) at each site via Zoom to gain an understanding of each site’s implementation of WI-DVHF and the criteria they are using to determine client eligibility. iv This is important as DCF has placed very few constraints upon the disbursement of WI-DVHF funds, meaning each site individually interprets what FFA covers and who it assists.
After establishing this baseline, we will undertake a quarterly site-specific survey, which will focus on (1) organizational functioning since receiving pilot program funding including organization size, employee workload, and funding availability; and (2) provision of WI-DVHF to clients including the amount of funds dispersed, their use, the impact on shelter stays, and the successes and challenges experienced with clients. These surveys will be administered to both DVHF and control sites. We will specifically ask DVHF sites about the implementation of mobile advocacy, including how staff stress and time constraints have changed in the transition to off-site service provision. Additionally, we will ask about client caseload and caseworker burnout. The DVHF philosophy emphasizes the importance of high-quality client contact and intensive advocacy, which may mean that caseworkers ultimately serve fewer clients, but in more efficient ways. When paired with the data on number of clients served by DVHF, these data will help us to understand whether this increase in advocacy intensity and decrease in caseload does or does not occur in the implementation of the WI-DVHF program. Since substantial sections of this survey will be longform responses, we will conduct follow-up interviews with relevant site staff (eg, leadership and DVHF implementers) to delve more deeply into how views of DVHF, implementation, and challenges change over the course of the pilot. These ongoing surveys and interviews will allow us to adapt our data collection strategies accordingly. We will also supplement these surveys and interviews with administrative data on WI-DVHF funds given to clients. These administrative data include the total number of people who consented to share their contact information with the evaluation team as well as those who did not, and the total amount of funds distributed to these respective groups. Sites will also detail the amount of funding and funding purpose for individuals who consent to release their contact information to the study team. Consistent with best practices for PHI data transfers, all data will be sent to the evaluation team via encrypted link.
The final part of our site analysis consists of attending monthly technical assistance Zoom calls organized by End Domestic Abuse Wisconsin. These meetings provide the participating DV organizations opportunities to network and learn from each other and outside experts. These opportunities help DV sites avoid known issues of service fragmentation and operational siloes that often lead DV organizations to recreate solutions to problems faced by other peer organizations.20-23 Attending these calls provides a touch point for both sites and the study team. The challenges and pitfalls sites describe in implementing DVHF will inform the study team’s development of semi-structured interviews for on-site visits. The study team is also cognizant of the time- and labor-intensive processes sites undergo to participate in the evaluation. By being present at site meetings, we will build relationships with sites and be able to answer questions they may have. We anticipate that the results from our site-level analysis will create tools for DV organizations to learn about common challenges and innovations during the implementation of DVHF.
Community analysis
Third, we intend to investigate WI-DVHF’s effect on local communities. In all of their proposals, sites indicated a substantial need and desire for stronger community relationships, especially with landlords, property managers, and other local gatekeepers for rental properties. Sites hope that stronger links with these community stakeholders will not only help to overcome the common biases against DV survivors—including preconceptions about property risk when renting to people with histories of DV, eviction, and substance abuse—but also racial and social prejudices.24,25 Moreover, sites believe these connections are essential given the current shortage of affordable housing in all of their service areas.
As part of our evaluation, we intend to study how sites’ community networks change—both in terms of the number and the quality of network connections—over the course of the WI-DVHF pilot program. To do this, we will be investigating sites’ links with the community through our quarterly site survey and asking sites about the number of referrals they are providing to other local service providers, their relationships with landlords and how they cultivated those connections, what education and outreach has been done to correct misconceptions about DV survivors, and whether sites have participated in broader community efforts to secure affordable, local housing. Additionally, if our budget permits, we also intend to do repeated interviews with community stakeholders, including landlords, who have worked with WI-DVHF sites to learn how their expectations of and willingness to rent to DV survivors has changed.
“Control” sites and study participants
The final aspect of the study design involves the inclusion of “control” sites and participants. While previous work has studied the outcomes of participants who receive DVHF assistance13,26,27 or the difference between participants and non-participants at DVHF sites, 28 these approaches may underestimate the effects of DVHF programing because of spillover effects. Since demand for DV services always outpaces available funds and programs, program staff must often make difficult decisions about how and to whom services are offered. Site access to DVHF funds along with the concept of FFA could not only impact the needs of clients who receive DVHF but also could impact the needs of non-participants by increasing funds available to that site more broadly. v Ultimately, such spillovers would yield violations of the Stable Unit Treatment Value Assumption (SUTVA),29-31 causing issues with interpreting the effectiveness of treatment in the DVHF program.
In our evaluation we address some of these concerns about DVHF effectiveness and expand on existing DVHF work 32 by including matched control sites in our analysis. These control sites are not participating in the WI-DVHF program either because they did not submit a proposal to DCF or because they were not selected. Critically, we have matched control sites with participant sites based on a series of characteristics including degree of urbanity, program size, staffing capacity, available programing (including whether the site operates an on-site shelter), and cultural competencies. Due to research capacity and the limited number of sites in certain areas of Wisconsin, we have identified 5 possible control sites, 4 of which agreed to participate in the evaluation. Study participants from the control sites will complete the same individual-level surveys and will be monetarily compensated like participants from DVHF sites. Because our site-level analysis requires significant effort on the part of site staff, including regular communication, interviews, surveys, and data collection, we will compensate each site by purchasing approximately $1000 of items off of their site Amazon wish list (or equivalent)—wish lists like this are often used to provide resources to survivors in shelter or those who have had a forensic sexual assault examination. All study materials will be adjusted appropriately to remove references to DVHF, but otherwise content will remain the same.
Although DVHF sites may have spillover effects to other local service providers in via relaxed budget constraints and stronger community networks, we argue that spillover effects within communities will be of a lower magnitude than will spillover effects within a site. Almost all of our DVHF and control sites are located in distinct counties except for those sites within large metropolitan areas. Further, we expect that spillovers between sites in large metropolitan areas will have smaller effects due to dilution among the larger network of service providers.
Furthermore, including these control sites within the evaluation will allow us to better understand how “external” forces, such as state or federal policy changes, are affecting DV programing across the state. The most evident example of this would be the Dobbs decision, but given that our study period will include the 2022 US midterm elections and Wisconsin has a number of close electoral races at both the federal and state levels, other external forces may include post-election state budget priorities and spending.
Finally, to address the potential that “control” DV sites may have limited recruitment ability (and may prioritize other tasks/activities when not required to participate in our evaluation, in contrast to the DVHF sites), we will use administrative data provided by DCF on all DV service providers in the state. These annual data consist of year-end reporting on service provision and general funding for all providers, including the DVHF and matched control sites.
Empirical Strategy and Analytical Limitations
We believe that this evaluation of WI-DVHF is the first multi-site, longitudinal analysis of the DVHF philosophy, which will analyze the impact of the pilot program on not only individual survivors but also DV organizations and their communities. We anticipate that the study will provide a greater understanding about the benefits of prioritizing safe and stable housing solutions to survivors leaving abusive relationships and situations. However, despite this, we are also aware of the limitations and concerns of the study.
First and foremost, because of DCF’s solicitation process, we were unable to design a standardized intervention and then randomize site selection and DVHF implementation. We are thus unable to empirically test for causal relationships between program implementation and survivor/site outcomes. However, randomization of DV survivors or sites to specific interventions is often not feasible or practical, and in some cases, may pose ethical challenges. 32 Thus, while randomization was not possible for this evaluation, our study design can still provide important information about the real-world effectiveness of WI-DVHF.
Although our study includes 13 sites—9 DVHF and 4 control—the potential pool of WI-DVHF individual participants at certain sties is quite small. While most DV organizations in Wisconsin serve several hundred people a year, others, in comparison, only have a few dozen clients per year. 33 This sample size will be further limited as individuals must first consent to releasing their contact data to the evaluation team and then consent to participate in the longitudinal data collection process. This limited sample size will affect our statistical power to identify significant between-person effects of the DVHF program, particularly within small sites.
This potentially small sample size is the reason we have chosen to leverage the use of multiple sites and the longitudinal study design as this should allow us to model within-person change over time with adequate statistical power. We will use a combination of correlations, descriptive statistics, and t-tests to assess how the DVHF and control sites differ, and how individuals’ lives (including their personal wellbeing and housing security) have changed both as a result of and in the months after participating in the program. We also will use mixed effects models to examine change in outcomes over time for individual participants that result from participating in DVHF versus being served at a control site. Individuals will be nested within sites for these analyses to account for dependency within sites. We will also conduct qualitative analysis on recorded interviews, focus groups, and long-form survey responses to identify key themes, challenges, and lessons. After interviews and focus groups are transcribed, we will use Braun and Clarke’s steps for thematic analysis to conduct a thematic analysis,34,35 which include: (1) data familiarization, (2) generation of initial codes, (3) identification of themes, (4) review of themes, (5) identification and naming of themes, and (6) production of findings.
One concern related to the study’s small sample size is participant retention. Although subject retention is always a concern in longitudinal studies, it is especially difficult when working with vulnerable populations such as DV survivors. 36 Not only might DV survivors be prevented from using their phones and emails, 37 but internet stalking has become easier and more prevalent,38,39 increasing security and safety concerns for DV survivors.
Because trauma-exposed populations are difficult to engage and retain in research, 40 we have prioritized evaluative surveys that will allow us to incentivize participation to improve retention. For all surveys, we are providing a $50 electronic gift card to Walmart, Target, Amazon, or Kohls. To incentivize participation in all surveys, we also will provide a $20 bonus to those participants who complete all 4 surveys. Lotteries and raffles have shown to be less helpful in retaining participants in longitudinal studies compared to guaranteed incentives. 41 Participants will also receive monetary compensation if the evaluation team hosts focus groups and interviews. Since DV survivors are also forced to change their contact information when they leave abusive and violent situations, at the end of each survey, we will ask participants if they are willing to be re-contacted for the next survey round and, if so, to provide their best contact information. Prompting for this information is considered best practices for retaining DV survivors in longitudinal research 36 and should allow us to maintain contact with our respondents.
A second and critical concern for the evaluation team is the potential for harm to participants. Harm, in this context, could occur through several different avenues. First, the study team is concerned about causing participant distress by implying their participation is required in order to access services provided by DVHF sites or through re-traumatization. We have, therefore, been careful to emphasize in both the release of contact information and the survey consent form that participation is voluntary and that if they consent, participants may skip questions or stop at any time without losing DVHF or other benefits. The evaluation team has also provided context before asking questions from validated scales and throughout the survey instrument to ensure that questions have been asked sensitively and cause minimal distress. vi
Harm to participants could also occur through breaches of confidentiality and privacy. For example, although it is unlikely, knowledge that a respondent participated as well as their survey responses and contact information could be accessed by abusers or the judicial system via subpoena. Indeed, there was particular concern about abusers using subpoena power to access survey questions about children’s safety and wellbeing for use in custody disputes.
The study team has therefore taken several steps to mitigate and minimize these concerns. First, we are collecting the minimum information necessary to carry out this evaluation—for instance, we are not asking for full names, only first or preferred names—and we will never ask for location data, such as addresses. Second, although the study was initially approved by the University of Wisconsin-Madison IRB as program evaluation, we have voluntarily undergone the full IRB review process in order to secure a Certificate of Confidentiality, which will protect study materials from subpoena. Third, all sites are responsible for obtaining a release of information prior to transmitting the contact information and administrative data for potential study participants, and we also negotiating a Data Transfer and Use Agreement with sites which will outline the legal restrictions for how the evaluation team is able to utilize the collected data. Fourth, we are using encrypted data transmission and protected data storage systems to receive participant contact information and collect and maintain research data. Crucially, these encrypted measures are HIPAA- and PHI-compliant. Fifth, to mitigate the risk of abusers (or others) seeing the email to participate in the survey, we limited description of the study on the landing page to being a survey about “housing” and potential participants must enter a password given by the DV site prior to viewing the informed consent form and the survey questions. Sixth, data are never stored with identifiers and only de-identified aggregate data will appear in policy briefs or journal articles. Finally, in published articles and public presentations, table cell sizes of <3 will be suppressed to protect survivors who hold potentially re-identifiable minoritized identities. We believe that these measures will minimize any potential harm to our research participants.
However, these stringent protections for DV survivors also impact the generalizability of our evaluation. For example, we are unable to study the impact of DVHF funding on individuals who do not consent to release their contact information to the study team. This could mean that there may be differences between individuals who do and do not consent to participate, and those who consent could self-select into the study for unknown reasons. Consequently, this could limit the generalizability of our individual-level analysis. To address this concern, we will collect data on the number of DVHF recipients who decline to provide their contact information and the overall amount of DVHF funding provided to these participants to understand the scope of potential bias in our analysis. We will also face similar limitations to generalizability if there are differences in participant retention. However, our site-level analysis will include aggregated information on all funds disbursed, allowing us to compare whether individuals who did and did not consent (or reply over time) differ from each other in terms of the services they received.
Discussion
Safe and secure housing is one of the most critical needs for DV survivors. Indeed, DV survivors self-identify stable housing and the resources to maintain stable housing as one of their most pressing needs. 42 Despite this, limited and restricted funding as well as severe housing shortages have made it incredibly difficult for DV organizations to provide adequate housing services and assistance to their clients. The WI-DVHF pilot program is designed to address these housing challenges and allows providers to trial and implement innovative and unique solutions that are consistent with the DVHF philosophy. Critically, our evaluative framework will offer insight into the effectiveness of these implementations of DVHF on the lives of DV survivors, including DV survivors from previously understudied communities.
Our study advances research on the interaction between DV, DVHF, and service provision in several distinct ways. First, we move beyond the outcomes that are typically studied, such as housing stability and decreased exposure to violence, to incorporate measures of general physical and mental health, family functioning, personal well-being, resilience, and access to additional social services. Additionally, we expand on existing work that has typically studied outcomes among DVHF recipients at a single site by examining implementation of DVHF at multiple sites and including matched control sites. This design will allow us to draw comparisons between WI-DVHF and non-WI-DVHF sites, thereby avoiding spillover concerns regarding recipients who have not received DVHF funding within WI-DVHF sites. Finally, we are the first to study DVHF as implemented for Hmong survivors of violence.
In addition to these academic contributions, this program evaluation will be able to inform policy decisions about the provision and funding of DV services in Wisconsin and potentially beyond. For example, traditionally, housing assistance to DV survivors has been limited to vouchers in set funding amounts. However, these vouchers often do not begin to approach market values for rent, especially during the current affordable housing crisis in Wisconsin and across the US. Further, survivors leaving abusive situations are also confronted with costs such as hiring a moving truck, installing security systems, or additional transportation costs, which are not covered by vouchers. To address these concerns, DV organizations are forced to use their unrestricted funds that are often minimal and limited. In contrast, the flexible financial assistance approach of DVHF relaxes the common restrictions placed on funding and adopts a holistic approach to housing assistance. Therefore, our findings could be used to advocate for the expansion of flexible funding to more DV organizations and the relaxation of restrictions on housing assistance.
Our evaluation also has the potential to reveal effective strategies and policies for DV organizations seeking to build greater linkages and coalitions with housing gatekeepers such as landlords and property management companies. Given the current affordable housing crisis, developing effective approaches for cultivating landlord relationships and providing education about the challenges faced by DV survivors is crucial for overcoming this challenge. Therefore, this study has the ability to assess which strategies and tactics are effective and provide guidance on the best methods for developing these critical landlord relationships. This opportunity for shared learning has the potential to greatly impact DV organization policy regarding landlords who are frequently a substantial roadblock in finding stable and safe housing for DV survivors.
Ultimately, our analysis of the WI-DVHF will enhance our understanding of DV and the provision of DV services to survivors. By comparing 9 WI-DVHF sites as well as 4 control sites, we will be able to collect evidence about the impact of a flexible, survivor-focused program designed to secure safe and stable housing for people leaving abusive situations. Conducting this analysis has the potential to encourage widespread changes into how we provide assistance to DV survivors in Wisconsin and beyond.
Footnotes
Acknowledgements
Thanks are due to Makenzie Peake and Nicole Meyer for excellent research assistance.
Abbreviations
DCF: Wisconsin Department of Children and Families
DV: Domestic Violence
DVHF: Domestic Violence Housing First
FFA: Flexible Financial Assistance
HF: Housing First
WI-DVHF: Wisconsin Domestic Violence Housing First Pilot Program
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding was provided by the Wisconsin Department of Children and Families ($337,500) and a Wisconsin Idea Collaboration Grant ($59,737).
Ethics Statement
This study, including the participant consent process, was approved by the University of Wisconsin-Madison Institutional Review Board on 7/11/22 (submission number 2022-0791).
