Abstract
Aims and context:
This paper reports the evaluation of the Doorway program (2015-18) in Melbourne, Australia. Doorway extends the original Housing First (HF) model in providing housing support to people with precarious housing at-risk of homelessness with Serious and Persistent Mental Illnesses (SPMIs) receiving care within Victoria’s public mental health system. Doorway participants source and choose properties through the open rental market, and receive rental subsidies, assistance, advocacy and brokerage support through their Housing and Recovery Worker (HRW). The aim of this study is to estimate Doorway’s impact on participants’ housing, quality of life and mental health service use.
Methodology:
The study employed a a quasi-experimental study design with a comparison group, adjusted for ten potential confounders. The primary outcome measure was days of secure housing per participant. Secure housing status, health service usage and quality of life (HoNOS) data were extracted from participants’ electronic hospital and Doorway records in deidentified, non-reidentifiable form. Analysis for continuous outcome variables was based on multivariate GLM modelling.
Results:
Doorway housed 89 (57%) of 157 accepted participants. The 157 Doorway participants overall were also housed for significantly more days (119.4 extra days per participant) than control participants, albeit after some delay in locating and moving into housing (mean 14 weeks). There was a significant, positive Doorway effect on health outcomes (all and one dimension of the HoNOS). Doorway participants had significantly reduced length of stay during acute and community hospital admissions (7.4 fewer days per participant) compared with the control group.
Conclusion:
The Doorway model represents a new and substantial opportunity to house, enhance health outcomes and reduce mental health service use for people with SPMIs from the public mental health sector and at-risk of homelessness.
Keywords
Introduction
This paper reports on the evaluation of the Doorway programme in Melbourne, Australia. Doorway extends the Housing First (HF) model which was pioneered in the early 1990s by Tsemberis and his Pathways to Housing programme in New York City (Tsemberis et al., 2004). HF is an evidence-based intervention model that involves the immediate provision of permanent housing and wrap-around supports to individuals who are homeless and living with serious mental illness, rather than traditional ‘treatment then housing’ approaches. Baxter et al. (2019) conducted a systematic review and meta-analysis of HF approaches. The authors included four studies, in all of which, intervention participants spent more days housed and were more likely to be housed at 18–24 months. Intervention groups experienced fewer hospitalisations and less time spent hospitalised, as well as fewer emergency department visits than control groups. No clear difference in substance use was seen.
The Doorway programme provides care to people with Severe and Persistent Mental Illnesses (SPMIs) and precarious housing who are at immanent risk of becoming homeless, referred from the public mental health sector where there are low levels of absolute homelessness. This is in contrast to previous HF studies of people with very high levels of absolute homelessness, referred from the homeless persons sector (from the streets or emergency housing shelters). This was estimated at 82% absolutely homeless referred from the streets and emergency homeless shelters in the Chez Soi study (Goering et al., 2014). The former group has not been well studied, usually being combined in populations mainly consisting of people who are homeless (Baxter et al., 2019). This group with precarious housing referred from the public mental health sector involves a large number of people. In Australia (population, 23.8 million in 2015), it was estimated there were 59,000 adults (0.4%) with a SPMI who had complex, multi-agency needs including housing. Forty percent or 23,600 of these in this complex needs group can be identified as at risk of being homeless (Whiteford et al., 2017). This number greatly exceeds the number of people with SPMIs who had experienced homelessness (estimated in this study as being 5% or 2950 for SPMI with psychoses, no-figure for SPMIs without psychoses, but unlikely to be greater than a further 5% or 2950).
Other rationales for this paper are that there remain some unanswered questions about the impacts of HF programmes. First, all four trials reported in a recent systematic review of HF impacts on homeless or at risk of homelessness were conducted in Canada or the United States, limiting the generalisability of the review’s results outside North America (Baxter et al., 2019; Stergiopoulos et al., 2015). Second, while this systematic review consistently demonstrated improvements in housing stability and reductions in health service usage, it concluded there should be more investigation on long-term health outcomes. Recently established programmes like the Doorway programme in Victoria, Australia, can contribute to this.
The Doorway programme
The Doorway programme is an integrated housing and recovery support programme delivered by Wellways, a non-governmental, not-for profit agency in Victoria, Australia, providing care for people with SPMIs with precarious housing (Dunt et al., 2017; Nous Group, 2014), on referral from the Victoria public mental health sector. Doorway demonstrates fidelity to almost all of the characteristics of the HF model (see Table 1) using a scale developed to assess this (Gilmer et al., 2013a, 2013b). A limitation of the Doorway programme as an HF programme is that it offers accommodation up to 18 months only, not ongoing secure housing as in other HF programmes.
Doorway’s fidelity to the HF Model based on Pathways HF Partnerships analysis (Gilmer et al., 2013a, 2013b).
HF: Housing First.
Required only in situations where they are danger to themselves or others.
Programme provides a number of options for this activity.
Doorway is different to most other HF programmes as it is designed to offer services to people with SPMIs as well as precarious housing on referral from the public mental health system. Ninety-five percent of its participants had precarious housing and only 5% lived on the street – see Results section for further details.
The Doorway model, like a number of HF programmes, e.g., Chez Soi, provides support to participants to source their property of choice. They do this in Doorway using inexpensive properties available on the open private rental market. Housing and Recovery Workers (HRWs) at Doorway are responsible for implementing the model on a day-to-day basis, on behalf of an integrated team. HRWs have a role similar to Intensive Case Management (ICM) in the Chez Soi trial in Canada. This provides support to people with more moderate needs than people requiring Assertive Community Treatment (ACT).
Doorway also provides rental subsidies, advocacy and brokerage support, varying between A$464 and A$6038 per participant across clinical partner regions. These wide differences in rental subsidies reflect the differences in rents in high-demand inner metropolitan and disadvantaged rural areas. Their role combines housing and (non-clinical aspects) of mental health recovery work. HRWs sit, co-located at a partnered clinical service with a team that was already providing ongoing clinical support. Their dual roles are unusual as typically their activities have been divided between a Home-Based Mental Health Worker and a Housing Worker. More information about Doorway and the HRW role is included in Supplemental Material 1.
The control programme offers ‘Treatment as Usual’ available for all people in receipt of public mental health care in Victoria. It does not offer subsidies and services only available under the Doorway programme.
The aim of this study then is to estimate Doorway participants’ outcomes for housing, health and mental health service use for people with SPMIs and precarious housing, referred from the public mental health system.
Methods
Study design
It comprises a quasi-experimental study with a comparison group, adjusted for 10 potential confounders including housing at start date [as proxy for homelessness], age group, gender, employment status, education status, pension, country of birth, indigenous status, preferred language, SPMI diagnosis and days on the programme.
Secure housing status, health service usage and health outcome (Health of the Nation Outcome Scales [HoNOS]) data were extracted from participants’ electronic hospital and Doorway records in deidentified, non-reidentifiable form. Where possible, data from all Doorway Partners (or equivalent in control group) were collected for 12 months pre-entry, entry, 6-monthly throughout the Doorway period and finally at exit. Hospital records are the same statewide in Victoria and thus do not differ between Doorway and control group.
To be eligible for entry to the study, individuals needed to be aged between 18 and 65 years; to have an SPMI; to be homeless/at risk of being homeless whose housing was precarious and at immanent risk of becoming homeless as identified by the clinical partner health services for the Doorway group and equivalent for the control group; to be on referral from and in current receipt of public mental health care from Doorway’s clinical partners or equivalent in control group; and to be case managed. Operationally, they needed to have had contact with their health service after 30 June 2014 with some of that contact being between 1 July 2015 and 30 June 2018 (the Doorway study period) – see Figure 1.

Data collection flow diagram.
The study group (n = 157) was drawn from four clinical Mental Health Services, three in metropolitan Melbourne – St Vincent’s Hospital Melbourne, the Alfred Hospital and the Peninsula Hospital – and one in regional Victoria – the Latrobe Regional Hospital. Eligible participants entered the study on their acceptance by the programme. The control group (n = 80) was drawn from Austin Health, a metropolitan mental health service geographically adjacent to that of St Vincent’s (n = 63). A further 17 controls were drawn from the Latrobe Regional Hospital (completely separate from the drawing of Doorway participants).
The primary outcome measure was days of secure housing per participant during the study period (for the whole Doorway programme and control programme, n = 157 and n = 80, respectively). Secure housing was defined as private rental units either paid for entirely by the person concerned, or subsidised and supported by Doorway (or equivalent agencies in the catchment area of the Latrobe Hospital for the control group). Non-secure/independent housing included all other arrangements such as rooming houses, supported residential services, caravan parks, living with family, couch surfing, etc. It also included Prevention and Recovery Care (PARC) (short-term up to 30 days) and Community Care Units (CCUs) (long-term – up to 2 years, though sometimes more), both of which being forms of bed-based mental health care. There were no participants in social or community housing.
The HoNOS constituted the principal secondary outcome measure. The HoNOS scale comprises 12 items which can be combined into four subscales (Behaviour, Impairment, Symptoms and Social) and an Overall score. The lower the scores, the less severe are the problems (Pirkis et al., 2005). These data were recorded in the participant’s clinical record by a member of their care team.
Other descriptive and potentially confounding variables are set out in Table 2.
Comparison of characteristics of participants of the Control and Doorway groups (% include missing data).
CCU: Community Care Unit; PARC: Prevention and Recovery Care; TAFE: Technical and Further Education; CRCU: Community Residential Care Unit; SRS: Supported Residential Service.
Other includes: Building, Caravan, Garage, Motel, Transitional property, Residential rehab, Other Unspecified.
Service usage (Doorway and the control groups)
These include mental health service usage – mental health bed–based number of days; whether Acute, Subacute (PARC) or Ongoing (CCU); mental health presentations at ED; and contacts with community mental health services staff (number of contacts by the length of contact). These data were extracted from the participant’s clinical record. It does not include GP visits, pharmaceutical use or ambulance call-outs.
Supplemental Material 2(a) and (b) reports data that were collected on Doorway participants only:
some other psychometric health outcome variables;
episodes of family violence; level of contact with the legal system; employment; ongoing training;
some additional qualitative data.
Data analysis
We estimated that 63 in each group would be sufficient to detect 100 fewer days of secure housing in the control group compared with Doorway, assuming level of significance of 5%, power of 80% and secure days of housing in Doorway as 200 days per participant (standard deviation, 200 days).
Participants’ mental health service usage data and HoNOS scores were measured from 12 months pre-entry, until exit. Participants entered and exited the programme at different time points throughout the study. To aid the analysis, a Set date (date of intake) as well as 6-monthly time periods and 6-monthly time points in relation to the Set date were established for each participant – see Supplemental Material 3.
Analyses for continuous outcome variables were based on multivariate general linear model (GLM) modelling since these models allow data on people with missing data to be used, as long as the missing data met the ‘missing-at-random’ definition. They could also handle uneven spacing of repeated measures (which still occurred in the presence of averaging) and could be extended as a GLM model to non-normal distributions of outcomes (Seltman, 2018; SPSS, n.d.).
The Group × Time Period (or Group × Time Point) interaction term was used to measure the Doorway effect using indicator levels at pre-/at entry and post-entry. To minimise bias and confounding, analysis controlled for 10 potential confounders including length on the programme. These consisted of the same initial set of 10 variables (as described above) deleting sequentially, in individual analyses, those variables with an F value of ⩽0.200, as it was found that removal of these variable had little effect on the Rsq value of the model and slightly improved the adjusted Rsq value (SPSS, undated). Cohen’s d was used to estimate effect size. Dichotomous outcome variables were analysed using logistic regression. Odds ratios were used to estimate effect size.
A Holm–Bonferroni correction was used to take into account multiple statistical testing with p for significance set at 0.05/number of relevant hypotheses (Haynes, 2013). This correction evenly balances the avoidance of Type 1 and 2 errors. This is more suitable for the evaluations of social programmes than the Bonferroni correction which is more oriented to avoiding Type 1 than Type 2 errors, more appropriate in pharmaceutical trials. An outline of the method used is included in Supplemental Material 4.
Ethics approval for the evaluation was granted by the St Vincent’s Hospital Melbourne (SERP HREC reference number HREC/16/SVHM/241) on behalf of all participating hospitals. Site Specific Approvals were also received from all participating health services. Doorway participants had to provide informed consent for the collection of their deidentified health service usage, as they formed the study population for the interview study (see above) (81 of 157 providing consent). The 80 control participants did not need to provide informed consent, as they were not interviewed.
Results
Doorway participants (n = 157) were mainly people:
(64.3%) who had moved between various forms of temporary shelter including staying with friends; family (albeit where difficulties and conflict were present); or in emergency accommodation, youth refuges and hostels (64.3%);
experiencing accommodation that fell below community expectations, e.g., boarding houses or caravan parks (17.8%);
who were ‘marginally housed’, i.e., living in accommodation that was inadequate as there was, e.g., no security of tenure or was overcrowded (12.7%);
who were living on the street, in deserted buildings or improvised dwellings, under bridges and in parks (5%).
Their housing was also unstable. In the 3 years prior to their intake into Doorway, they had an average number of four moves. Only a small number had had no moves. Ten percent had 10 or more moves.
The mean age of participants’ ages was 34.7 years. Almost 60% were male; 80% were born in Australia. Ten participants (6.4%) identified as Indigenous. English was overwhelmingly the preferred language (97.5%). Just under 60% of the participants had only a high school education. At entry, 109 (69%) were in receipt of the Disability Support Pension (DSP), 41 (26%) were on a Newstart allowance (unemployment benefits); 6 (4%) were employed. At acceptance into the programme, 94 participants (59.9%) had a diagnosis of psychosis, 48 (30.6%) an affective or mood disorder (i.e. depression, anxiety and bipolar disorder), and 15 (9.6%) a personality disorder.
As well as having a SPMI, participants had a range of other issues at intake. These issues, often occurring together, most commonly related to alcohol and other drugs (53%), family conflict and difficulties (34%), social isolation (26%) and physical health (26%). Twenty-two participants were on Community Treatment Orders. Some other characteristics of the Doorway participants including levels for a number of psychometric measures at entry are set out in Supplemental Materials 2 and 5.
A comparison of the characteristics of study and control group participants is shown in Table 2. The only major difference in the demographic data was that, compared to the control group, lower proportions of the Doorway group were in the 18–20 year age group and the 50+ age groups (p = 0.016). There were major differences in Education status and Pension status between the two groups; however, the amount of missing data in the control group (37.5% and 36.3%) made it difficult to draw any firm conclusions. There was also a major difference in the type of housing occupied at Start date with (1) lower proportions of the Doorway group being of no fixed address or living in a house or flat and (2) higher proportions of Doorway participants couch surfing or living/staying with family. Effects of differences in these variables were controlled in multivariate analysis.
Primary outcome – secure and independent housing
Of the 157 Doorway participants, 89 were in secure, independent housing for a total of 30,809 days, an average of 198.8 days per participant (66.3 days per Doorway participant per year). This was shorter than the maximum duration of participation in Doorway (18 months). In part, this was due to some participants leaving the programme before being housed (the average length of time for participants housed, to find appropriate housing was 14 weeks and the median 11 weeks). In part, it was due to some participants, after being housed, leaving the programme before 18 months.
Of the 80 control participants, an estimated 16 participants had secure housing. The Doorway population overall (n = 157) was housed for 30,809 days (mean 198.8, SD 216.0) compared with 6199–6515 days in the control group overall (mean 77.5–81.4 days per participant) – see Table 3. The Doorway population was therefore housed 119.4 extra days (midpoint estimate) per participant. Expressed differently, 51.6% and 17.5% of Doorway and control group participants were housed for 3 months or more, respectively; 45.9% and 13.8% of Doorway and control group participants were housed for 6 months or more, respectively.
Days of secure, independent housing for participants in the control and Doorway groups.
SD: standard deviation.
Because of missing data for four of the control group records, it was not possible to calculate the exact number of days in secure/independent housing for the control group. To overcome this, two approximations were used for the missing data: a ‘High’ estimate used the average number of secure/independent housing days of the Austin control group (319 days) and a ‘Low’ estimate used the median number of secure/independent housing days for the Austin control group (219 days).
After controlling for the effects of potential confounders, the difference in the two groups (Group) was statistically significant (p < 0.001), d between 1.09 and 1.12 (see Table A1 in Supplemental Material).
Service usage
Mental health admissions – acute and community bed–based length of stay
After adjusting for independent variables, GLM analysis indicated that the Group × Time Period interaction (Doorway effect) on length of stay (LOS) was significant (F = 3.178, p = 0.013) (Table 4) (see Table A2 in Supplemental Material). The Doorway Group had a greater reduction in LOS of 7.4 days per participant compared with the control group.
Mental health bed–based service use for External control group and Doorway participants.
First 6 months in the programme for Doorway participants.
ED presentations
The Doorway group had 0.28 fewer presentations per participant (Table 5). After adjusting for independent variables, GLM analysis indicated that Group × Time Period interaction (Doorway effect) was not a significant predictor of the number of presentations (F = 2.667, p = 0.031) compared with the control group.
HoNOS unadjusted scores for Doorway and External control group participants pre-entry versus at entry/post-entry (Control = 80; Doorway = 157).
Lower scores indicative of less severe problems.
Community health services contacts
Doorway had a reduction of 0.05 days per participant. After adjusting for independent variables and compared with the control group, GLM analysis indicated that the Group × Time Period (Doorway effect) was not significant.
Mental health and well-being
HoNOS
The impact of Group × Time Point (Doorway effect) (Pre-entry/Entry vs Post Entry) (× 3) on HoNOS scores (after adjusting for independent variables) showed that Doorway had positive effects on HoNOS scores – Social subscale (p = 0.000, d = 0.55) and Overall scale (p = 0.002, d = 0.42) (Table 5) (see Table A3 in Supplemental Material).
Other findings
Results of psychometric tests and interviews only conducted in the Doorway population are reported in Supplemental Materials 2(a) and (b) and 6.
Discussion
Doorway securely housed 119.4 extra days per participant compared with the control group in this group of people with SPMIs who were at risk of homelessness. This study also reported a significant, positive effect on health outcomes of Doorway participants as judged by their HoNOS total and Social subscale scores as well as reductions in mental health bed–based.
While Doorway shows significantly better housing outcomes than the control group (119.4 days per participant), the mean number of days in housing (66.3 days per year) and percentage of Doorway recipients in secure independent housing (86 of 157 participants) is lower than that achieved in other HF programmes previously reported. For example, in the last 6 months of the Chez Soi study, 62% of HF participants were housed all of the time, 22% some of the time and 16% none of the time. The comparable figures for Treatment as Usual participants were 31%, 23% and 46%.
Two reasons for this suggest themselves. First, the nature of property selection and occupancy (participant-led in the private rental market) by its nature can lead to considerable delays and also participant drop-out. Second, participants from the mental health sector with SPMIs with very low levels of absolute homelessness may react quite differently to participants from the homeless services sectors where there are much higher rates of absolute homelessness. In other words, the need for new housing from the public mental health sector is much less pressing.
The improvements in health outcomes (HoNOS) in Doorway participants are welcome. These are not consistent findings in previous studies (Baxter et al., 2019) though such results have been previously reported (Goering et al., 2014).
To our knowledge, this is the first controlled study of a programme for people with SPMIs, with precarious housing and referred from the public mental health sector where there are very low levels of rough sleeping, both in Australia and internationally.
There has been little study of people with SPMIs with precarious housing and from the public mental health sector. The recent systematic review of HF impacts on people who are homeless or at risk of being homeless (Baxter et al., 2019) reported findings on four studies. All four however did not disaggregate groups being homeless and those with precarious housing and at risk of being homeless. Among these four studies, the Pathways Housing First studies (Gulcur et al., 2003; Tsemberis et al., 2004) included no participants in the at risk of being homeless group but did include participants with mental health disorders. The At home/Chez Soi included a proportion of people from their Precariously Housed category but not their Relatively homeless category groupings. Both categories would have needed to be included to properly cover the at risk of being homeless group. The study did focus specifically on people with mental health disorders (Aubry et al., 2016; Chung et al., 2018; Stergiopoulos et al., 2105). Neither of the other two studies (the Housing Opportunities for Persons with AIDS [Wolitski et al., 2010] nor the Chicago Housing for Health Partnership) focused on people with SPMIs (Buchanan et al., 2009; Sadowski et al., 2009).
This group (precariously housed people with SPMIs referred from the public mental health care sector) is much larger than the ‘rough sleepers’ group, at least in Australia. Their overall quality of life potentially is likely to be much improved if programmes like Doorway were widely available.
The study had some strengths and some weaknesses. A strength is that the study employed an external control group and also included data for participants, prior to entry. A randomised control trial was not possible under the programme’s funding arrangements or conditions. It suffers from missing data, arising from the study’s use of routine public mental health clinical and service usage data. It is unlikely in our view that primary data collection would have been more complete. Dropout bias remains a possibility despite strategies to minimise this. No data on participants could be collected after exit from Doorway (or equivalent in control group). A limitation of Doorway programme as an HF programme is that accommodation, under the terms of Doorway’s funding is limited to 18 months, and is not ongoing, unlike other HF programmes. The effect of this is to diminish longer-term gains.
Policy implications
The model based on subsidising and supporting people with SPMIs in the public mental health sector, experiencing precarious housing, to find accommodation at the cheaper end of the private rental market can be readily scaled up with advantage to other regions in Victoria and Australia. There are no capital costs to government substantially reducing overall governmental costs.
Scalability will be strongly influenced by the availability of affordable, low-demand rental properties within and between regions. This is very variable – in some regions cheap rental properties are numerous, in others not. But even when they are not, the Doorway experience demonstrates that properties can be found to house local participants with SPMIs and at risk of homelessness (Department of Health and Human Services, 2019).
On separate points, first, it would improve days of secure housing per participant if waiting times to enter housing were reduced (currently averaging between 12–16 weeks). A strategic alliance with a provider of short-term, immediate housing would be one way of doing this. Second, supported by a senior clinical manager, the HRWs seem well suited to provide day-on-day support to participants, e.g., for inspection of prospective properties to rent as well as ongoing support. Their role is non-clinical rather than clinical, as is common in HF programmes offering either ACT or ICM care. This arrangement was appropriate with Doorway being added to a team of workers already led by a case manager, providing the participant’s care within the Victorian public mental health system. There is no reason though why HRWs could not add clinical care responsibilities to their role as long as day-to-day support to the participant continued in some other way.
Conclusion
The results of this study extend findings about positive outcomes, including health-related outcomes, for the HF model for people with SPMI and precarious housing which are referred from the public mental health sector. Results are broadly in line with those in the recent systematic review (Baxter et al., 2019). The study also adds to the generalisability of the review’s results outside North America.
Supplemental Material
sj-docx-1-anp-10.1177_00048674211011702 – Supplemental material for Evaluation of a Housing First programme for people from the public mental health sector with severe and persistent mental illnesses and precarious housing: Housing,health and service use outcomes
Supplemental material, sj-docx-1-anp-10.1177_00048674211011702 for Evaluation of a Housing First programme for people from the public mental health sector with severe and persistent mental illnesses and precarious housing: Housing,health and service use outcomes by David Robert Dunt, Susan Elizabeth Day, Laura Collister, Beth Fogerty, Rosie Frankish, David J Castle, Cayte Hoppner, Simon Stafrace, Sharon Sherwood, J Richard Newton and Suzy Redston in Australian & New Zealand Journal of Psychiatry
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The first and second author received funding to undertake this study from Wellways, the organisation delivering Doorway. This funding in turn was derived from the Victorian Government Department of Health and Human Services as part of its funding of Wellways for the Doorways programme.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The first and second author received funding to undertake this study from Wellways, the organisation delivering Doorway. This funding in turn was derived from the Victorian Government Department of Health and Human Services as part of its funding of Wellways for the Doorways programme.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
