Abstract
This Campbell systematic review examines the impact of reducing the maximum duration of unemployment benefits on job-finding rates. Seven studies were included in the review, all of which are from European countries.
Included studies had to examine the effect of a reduction in the maximum duration of entitlement of any kind of unemployment benefits on employment using a well-defined control group.
Whilst 41 studies were identified, after allowing for study quality and data issues, only seven studies were included in the review. The included studies covered Austria (2 studies), France, Germany (3 studies) and Slovenia. Maximum entitlement ranged between 26 and 209 weeks. The studies analyzed reductions between 9 and 179 weeks, with an average of 43 weeks. The studies analyze data from 1,154,090 unemployment spells.
Reducing the duration of unemployment benefits increases the exit rate from unemployment. Data from seven studies show that the exit rate from unemployment for those with reduced duration of benefit entitlement on average is 10 per cent. This corresponds to a 52% chance that those with reduced duration will find a job before an unemployed person with the existing, longer duration (no effect corresponds to a 50% chance).
There is not enough evidence to determine effects on the exit rate from re-employment or on the wage rate in the job found. There are insufficient high-quality studies to allow an examination of variation of effects.
Plain language summary
Reducing the maximum duration of unemployment benefits increases the job finding rate of the unemployed
Reducing the maximum duration of unemployment benefits is one strategy used to reduce unemployment. Evidence from seven studies confirms such an effect. However, the effect is small and more studies of higher quality are needed to give more detailed findings to inform policy.
The review in brief
Evidence from seven studies shows shortening the maximum duration of unemployment benefit entitlement has a small effect on the job finding rate of the unemployed.
What is this review about?
Policymakers may wish to reduce the generosity of the unemployment benefits system in order to reduce unemployment levels. Reducing benefit levels may be politically more difficult than shortening the length of the unemployment benefit eligibility period to create work incentives for the unemployed.
This review summarizes studies that measure the effects of shortening the maximum duration of unemployment benefit entitlement on job finding rates.
What is the aim of this review?
This Campbell systematic review examines the impact of reducing the maximum duration of unemployment benefits on job-finding rates. Seven studies were included in the review, all of which are from European countries.
What are the main findings of this review?
What studies are included?
Included studies had to examine the effect of a reduction in the maximum duration of entitlement of any kind of unemployment benefits on employment using a well-defined control group.
Whilst 41 studies were identified, after allowing for study quality and data issues, only seven studies were included in the review. The included studies covered Austria (2 studies), France, Germany (3 studies) and Slovenia. Maximum entitlement ranged between 26 and 209 weeks. The studies analyzed reductions between 9 and 179 weeks, with an average of 43 weeks. The studies analyze data from 1,154,090 unemployment spells.
What are the main results of this review?
Reducing the duration of unemployment benefits increases the exit rate from unemployment. Data from seven studies show that the exit rate from unemployment for those with reduced duration of benefit entitlement on average is 10 per cent. This corresponds to a 52% chance that those with reduced duration will find a job before an unemployed person with the existing, longer duration (no effect corresponds to a 50% chance).
There is not enough evidence to determine effects on the exit rate from re-employment or on the wage rate in the job found. There are insufficient high-quality studies to allow an examination of variation of effects.
What do the findings of this review mean?
On the basis of this limited number of studies, shortening the maximum duration of unemployment benefit entitlement has a small effect on the job finding rate of the unemployed. Whether unemployed workers responding to a shorter potential benefit entitlement may be worse off, in the sense that they accept “lower quality” jobs, has not yet been fully investigated.
But the review finds a surprisingly low number of studies with a sufficiently low risk of bias to be used for synthesis to determine the effect size of shortening the maximum duration of unemployment benefit entitlement. Many studies had to be excluded as they had a high risk of bias. This is a finding in its own right.
There is a need for future studies to more thoroughly discuss the assumptions of the study design and justify the choice of method by considering and reporting all relevant data and tests. Future studies should also use data with all relevant information, in particular, information on whether eligible individuals actually received unemployment benefits and information on individual maximum entitlement duration.
How up-to-date is this review?
The review authors searched for studies published up to December 2016. This Campbell Systematic Review was published in February 2018.
Executive Summary/Abstract
BACKGROUND
Unemployment benefit programmes protect individuals against loss of income and provide unemployed individuals with the possibility of finding a better match between their qualifications and job vacancies. However, unemployment benefits may also distort incentives by subsidizing long and unproductive job searches. In order to reduce unemployment levels, policymakers may wish to reduce the generosity of the unemployment system. While it may be politically intractable to lower the monetary amount of unemployment benefits available, the length of the unemployment benefit eligibility period is often used as a political instrument to create work incentives for the unemployed. If a shorter benefit period results in a significantly increased incentive for finding work, shortening the benefit eligibility period may reduce the share of long and unproductive job searches and thereby decrease the overall unemployment level.
OBJECTIVES
The purpose of this review is to systematically uncover relevant studies in the literature that measure the effects of shortening the maximum duration of unemployment benefit entitlement on job finding rates, and to synthesize the effects in a transparent manner. As a secondary objective we will, where possible, investigate the extent to which the effects differ among different groups of unemployed people, such as those with high/low levels of education or men/women, and further explore from which point in the unemployment spell unemployed individuals react to the length of benefit entitlement.
SEARCH METHODS
The search was concluded in March 2016. Relevant studies were identified through electronic searches of bibliographic databases, government policy databanks, internet search engines and hand searching of core journals. We searched to identify both published and unpublished literature. The searches were international in scope. Reference lists of included studies and relevant reviews were also searched.
SELECTION CRITERIA
The intervention of interest was a reduction (change) in the maximum duration of entitlement of any kind of unemployment benefits. We included unemployed individuals who received any type of time-limited benefit during their unemployment spell. All study designs that used a well-defined control group were eligible for inclusion in this review. Studies that utilised qualitative approaches were not included in the review due to the absence of adequate control group conditions.
DATA COLLECTION AND ANALYSIS
Random effects models were used to pool data across the studies. We used the point estimate of the hazard ratio. Pooled estimates were weighted with inverse variance methods, and 95% confidence intervals were used. A sensitivity analysis was performed to evaluate whether the pooled effect sizes were robust across components of methodological quality, in relation to the quality of data and whether the study analysed an extension of entitlement duration.
RESULTS
The initial search for potentially relevant studies resulted in a total number of 34,930 hits. A total of 41 studies, consisting of 66 papers, from 15 different countries, met the inclusion criteria and were vetted by the review authors. Only 38 studies provided data that permitted the calculation of an effect size for the primary outcome. Of these 38 studies, 28 studies could not be used in the data synthesis due to a too high risk of bias. A further 3 studies could not be used in the data synthesis due to overlapping of data samples. As a result, only 7 studies were included in the data synthesis and one of these studies only provided results on the secondary outcome. In total, 6 studies provided data that permitted the calculation of an effect size for the primary outcome and 3 studies provided data that permitted the calculation of secondary outcome. The sample size used in the studies ranged from 5,017 spells of unemployment to 509,355 spells. The total number of unemployment spells was 1,154,090, implying an average sample size of 164,870 spells of unemployment per study.
The seven studies covered Austria, France, Germany and Slovenia. There was a high degree of variation in maximum entitlement, ranging between 26 and 209 weeks. On average the studies analysed a reduction of 43 weeks in maximum entitlement; the smallest being a reduction of 9 weeks and the largest a reduction of 179 weeks. Four studies restricted the analysis to a specific age group and three studies restricted the analysis to specific work experience levels. All studies used non-randomised designs. In the majority of studies the risk of bias was high.
This review found a statistically significant effect of shortening the maximum duration of unemployment benefit entitlement. The overall impact of shortening the maximum duration of unemployment benefit entitlement, obtained using hazard ratios, was estimated at 1.10 (95% CI 1.03 to 1.17, p=0.0005), which translates into an increase of approximately 10% in the exit rate from unemployment into employment and corresponds to a 52% chance that a treated unemployed person will find a job before a non-treated unemployed person.
Thus, although small, the available evidence associated with a sufficiently low risk of bias supports the hypothesis of an incentive effect of shortening the maximum duration of unemployment benefit entitlement. There was a lack of evidence to conclude that shortening the maximum duration of unemployment benefit entitlement has an impact on the quality of the job obtained.
Only three studies provided data on the exit rate from re-employment and three studies provided data on the log wage ratio in the re-employment job.
The overall impact of shortening the maximum duration of unemployment benefit entitlement on the exit rate from the re-employment job, obtained using hazard ratios, was 0.99 (95% CI 0.97 to 1.02, p=0.64) and the overall wage ratio was 1.00 (95% CI 0.99 to 1.01, p=0.089).
We did not find any adverse effects.
Sensitivity analyses resulted in no appreciable change in effect size, suggesting that the results are robust. The limited number of studies used in the meta-analysis should, however, be considered when interpreting the results.
Due to having an insufficient number of studies available for moderator analysis to be performed, it was not possible to examine whether the effect of reducing the maximum potential benefit duration on job-finding differs for men and women, for particular age groups or educational groups, or if factors such as good or bad labour market conditions, the type of unemployment benefit, the availability of alternative benefits, or whether compulsory activation is part of the institutional system, have an impact on the effect.
AUTHORS' CONCLUSIONS
To the best of our knowledge, this is the first systematic review analysing the magnitude of the effect of shortening the maximum duration of unemployment benefit entitlement on the job finding rate. The review finds a surprisingly low number of studies with a sufficiently low risk of bias to enter a synthesis of the effect size of shortening the maximum duration of unemployment benefit entitlement. On the basis of this limited number of studies, shortening the maximum duration of unemployment benefit entitlement displays a limited potential for altering the employment prospects of the unemployed individuals. The available evidence does suggest an effect on the job finding rate of shortening the maximum duration of unemployment benefit entitlement, but the effect is small. Further, whether unemployed workers responding to a shorter potential benefit entitlement may be worse off, in the sense that they accept “lower quality”” jobs, has not yet been fully investigated.
The number of studies used in the data synthesis (7) is relatively low compared to the large number of studies (41) meeting the inclusion criteria for the review. The reduction in studies eligible for inclusion in the data synthesis was primarily caused by a judgment of too high a risk of bias. Thus, the process of excluding studies with too high risk of bias from the meta-analysis applied in this review left us with only seven studies to synthesize. This is a finding in its own right, entailing important information for stakeholders on the degree of confidence to place on the expected gains from changing the maximum potential unemployment benefit duration; fewer studies with too high risk of bias would have provided a more robust literature on which to base conclusions. There is a need for future studies to more thoroughly discuss the identifying assumptions of the study design and justify the choice of method by considering and reporting all relevant data and tests. Further, future studies should rely on data where all relevant information is available, in particular information on whether eligible individuals actually received unemployment benefits and information on individual maximum entitlement duration.
1 Background
1.1 THE PROBLEM, CONDITION OR ISSUE
Benefit programmes protect individuals against loss of income and provide unemployed individuals with the possibility of finding a better match between their qualifications and job vacancies. This positive aspect of inducing workers to achieve better job matches has been shown, theoretically, to potentially increase economic efficiency (Acemoglu & Shimer, 1999; Marimon & Zilibotti, 1999).
However, unemployment benefits may also distort incentives by subsidizing long and unproductive job searches. In fact, the generosity of unemployment benefits is generally considered the main factor by which benefit systems affect unemployment. From a societal point of view, therefore, the optimal unemployment benefit system will balance considerations for protection with those for distortion (Feldstein, 2005; Mortensen, 1987).
Theory suggests that putting a limit on benefit duration will tend to accelerate job search from the beginning of the unemployment spell and thereby shorten unemployment duration (Pissarides, 2000). Thus, generosity of benefits is determined not only by the amount paid but also by the duration of benefit entitlement. In the US, replacement rates 1 are low and duration is short compared to benefit systems in most European countries. In 2005, the maximum duration of unemployment insurance entitlement among OECD countries 2 was shortest in the US at 6 months 3 and longest in Denmark, Norway, Portugal, the Netherlands, France, Finland and Spain, varying between 23 and 48 months (OECD, 2007). At the same time, the gross initial replacement rate was around 50% in the US, while varying between 62% and 90% in the aforementioned European countries.
The lower level of generosity of benefits in the US compared to Europe is consistent with the observation of higher levels of active searches and a greater willingness to accept inferior jobs by unemployed workers in the US compared to Europe (Layard, Nickell & Jackman, 2005). As a consequence, European policy-makers may consider reducing the generosity of unemployment systems in order to reduce high unemployment levels 4 . While lowering the replacement rate may be politically intractable (indeed, examples of reductions of benefit rates and amounts are rare), the length of the unemployment benefit entitlement period is often used as a political instrument to improve work incentives for the unemployed. In Spain, for example, the benefit period was altered in 1992, in Slovenia in 1998, in Norway in 1997, in the UK in 1996, in Denmark in 1996, 1998 and 1999, and, more recently, in the Czech Republic in 2004, in Hungary and Portugal in 2006, and in Denmark again in 2010.
A crucial public policy question is whether a more generous unemployment benefit system is causally related to higher unemployment rates. As pointed out in Card and Riddell (1993), there are several complementary potential explanations for differences in unemployment rates between countries, including differences in the overall distributions of working and nonworking time, and differences in the fraction of nonworking time that is reported as unemployment (particularly among individuals with very low levels of labour supply). Recent research on the effect of extended duration of unemployment insurance benefits in the US shows that benefit extensions raised the unemployment rate, but at least half of the effect is attributable to reduced labour force exit among the unemployed rather than to the changes in reemployment rates that are of greater policy concern (Rothstein, 2011).
This review focuses on the effect on job finding rates of reducing the maximum duration of entitlement of unemployment benefits, and secondarily on the effects on the quality of these re-employment jobs.
1.2 THE INTERVENTION
The intervention of interest is reduction 5 in the maximum duration of entitlement of any kind of unemployment benefit with a known expiration date. The benefits may be unemployment insurance (UI) benefits or unemployment assistance (UA)/social assistance (SA) benefits, as long as they have a known expiration date.
In the majority of OECD countries, the UI benefit has a time limit. In fact, only Belgium has an unlimited UI period. In other countries, the maximum duration varies between 6 months (as for example in the UK and the US) and 36 months (in Iceland).
In most OECD countries, a secondary benefit is available for those who have exhausted regular UI benefits. These are known as SA benefits. Unlike UI benefits, SA benefits are generally means-tested without any necessary connection to past employment; they pay a lower level of benefit and are indefinite. We know of only one example of an SA benefit with a time limit: the Temporary Assistance to Needy Families (TANF) which is available in the US. The federal government requires states to impose between 2- or 5-year limits on TANF (Gustafson & Levine, 1997). In a minority of OECD countries, UA benefits are paid after exhaustion of UI benefits. Like SA benefits, they are generally means-tested, pay a lower level of benefits and, excepting Hungary, Portugal and Sweden, are indefinite.
Unemployment benefits with an indefinite time limit or non-financial benefits are excluded from this review.
1.3 HOW THE INTERVENTION MIGHT WORK
Search theory offers an explanation for how reducing unemployment benefits duration might increase job finding rates. According to search theory, one can derive a relationship between the job finding rate and the parameters of the benefit system, in particular the maximum benefit duration and the replacement rate (Mortensen, 1977). This relationship is driven by adjustments in search effort and reservation wages. The reservation wage is the minimum wage at which the unemployed are willing to accept a job. Forward-looking unemployed workers chose their current search effort and reservation wage in order to maximize the sum of the utility flow realized during the current period, plus the expected discounted future utility flow given that an optimal strategy will be pursued in every future period. The current search effort and reservation wage are thus affected by the future level of benefits. When the benefit period expires, the unemployed person experiences a potentially large drop in income. As the time of benefit exhaustion approaches, the value to that person of remaining unemployed falls, implying a higher search effort and/or a fall in the reservation wage, leading to a higher exit rate out of unemployment (Mortensen, 1977). This non-stationarity implies that unemployed individuals with different lengths of benefit entitlement have different optimal paths of reservation wage and search effort over time (van den Berg 1990).
A shorter entitlement period gives the unemployed individual a stronger incentive to quickly gain employment in order to avoid the drop in income after the exhaustion date. How strong the incentive is depends on the magnitude of the income drop. If no secondary benefit is available for those who have exhausted their current benefit, the incentive to gain employment will be stronger. If an increased job finding rate is mainly driven by lowering the reservation wage, a lower job match quality is to be expected, for example, in the form of lower wages and/or lower re-employment duration.
A number of factors may have an impact on the magnitude of the expected increase in the job finding rate. In general, the overall labour market conditions (i.e. the vacancy rate 6 and, in particular, the unemployment rate) have an impact on the availability of and competition for jobs. If the vacancy rate is high (i.e. the number of vacancies is high in relation to job seekers) we would expect a bigger effect on job finding rates than if the vacancy rate is low. We would further expect a lower effect if the unemployment rate is high, regardless of the vacancy rate. If the vacancy rate is low coincident with a high unemployment rate, competition for available jobs is likely to be high. If the vacancy rate is high coincident with a high unemployment rate, it suggests mismatch in the labour market (i.e., the process by which vacant jobs and job seekers meet is not efficient) (Filges & Larsen, 2000; Pissarides, 2000).
Whether compulsory participation in active labour market programmes is part of the unemployment system may also have an impact on the effect of maximum duration of entitlement. The compulsory aspect of activation may provide an incentive for unemployed individuals to look for and return to work prior to programme participation; the so called threat effect. Filges and Hansen (2015) summarize the available evidence on the threat effect of active labour market programmes and report a significant threat effect of compulsory participation in active labour market programmes. Further, actual participation in active labour market programmes may improve some of the participants' qualifications, thus helping them to find a job. Alternatively, active labour market programmes may have negative stigmatization and signalling effects to employers. Programmes associated with participants having poor employment prospects may carry a stigma. Because of asymmetric information, employers do not know the productivity of new workers, some of whom they might hire from the pool of the unemployed. Prospective employers might then perceive participants in such programmes as low productivity workers or workers with tenuous labour market attachment (Kluve et al. 1999; Kluve et al., 2007).
A recent systematic review by Filges et al. (2015b) investigated the effect of participating in active labour market programmes and found that there is a significant positive effect, although small, of participating in active labour market programmes. The effect reported in Filges et al. (2015b) is, however, a pure post-programme effect of active labour market programmes; it refers to the period after participation in a programme. The net effect of active labour market programme participation on job finding rates is, however, composed of two separate effects: a lock-in effect and a post-programme effect. The lock-in effect refers to the period of participation in a programme. During this period, job-search intensity may be lowered because there is less time to search for a job, and participants may want to complete an on-going skill-enhancing activity; hence the lock-in effect. The combination of the two effects, lock-in and post-programme, consequently determines the net effects of active labour market programme participation on unemployment duration.
These additional effects on the search behaviour and employment prospects when compulsory participation in active labour market programmes is part of the unemployment system may dampen the observed effects of maximum duration of entitlement on job finding rates.
Finally, the type of unemployment benefit may have an impact on the effect on the job finding rate. As mentioned above, some countries employ two systems to provide benefits to unemployed individuals: an unemployment insurance system for individuals who typically have a strong labour market attachment (UI benefits) and a social welfare system for individuals who often have other problems in addition to unemployment (SA or UA benefits). The effect size in social welfare systems offering unemployment benefits with a known expiration date is, due to the participants' lower labour market attachment, expected to be less than the effect size in unemployment insurance systems with a known expiration date.
1.4 WHY IT IS IMPORTANT TO DO THE REVIEW
In order to reduce unemployment levels, policy-makers may wish to reduce the generosity of the unemployment system either in amount (the replacement rate) or in maximum potential duration.
The positive correlation between unemployment duration and the replacement rate is well established at the empirical level (Layard et al., 2005). However, it may be politically intractable to lower the replacement rate, and there are indeed strong efficiency and equity arguments for having a reasonable value of unemployment benefits (Acemoglu & Shimer, 1999; Marimon & Zilibotti, 1999).
Search theory suggests that an increase maximum duration of benefit entitlement has a negative impact on the job search activities of the unemployed, thus increasing their unemployment duration. Indeed, although the effect is small, there is clear evidence that the prospect of exhausting benefits results in a significant increase in job finding (Filges et al., 2013).
Hence, shortening the benefit eligibility period may reduce the share of long and unproductive job searches somewhat. The conclusion in Filges et al. (2013), however, leaves unanswered the question of how much of a reduction in maximum unemployment benefit entitlement decreases unemployment duration.
There are many empirical papers on the effect of maximum benefit entitlement on unemployed individuals (Caliendo, Tatsiramos and Uhlendoff 2009; Bennmarker, Carling & Holmlund, 2007; Ham & Rea, 1987; Hunt, 1995; Katz & Meyer, 1990 and Lalive & Zweimüller, 2004), but the empirical research has not been summarized in a systematic review to obtain a clearer picture of the available evidence on the employment effect of reducing maximum duration of benefit entitlement.
Fredriksson and Holmlund (2006) provide a non-systematic review of the literature on how incentives in unemployment insurance can be improved, but do not make the important distinction between exits to employment and exits to other destinations such as such as other kinds of benefits or out of the labour force. As shown in Card, Chetty and Weber (2007), the exit rate from registered unemployment can increase by more than 10 times than that of the rate of re-employment at the expiration of benefits. The difference between the two measures arises because many individuals leave the unemployment register immediately after their benefits expire without returning to work.
There is a great deal of political interest in optimizing the unemployment benefit system to balance concerns for an adequate social safety net with concerns for the implicit distortionary effects of providing such safety net.
A less generous unemployment benefit system provides less protection from the consequences of involuntary unemployment, but is also expected to increase unemployment by diminishing efforts to gain re-employment. Achieving the desired balance between the apparently conflicting goals of ensuring both a sufficiently high level of protection and a sufficiently low level of unemployment requires reliable information on the effect size of altering benefit generosity on job finding. To the best of our knowledge, no systematic review exists on the magnitude of this effect. This review provides the important contribution of synthesizing existing effect size estimates through a systematic review of the empirical literature on reducing the maximum duration of unemployment benefit entitlement on employment probabilities.
2 Objectives
The purpose of this review is to systematically uncover relevant studies in the literature that measure the effects of shortening the maximum duration of unemployment benefit entitlement on job finding rates, and to synthesize the effects in a transparent manner. As a secondary objective we will, where possible, investigate the extent to which the effects differ among different groups of unemployed such as those with high/low levels of education or men/women, and further explore from which point in the unemployment spell unemployed individuals react to the length of benefit entitlement.
3 Methods
3.1 TITLE REGISTRATION AND REVIEW PROTOCOL
The title for this systematic review was registered July, 2015. The systematic review protocol (Filges, Jonassen & Jørgensen, 2015a), was published November, 2015. Both the title registration and the protocol are available in the Campbell Library at: https://www.campbellcollaboration.org/library/reducing-unemployment-benefit-duration-to-increase-job-finding-rates.html
3.2 CRITERIA FOR CONSIDERING STUDIES FOR THIS REVIEW
3.2.1 Types of studies
The study designs eligible for inclusion were:
Controlled trials: RCT - randomized controlled trial QRCT - quasi-randomized controlled trial (i.e., participants are allocated by means such as alternate allocation, person's birth date, the date of the week or month, or alphabetical order) NRCT - non-randomized controlled trial (i.e. participants are allocated by other actions controlled by the researcher) Non-randomized studies (NRS) where allocation is not controlled by the researcher and two or more groups of participants are compared. Participants are allocated by means such as time differences, location differences, decision-makers, or policy rules.
Study designs that used a well-defined control group were eligible. The main control or comparison condition was no change in maximum duration of benefit entitlement.
Non-randomized studies, where the reduction in maximum duration of benefit entitlement has occurred in the course of usual decisions outside the researcher's control must have demonstrated pre-treatment group equivalence via matching, statistical controls, or evidence of equivalence on key risk variables (e.g., labour market conditions) and participant characteristics. These factors are outlined in section 3.3.3 under the subheading of Confounding, and the methodological appropriateness of the included studies was assessed according to the risk of bias model outlined in section 3.3.3.
Studies of the effect of reducing unemployment benefit entitlement typically are estimated on data collected from administrative registers or by questionnaires. Studies that used different data sources for treatment and control groups were not eligible.
Only studies that used individual micro-data were eligible. Studies that relied on regional or national time series data were not eligible, even though micro-econometric estimates of individual effects merely provide partial information about the full impact of shortening the maximum duration of benefit entitlement (Calmfors, 1994; Calmfors, 1995).
We included studies irrespective of their publication status, and their electronic availability.
3.2.2 Types of participants
We included unemployed individuals who received some sort of time limited benefit during their unemployment spell. The International Labour Office (ILO) definition of an unemployed individual is a person, male or female, aged 15-74, without a job who is available for work and either has searched for work in the past four weeks or is available to start work within two weeks and/or is waiting to start a job already obtained (ILO, 1990); however, different countries may apply different definitions of an unemployed individual, see for example Statistics Denmark (2009). We included participants receiving all types of unemployment benefits with a known exhaustion date. The only restriction was that the benefits must be related to being unemployed. We therefore excluded individuals who only received other types of benefits not related to being unemployed. We included all unemployed participants regardless of age, gender, etc. who received some sort of time limited benefit during their unemployment spell.
3.2.3 Types of interventions
The intervention was reduction 7 in the maximum duration of entitlement of any kind of unemployment benefits. The benefits may be unemployment insurance (UI) benefits or they may be unemployment assistance (UA)/social assistance (SA). The only requirement was that the benefit must have a known expiration date. The UI benefit usually has a known time limit whereas UA and SA usually are indefinite. Unemployment benefits with an indefinite time limit or non-financial benefits were excluded from this review.
3.2.4 Types of outcome measures
The objective was to determine whether reducing the maximum entitlement to unemployment benefits motivates unemployed individuals to find a job more quickly. Distinguishing between destinations was therefore vital. The primary outcome was exits to employment. Studies only looking at exits to other destinations such as other types of social benefits or non-employment and studies that do not distinguish between destinations were not eligible.
We considered secondary outcomes in terms of the impact that reducing the maximum duration of entitlement of benefit has on the duration of re-employment and on income. This was done in order to obtain a clearer picture of the effect that reducing the maximum entitlement of unemployment benefit has on the quality of the job. If the duration of re-employment or the wage is low, this could indicate that reducing entitlement forces unemployed individuals to find jobs that do not match their qualifications and therefore they may return to unemployment more quickly.
Primary outcomes
Primary outcomes we planned to include refer to employment status:
exit rate, measured as a hazard rate, from unemployment to employment (= work with standard wages and which anyone can apply for) proportion employed (= proportion of participants who have obtained work with standard wages and which anyone can apply for) duration until employment (= work with standard wages and which anyone can apply for)
Secondary outcomes
Secondary outcomes we planned to include were:
duration of first employment spell post-intervention re-employment wage
3.2.5 Duration of follow-up
Outcomes measured as hazard ratios were reported as an overall effect on the hazard rate, and in addition some were reported separately for different unemployment duration intervals. All time points reported were considered.
3.2.6 Types of settings
All types of settings were eligible.
3.3 SEARCH METHODS FOR IDENTIFICATION OF STUDIES
Identification of studies were based on updated searches from an earlier similar Campbell review (Filges et al., 2013). The first part of the search period (from 1985 to March 2011) was covered by re-examining results of the searches for that review, where an identical search strategy was used. The search documentation described in 3.3.1-3.3.5 cover the two updates from 2011-2015 and from 2015-2016. The search was performed by one review author (AKJ) and one member of the review team (BVN) 8 .
3.3.1 Electronic searches
Relevant studies were identified through electronic searches of bibliographic databases. The following bibliographic databases were searched:
PsycInfo (searched through EBSCO) – Latest search performed 11/3-2016. SocIndex (searched through EBSCO) – Latest search performed 11/3-2016. Econlit (searched through EBSCO) – Latest search performed 26/2-2016. Business Source Complete (searched through EBSCO) – Latest search performed 10/3-2016. IBSS: International Bibliography of the Social Sciences (searched through ProQuest) – Search performed 10/3-2016. ProQuest Dissertations and Theses (searched through ProQuest) – Latest search performed 10/3-2016. SSCI: Social Science Citation Index & SCI: Science Citation Index (searched through Web of Science) – Latest search performed 10/3-2016.
3.3.2 Search terms
An example of the search string used to search SocIndex from the 2015-2016 update is listed in section 12.1. The search string was modified in accordance to the different search terminology on the databases searched.
3.3.3 Searching other resources
We examined the reference lists (snowballing/citation-tracking) from relevant reviews and studies identified in the electronic searches, and from included primary studies for studies that potentially met inclusion criteria.
3.3.4 Searching for Unpublished/Grey Literature
Searching for unpublished/grey literature was performed by searching governmental repositories, evidence-based practice repositories (such as clearinghouses) and internet search engines. The search strategy for the grey literature search was based on the search string for the electronic database search. Due to the limited search capacity on grey literature information resources, web pages and search engines, a shortened search string was used. An example of the search strategies used to identify grey literature and google searches can be found in section 9.1. The most recent search for grey literature was performed between 26th of February and 10th of March 2016. The following websites, repositories and resources were searched for relevant grey literature:
Cochrane Library (http://www.cochranelibrary.com/) Forskningsdatabasen - The Danish National Research Database (http://www.forskningsdatabase.dk/) Social Care Online (http://www.scie-socialcareonline.org.uk/) IBZ – De Gruyter (https://www.degruyter.com/view/db/ibz) SSRN: Social Science Research Network (https://www.ssrn.com/en/) IDEAS (https://ideas.repec.org/) OpenGrey (http://www.opengrey.eu/) IZA – Institute of the Study of Labor (www.iza.org) CEPR – Centre for Economic Policy Research (www.cepr.org) NBER – National Bureau of Economic Research (www.nber.org) MDRC – the Manpower Demonstration Research Corporation – (www.mdrc.org) Danish Economic Councils (www.dors.dk) OECD - the Organisation for Economic Co-operation and Development (www.oecd.org) IMF - The International Monetary Fund (www.imf.org) AIECE - Association of European Conjuncture Institutes (www.aiece.org) ESRC - Economic Social Research Council (www.esrc.ac.uk) Copenhagen Economics (www.copenhageneconomics.com) Google Scholar (https://scholar.google.dk/)
Due to changes in access possibilities, Theses Canada was not searched after the 2011 update.
3.3.5 Hand searching
Reference lists of included studies and reference lists of relevant reviews was searched. “The Journal of Labor Economics” and “Labour Economics” was searched for the years 2011-2015 and the available issues of 2016 up to March:
Labour Economics (issn: 0927-5371) - Latest search was performed 15/3-2016. Journal of Labour Economics (issn: 0734-306X) - Latest search was performed 15/3-2016.
3.4 DATA COLLECTION AND ANALYSIS
3.4.1 Selection of studies
Under the supervision of review authors, two review team assistants first independently screened titles and abstracts to exclude studies that were clearly irrelevant. Studies considered eligible by at least one assistant or studies were there was insufficient information in the title and abstract to judge eligibility, were retrieved in full text. The full texts were then screened independently by two review team assistants under the supervision of the review authors. Any disagreement of eligibility was resolved by the review authors. Exclusion reasons for studies that otherwise might be expected to be eligible were documented and presented in Section 11.
The study inclusion criteria were piloted by the review authors (see Appendix 2.3). None of the review authors were blind to the authors, institutions, or the journals responsible for the publication of the articles.
3.4.2 Data extraction and management
Two review authors independently coded and extracted data from included studies. A coding sheet was piloted on several studies (See Appendix 2.3 and 2.4). Disagreements were resolved by discussion.
Information was extracted on: available characteristics of participants, intervention characteristics, research design, sample size, time period, outcomes, and results. Extracted data were stored electronically. Analysis was conducted in RevMan 5.
3.4.3 Assessment of risk of bias in included studies
Two review authors independently assessed the risk of bias for each included study. There were only minor disagreements and they were resolved by discussion.
We assessed the risk of bias using a model developed by Prof. Barnaby Reeves in association with the Cochrane Non-Randomised Studies Methods Group (Reeves, Deeks, Higgins, & Wells, 2011). 9 This model is an extension of the Cochrane Collaboration's risk of bias tool and covers risk of bias in non-randomised studies that have a well-defined control group.
The extended model is organised and follows the same steps as the risk of bias model according to the 2008 version of the Cochrane Handbook, chapter 8 (Higgins & Green, 2008). The extension to the model is explained in the three following points:
The extended model specifically incorporates a formalised and structured approach for the assessment of selection bias in non-randomised studies by adding an explicit item that focuses on confounding
10
. This is based on a list of confounders considered important and defined in the protocol for the review. The assessment of confounding is made using a worksheet, which is marked for each confounder according to whether it was considered by the researchers, the precision with which it was measured, the imbalance between groups, and the care with which adjustment was carried out (see Appendix 2.5). This assessment informs the final risk of bias score for confounding. Another feature of non-randomised studies that make them at high risk of bias is that they need not have a protocol in advance of starting the recruitment process. The item concerning selective reporting therefore also requires assessment of the extent to which analyses (and potentially other choices) could have been manipulated to bias the findings reported, e.g., choice of method of model fitting, potential confounders considered/included. In addition, the model includes two separate yes/no items asking reviewers whether they think the researchers had a pre-specified protocol and analysis plan. Finally, the risk of bias assessment is refined, making it possible to discriminate between studies with varying degrees of risk. This refinement is achieved by the use of a 5-point scale for certain items (see the following section Risk of bias judgement items for details).
The refined assessment is pertinent when considering data synthesis as it operationalizes the identification of those studies with a very high risk of bias (especially in relation to non-randomised studies). The refinement increases transparency in assessment judgements and provides justification for excluding a study with a very high risk of bias from the data synthesis.
Risk of bias judgement items
The risk of bias model used in this review is based on 9 items (see Appendix 2.5).
The 9 items refer to:
In the 5-point scale, 1 corresponds to Low risk of bias and 5 corresponds to High risk of bias. A score of 5 on any of the items assessed on the 5-point scale translates to a risk of bias so high that the findings will not be considered in the data synthesis (because they are more likely to mislead than inform).
Confounding
An important part of the risk of bias assessment of non-randomised studies is consideration of how the studies deal with confounding factors (see Appendix 212.5). Selection bias is understood as systematic baseline differences between groups which can therefore compromise comparability between groups. Baseline differences can be observable (e.g. age and gender) and unobservable to the researcher (e.g. motivation and ‘ability’). There is no single non-randomised study design that always solves the selection problem. Different designs represent different approaches to dealing with selection problems under different assumptions, and consequently require different types of data. There can be particularly great variations in how different designs deal with selection on unobservables. The “adequate” method depends on the model generating participation, i.e. assumptions about the nature of the process by which participants are selected into a programme. A major difficulty in estimating causal effects of the maximum duration of benefit entitlement is the potential endogeneity of the change to benefit rules stemming from the policy process that leads to the change.
The determinants of the change are often labour market conditions and if not accounted for it yields biased estimates.
As there is no universal correct way to construct counterfactuals for non-randomised designs, we looked for evidence that identification is achieved, and that the authors of the primary studies justified their choice of method in a convincing manner by discussing the assumption(s) leading to identification (the assumption(s) that make it possible to identify the counterfactual). Preferably the authors should make an effort to justify their choice of method and convince the reader that the only difference between an individual with a short maximum benefit period and an individual with a longer maximum benefit period is exactly the difference in length of maximum benefit period and that the source of difference between their entitlement status is not endogenous to the individuals' exit rate to employment. The judgement is reflected in the assessment of the confounder unobservables in the list of confounders considered important at the outset (see Appendix 2.5).
In addition to unobservables, we identified the following observable confounding factors to be most relevant: age, gender, education, ethnicity, labour market conditions, and unemployment duration. In each study, we assessed whether these factors had been considered, and in addition we assessed other factors likely to be a source of confounding within the individual included studies.
Importance of pre-specified confounding factors
The motivation for focusing on age, gender, education and ethnicity is that they are the major determinants of the risk of being unemployed (Layard et al., 2005).
Another potential source of bias is differences in labour market conditions. If a study, for example, explores changes in the maximum duration of benefit entitlement over time or space as the source of variation, it is very important to control for changes in labour market conditions over time (as a consequence of the business cycle, for example) or over space as the exit rate to employment most certainly will depend on this factor.
Concerning unemployment duration, most studies find that the genuine duration dependence is negative, i.e. the longer the unemployment spell, the smaller the chance of finding a job 11 (see Serneels, 2002, for an overview). If the study does not disentangle the effect of shortening the maximum benefit period from the negative duration dependence the effect will be biased.
3.4.4 Measures of treatment effect
The treatment effect was measured as the impact on the exit rate from unemployment to employment (measured as a hazard ratio) in all studies except one where it was measured as the difference in mean duration (time to employment). Our main interest was to include studies in a meta-analysis where hazard ratios and variances were either reported or were calculable from the available data.
The hazard ratio measures the proportional change in hazard rates between unemployed individuals who have a short maximum benefit period and unemployed individuals who have a longer maximum benefit period. The hazard rate is defined as the event rate (in the present context, the event is finding a job) at time t conditional on survival (staying unemployed) until time t or later. A hazard rate is constructed as follows: 12
The length of an unemployment spell for an unemployed individual (in the present context the length of stay in the unemployment system until finding a job) is a realization of a continuous random variable T. In continuous time, the hazard rate λ(t) is defined as:
where the cumulative distribution function of T is:
and the probability density function is:
F(t) is also known in the survival analysis literature as the failure function and in the present context failure means finding a job. S(t) is the survivor function:
t is the elapsed time since entry to the state (since the individual entered the unemployment system).
Introducing covariates the hazard rate becomes:
where x(t,s) is a vector of personal characteristics that may vary with unemployment duration (t) or with calendar time (s).
A proportional hazard rate is given by:
where λ0 (t) is the baseline hazard, exp(x′β) is a scale function of the vector x of personal characteristics (and a treatment indicator) and β is a vector of estimated parameters.
The baseline hazard is typically not completely specified; often the hazard function is modelled as piecewise constant. Thus, whether the shape of the hazard generally increases or decreases with survival time is left to be estimated from the data, rather than specified a priori.
In the description of the hazard rate it is, so far, implicitly assumed that all relevant differences between individuals can be summarized by observed explanatory variables. But if there are unobservable differences, e.g. motivation and ‘ability’ (in the literature termed unobserved heterogeneity) and these differences are ignored, the estimated parameters will be biased towards zero. It is therefore common to control for both observed factors given by the vector x as well as unobserved factors, i.e. unobserved heterogeneity. The hazard rate, including unobserved heterogeneity, is now given by:
where v represents factors unobserved to the researcher and independent of x. It is necessary to assume the distribution of v has a shape where the right-hand tail of the distribution is not too fat and whose functional form is summarized in terms of only a few key parameters, in order to estimate those parameters with the data available. The unobserved components are typically assumed to follow a discrete distribution with two (or more) points of support.
The acceptable outcome measurement frequency for calculating hazard ratios in this review was three months or less. A study reporting only outcomes measured on time intervals of more than three months was not included in the meta-analysis (on secondary outcomes, the study by Barbanchon (2016) provided data on the survival probability in re-employment within 8 months which was not included in the data synthesis).
As stated in the protocol, Filges et al., 2015a, individual participant data was not requested to calculate log hazard ratios as this may introduce bias due to the time span of studies (the time span between the earliest we knew of and the latest is 30 years).
Studies providing estimates of hazard ratios and variances typically base the estimation on the maximum likelihood method 13 . The principle of maximum likelihood is relatively straightforward. The likelihood function, regarded as a function of the parameters of the model, is the joint density of the observations. The maximum likelihood estimator yields a choice of the estimator as the value for the parameter that makes the observed data most probable.
Ignoring unobserved heterogeneity, the contribution to the likelihood for complete observations is given by the conditional density function of t:
and for censored observations:
The likelihood function is:
where d=1 for complete observations and d=0 for censored observations. Often it is convenient to maximise the logarithm of the likelihood function rather than the likelihood function and the same results are obtained since logL and L attain the maximum at the same point.
The log likelihood function to maximize with respect to the parameters of the model is:
Introducing unobserved heterogeneity with the random components assumed to follow a discrete distribution with two points of support (v
1, v
2, Pr(v
1)=π1, Pr(v
2)=π2 the log likelihood function becomes:
For the continuous outcome mean duration, an effect size with 95% confidence intervals was calculated. Hedges' g was used for estimating the SMD and we applied the small N correction. Hedges' (adjusted) g and its standard error are calculated as (Lipsey & Wilson, 2001:47-49):
where N=n
1+n
2 is the total sample size,
Here, s 1 and s 1 denotes the standard deviation of the two groups.
Secondary outcomes were measured as the impact on the exit rate from the re-employment job to unemployment (measured as hazard ratio) and wage ratio.
Software for storing data and statistical analyses were Excel, STATA and RevMan 5.0.
3.4.5 Unit of analysis issues
To account for possible statistical dependencies, we examined a number of issues: whether individuals were randomised in groups (i.e. cluster randomised trials), whether individuals had undergone multiple interventions, whether there were multiple treatment groups, and whether several studies were based on the same data source.
Cluster randomised trials
No studies using cluster randomisation were found.
Multiple interventions groups and multiple interventions per individuals
Two studies reported separate effect estimates by gender. A synthetic (average) effect size was calculated and used in the analysis to avoid dependence problems. This method provides an unbiased estimate of the mean effect size parameter but overestimates the standard error. Random effects models applied when synthetic effect sizes are involved actually perform better in terms of standard errors than do fixed effects models (Hedges, 2007). However, tests of heterogeneity when synthetic effect sizes are included are rejected less often than nominal.
Multiple interventions per individual
There were no studies with multiple interventions per individual used in the analysis.
Multiple studies using the same sample of data
Two studies used the same sample of data from Germany and three studies used the same sample of data from Austria, i.e. the studies used administrative register data from the same country covering the same time period. We reviewed all studies, but in the meta-analysis we only included one estimate of the effect from each sample of data in order to avoid dependencies between the “observations” (i.e. the estimates of the effect) in the meta-analysis. The choice of which estimates to include was based on our risk of bias assessment of the studies. We chose the estimate from each sample of data from the study that we judged to have the least risk of bias due to confounding.
Multiple time points
All studies reported results as an overall effect, either measured as hazard ratios or mean difference in duration.
3.4.6 Dealing with missing data
The reviewers assessed missing data rates in the included studies in accordance with the risk of bias tool used (see section 3.3.3). As stated in the protocol, Filges et al., 2015a, we did not request information from the principal investigators if not enough information was provided to calculate an effect size and standard error due to the time span of studies (the time span between the earliest we know of and the latest is 30 years).
3.4.7 Assessment of heterogeneity
Heterogeneity among primary outcome studies was assessed with Chi-squared (Q) test, and the I-squared, and τ-squared statistics (Higgins, Thompson, Deeks, & Altman, 2003). Any interpretation of the Chi-squared test was made cautiously on account of its low statistical power. Values of τ-squared were however, interpreted with caution. The DerSimonian and Laird estimate of τ-squared is on average overestimated and when the number of studies is small the bias can be substantial (Borenstein, Hedges, Higgins & Rothstein, 2010).
3.4.8 Data synthesis
We carried out our meta-analysis using hazard ratios 14 . Hazard ratios were log transformed before being analysed. The reason is that ratio summary statistics all have the common features that the lowest value that they can take is 0, that the value 1 corresponds with no intervention effect, and the highest value that a hazard ratio can ever take is infinity. This number scale is not symmetric. The log transformation makes the scale symmetric: the log of 0 is minus infinity, the log of 1 is zero, and the log of infinity is infinity.
All analyses were inverse variance weighted using random effects statistical models that incorporate both the sampling variance and between study variance components into the study level weights. Random effects weighted mean effect sizes were calculated using 95% confidence intervals. Analysis was conducted in RevMan 5. Graphical displays (forest plots) for meta-analysis performed on ratio scales sometimes use a log scale, as the confidence intervals then appear symmetric. This is however not the case for the software RevMan 5. The graphical displays using hazard ratios and the mean effect size were reported as a hazard ratio. Heterogeneity among primary outcome studies were assessed with Chi-squared (Q) test, and the I-squared, and τ-squared statistics (Higgins, Thompson, Deeks, & Altman, 2003). Any interpretation of the Chi-squared test was made cautiously on account of its low statistical power.
Studies that were coded with a very high risk of bias (scored 5 on the risk of bias scale) were not included in the data synthesis.
3.4.9 Sensitivity analysis
Sensitivity analysis was used to evaluate whether the pooled effect sizes were robust across components of methodological quality. For methodological quality, we performed sensitivity analyses for the confounding, other bias, and selective reporting items of the risk of bias checklists, respectively. Sensitivity analysis was further used to examine the robustness of conclusions in relation to quality of data (outcome measures based on weekly or monthly data) and using an extension of maximum benefit entitlement to estimate the effect.
4 Results
4.1 DESCRIPTION OF STUDIES
4.1.1 Results of the search
The search was performed between November 2010 and March 2016.
Results are summarised in Figure 2.1 in section 12.2. The total number of potential relevant records was 34,930 after excluding duplicates. All 34,930 records were screened based on title and abstract; 34,342 were excluded for not fulfilling the first level screening criteria and 579 records were ordered for retrieval and screened in full text. Of these, 509 did not fulfil the second level screening criteria and were excluded. Four records were unobtainable despite efforts to locate them through libraries and searches on the internet (see section 9.3). A total of 41 unique studies, reported in 66 papers, were included in the review.
4.1.2 Included studies
The search resulted in a final selection of 41 studies (reported in 66 papers) that met the inclusion criteria for this review. In Table 4.1, we show the total number of studies that met the inclusion criteria for this review. The first column shows the total number of studies grouped by country. The second column shows the number of these studies that were coded with too high risk of bias to be included in the data synthesis. The third column gives the number of studies that did not provide enough data to calculate an effect estimate. The fourth column gives the number of studies that were excluded from the data synthesis due to overlapping samples. The last column gives the total number of studies used in the data synthesis, in total seven studies.
Total number of studies by country
Note: The reduction due to too high risk of bias preceded the reduction due to overlap of data sample.
Or data that enable the calculation of an effect estimate.
The data samples used are representative for the same population at a given time (see Section 3.4.5 for this methodological issue).
One study reported on secondary outcomes only
Of the 41 studies that met the inclusion criteria, 3 did not provide data that permitted the calculation of an effect size (Arntz, Simon & Wilke, 2014; Lee & Wilke, 2009; Micklewright & Nagy, 1995). Two of the three studies, analysing data from Germany, provided a bounds analysis and presented results as figures only (Arntz et al., 2014; Lee & Wilke, 2009). The third study, analysing data from Hungary, also presented results as figures only (Micklewright & Nagy, 1995).
Of the remaining 38 studies, 28 studies were coded with a very high risk of bias (5 on the risk of bias scale) and were therefore not used in the data synthesis (Addison & Portugal, 2008; Ahn & Ugidos-Olazabal, 1995; Amarante, Arim & Dean, 2013; Arranz, Bulló & Muro, 2008; Belzil, 1995; Bover, Arellano & Bentolila, 2002; Chang, 2010; de Groot & van der Klaauw, 2014; Engberg, 1990; Farber & Valletta, 2015; Farber, Rothstein & Valletta, 2015; Ferrada, 2011; Figura & Barnichon, 2014; Fitzenberger & Wilke, 2010; He, 2013; Hunter, 1990; Landais, 2015; Lubyova & van Ours, 1997; Machikita, Kohara & Sasaki, 2013; Newton & Rosen, 1979; Puhani, 2000; Rebollo-Sanz & García-Pérez, 2015; Rothstein, 2011; U.S. Department of Labor, 1995; Valletta, 2014; Van Ours & Vodopivec, 2006; Winter-Ember, 1998; Wolff, 1997).
Three additional studies (Lalive, 2007; Lalive, Landais & Zweimuller, 2015; Schmieder, von Wachter, Bender, 2016) could not be used in the data synthesis due to overlapping data samples (i.e., the studies used administrative register data from the same country covering the same time period or overlapping time periods; see Section 3.4.5 for this methodological issue). These studies analysed unemployment benefits in Germany and Austria.
After these exclusions from the data synthesis, seven studies remained that could be used in the data synthesis (Barbanchon, 2016; Caliendo, Tatsiramos & Uhlendorff, 2013; Card, Chetty & Weber, 2007; Lalive & Zweimüller, 2004; Nekoi & Weber, 2015; Schmieder, von Wachter, Bender, 2012; Van Ours & Vodopivec, 2008). One of these seven studies, however, only reported on secondary outcomes (Van Ours & Vodopivec, 2008).
For studies with overlapping samples, 2 studies on German data and 3 studies on Austrian data, the choice of which study to use in the data synthesis was based on our risk of bias assessments. The citations for the ten studies that provided effect size estimates and could be used in the data synthesis can be found in Section 9.1.
One of the two German studies (Schmieder, von Wachter & Bender, 2012; Schmieder, von Wachter & Bender, 2016), using data representative of the same population of unemployed at the same time, was deselected as it was judged to have the higher risk of bias due to Other bias (Schmieder, von Wachter & Bender, 2016).
Five studies analysed unemployed in Austria spanning the time period 1981 to 2011 15 (Card, Chetty & Weber, 2007; Lalive, 2007; Lalive & Zweimüller, 2004; Lalive, Landais & Zweimüller, 2015; Nekoei & Weber, 2015). Although the five studies analysed unemployed in Austria during the same time period (or time periods that overlapped) they did not all analyse the same population, but rather three different sub populations. Three studies could thus be included in the data synthesis. The study by Card et al. (2007) analysed unemployed workers 20-50 years of age with 1-5 years of work experience within the last 5 years preceding their unemployment spell and was included in the data synthesis. The study by Nekoei and Weber (2015) analysed workers 38-42 years of age with more than 6 years of work experience within the last 10 years preceding their unemployment spell and was included in the data synthesis. Three remaining three studies (Lalive, 2007; Lalive & Zweimüller, 2004; Lalive et al., 2015) analysed unemployed workers 45-54 years of age. Two of these studies further restricted the analysis to workers with a continuous work history during the 25 years preceding their unemployment spell (Lalive, 2007; Lalive & Zweimüller, 2004). Two of the studies (Lalive, 2007 and Lalive et al., 2015) were deselected as they were judged to have a higher risk of bias than Lalive and Zweimüller (2004) due to Confounding.
The detailed characteristics of all 41 studies are provided in Section 14.1.1. A summary of the characteristics of the seven studies that were used in the data syntheses are shown in Table 4.2. These seven studies are the ones on which the conclusions on the effect of maximum duration are based.
Characteristics of studies used in the data synthesis
All studies analysed variation in entitlement of unemployment insurance benefits in European countries. One study used data from the 1980s and 1990s. Two studies used data from the 2000s and four studies used data from the periods both before and after 2000. Data were drawn from administrative registers. The sample sizes were generally large; all of the studies used sample sizes of more than 5,000 and in total 1,154,090 spells were used. In three studies the time interval of the outcome measure was weekly and in four studies it was monthly.
One study included only males and two studies provided results separated by gender. Only one of the studies reported whether compulsory labour market activation was part of the unemployment system. All studies reported on the availability of alternative benefits, but only reported that means tested unemployment assistance was available. None of the studies reported the labour market conditions (unemployment rate, vacancy rate and/or labour market tightness 16 ).
There was a high degree of variation in maximum entitlement, ranging between 26 and 209 weeks. On average the studies analysed a reduction of 43 weeks in maximum entitlement; the smallest being a reduction of 9 weeks and the largest a reduction of 179 weeks. Four studies were restricted to a specific age group and three studies were restricted to specific work experience levels.
As expected none of the studies were based on randomisation of participants. The central problem in studies without randomisation of participants is the identification of the causal effect of the intervention. The restrictions on age and work experience in the studies were a consequence of the limitations in possible identification strategies. The studies relied on age dependent and/or work experience dependent and/or region dependent variation in benefit rules. The restrictions on age and/or work experience and identification strategy used in the primary studies included in the data synthesis are shown in table 4.3.
Restriction and methods used in the studies used in the data synthesis
Only the restrictions relevant for the effect estimate used in the data synthesis is shown
As a consequence of the identification strategies used, none of the studies analysed a representative sample of unemployed workers.
Two studies relied on legislative changes of the maximum entitlement period for specific age groups, work experience levels or regions. One of the two studies analysed an extension of maximum benefit entitlement in Austria (Lalive and Zweimüller, 2004), the other analysed a reduction of maximum benefit entitlement in Slovenia (van Ours and Vodopivec, 2008).
The reform of extended benefit entitlement in Austria, studied in in Lalive and Zweimüller (2004), was enacted to mitigate labour market problems in certain regions and for certain subgroups of workers. The extension was limited to job seekers aged 50 or more, living in certain regions and in effect for a limited time period only (the extension was rolled back after a few years). The specific implementation of the extension ensured that several groups of workers who were not entitled to the extension - yet quite similar to entitled individuals - could be used as control groups. The identification strategy used in Lalive and Zweimüller (2004) accounted for time trends using a difference-in-differences-in-difference strategy. The authors argue that treated individuals were not subject to idiosyncratic shocks during the observation period. Thus, the policy of extended benefit entitlement could be considered ‘exogenous’ 17 and used to identify the causal effect of the intervention (see Lalive & Zweimüller, 2004, for further details).
The study by van Ours and Vodopivec (2008) relied on a reform of the Slovenian unemployment insurance system in 1998 for identification. Maximum entitlement depended on work experience and the reform reduced the maximum duration of benefits by roughly half for most groups of recipients (except the group with the lowest level of work experience). The reform introduced different variations in potential benefit duration for different groups of unemployed, which, according to the authors, speaks to the credibility of the applied identification strategy. To identify the effect of reducing maximum entitlement, the authors adopted a difference-in-difference strategy and compared the probability of entering employment before and after the reform, for those affected by the reform and for those not affected. The reform affected inflows to unemployment around the time of its introduction, increasing inflows just before the introduction and reducing inflows just after. To avoid bias the authors do not consider data covering the 2 months before the introduction of the reform and the 2 months after.
In Caliendo, Tatsiramos & Uhlendorff (2013) and Schmieder, von Wachter & Bender (2012) analyses on German data were based on regression discontinuity designs. The identification strategy used in both studies relied on a sharp discontinuity in the maximum duration of unemployment benefits in Germany; at the age of 45 (used in Caliendo, et al. (2013)) and at the age of 42 (used in Schmieder et al. (2012)). Comparing newly unemployed individuals who were just below the age threshold with newly unemployed individuals just above the age threshold, and accounting for age trends, gives a measure of the effect of maximum duration of benefits on job finding at the cut-off.
In Nekoei and Weber (2015) and Card et al. (2007) the analyses, relying on Austrian data, were also based on regression discontinuity designs. The identification strategy used in Nekoei and Weber (2015) relied on a sharp discontinuity in the maximum duration of unemployment benefits in Austria at the age of 40. The identification strategy used in Card et al. (2007) relied on a sharp discontinuity in the maximum duration of unemployment benefits in Austria at 36 months of work within the past 5 years before the start of the unemployment spell.
Finally, Barbanchon (2016) relied on a sharp discontinuity in the maximum duration of unemployment benefits in France, where the discontinuity occurred at 8 months of work during the past 5 years before the start of the unemployment spell.
Section 144 provides a further description of all the individual studies (including those without effect estimate and those with very high risk of bias) and a more detailed description of how the maximum durations of unemployment benefits vary.
4.1.3 Excluded studies
In addition to the 41 studies that met the inclusion criteria for this review, 38 studies (reported in 49 papers) appeared relevant but did not meet our inclusion criteria. These studies and the reasons for exclusion are given in Section 9.2 and Section 11.
4.1.4 Studies awaiting classification
Four references were not obtained in full text despite repeated attempts to locate them (see Section 9.3).
4.2 RISK OF BIAS IN INCLUDED STUDIES
The detailed risk of bias coding for each of the 41 included studies is shown in Section 14.2. A summary of the risk of bias rating is shown in Table 4.4. Because all included studies used non-randomised designs, they were all judged to have a high risk of bias on the Sequence generation item and the Allocation Concealment item. The treated group has to know they are treated in order to react to it; therefore, it is not relevant to consider blinding of the participants. Furthermore, the nature of the outcome, exit into employment, is objective and obtained from administrative registers or questionnaires, which were not collected with the aim of analysing changes in the maximum benefit duration. We therefore rated all the studies 4 on the Blinding item. Two studies were rated 5 on the Incomplete outcome data item, both of these studies were also rated 5 on either the Other bias or the Confounding item. Approximately half of the studies did not provide enough information for the Incomplete outcome data item to be rated. No study was rated 5 on the Selective reporting item and almost all were free of any risk of bias on this item. Only five studies had serious issues on the Selective reporting item, leading to a rating of 4 (a detailed description of the reasons can be found in Section 14.2). Ten studies were rated 5 on the Other bias item, nine of these were also rated 5 on the Confounding item. In total 27 studies were rated 5 on the Confounding item.
Risk of bias - distribution of the 41 studies
Notes: 1 The judgment is based on a 5-point scale, where 1 indicates low risk of bias and 5 indicates high risk of bias.
Some of the studies judged to be at very high risk of bias on the Confounding item, based the analysis on sharp discontinuities in treatment status at a certain age level and/or work experience level but failed to deliver sufficiently credible arguments in favour of the validity of the identification strategies. The authors did not convincingly argue that the variation used for identification was not subject to selection around the cut off, i.e. insufficient arguments, or no arguments, concerning agents' inability to precisely control the assignment variable near the known cut off were provided.
In general, there was a lack of test of smoothness and density tests reported in the studies to support the validity of the designs chosen. Figures or tests addressing the central assumption behind the validity of the research design, such as of smoothness, should be provided; in the case of the regression discontinuity design, as evidence to support the assumption that all relevant factors (other than treatment) are evolving smoothly with respect to the assignment variable. Likewise figures or tests of density should be provided to assure that the density of the assignment variable is continuous at the discontinuity threshold (the cut off). For further information on smoothness and density tests, see Lee & Lemieux, 2010.
In a number of studies, the main concern was a lack of information in the data sets used. Especially studies using data from the US lacked information on whom among those eligible for unemployment insurance (UI) actually received UI (for example, the UI take up rate is around 50% according to Farber and Valetta (2015)). Further, in most of the studies using data from the US, entitlement (to ordinary UI) was set to the maximum (26 weeks) for all, regardless of the individuals' actual entitlement, as the data set used by the researchers did not include information on individual entitlement.
For details concerning confounding in the individual studies, see Section 14.2.
None of the studies had an a priori protocol or an a priori analysis plan.
In total 28 studies were given a score of 5 on at least one of the risk of bias items and were therefore not included in the data synthesis.
4.3 SYNTHESIS OF RESULTS
The majority of studies reported hazard ratios and variances. One study reported mean difference in duration of non-employment and one study reported only on secondary outcomes. Two studies reported effect measures separately for men and women. For these two studies, the average effect size was calculated and used in the meta-analysis to avoid dependence problems. One study analysed an extension of maximum entitlement to unemployment benefits.
4.3.1 Primary outcomes result
Five studies provided effect estimates measured as hazard ratios. A hazard ratio greater than 1 indicates that the treated (with reduced maximum entitlement to unemployment benefits) is favoured. That is, the conditional exit rate from unemployment into employment is higher for persons who have a lower maximum entitlement to unemployment benefits. All reported results indicated a positive effect favouring the treated. The total number of spells used was 1,091,088.
The weighted average was positive and statistically significant. The random effects weighted mean hazard ratio was 1.10 (95% CI 1.03 to 1.17, p=0.0005). The forest plot is displayed in Figure 4.1. There is some heterogeneity between the studies; the estimated τ2 is 0.00 (the more exact value of the estimated τ2 is 0.003, Q= 30.12, df=4, p<0.00001) and I2 is 87% as displayed in figure 4.1.
Forest plot, exit to employment, hazard ratio
The I2 indicates a high degree of heterogeneity; however this appears to be mainly due to the sensitivity of the I2 statistic to the precision of the primary studies effect sizes (Rücker, Schwarzer, Carpenter, & Schumacher, 2008). The value of I2 is sensitive to the precision of the primary studies effect sizes, in the sense that the more precisely the primary studies effect sizes are estimated, the higher the values of I2, all else equal (Rücker et al., 2008). In this case the estimated between study variance is more informative about the consistency of the evidence. There is some degree of heterogeneity, but as indicated by the value of τ2, it may not be of high practical importance.
One study (Schmieder et al., 2012) provided data on the mean difference (and standard deviation) in non-employment duration. The number of spells used was 45,301. The standardized mean difference in non-employment duration is -0.053 [95% CI -0.060, -0.046].
Overall, the data synthesis for the effect on the exit rate to work revealed a small and statistically significant effect. The effect favoured the treated in the sense that reducing the maximum entitlement to unemployment benefits increases the exit rate to work. The one study that reported mean difference in non-employment duration supported this result. Workers experienced shorter periods of non-employment when the maximum entitlement to unemployment benefits decreased.
4.3.2 Secondary outcomes result
In addition to the primary outcome, we considered secondary outcomes that were relevant to the impact that reducing the maximum entitlement to unemployment benefits can have on re-employment. Results on the exit rate from re-employment and wage ratio were provided.
Three studies (Caliendo et al., 2013; Card et al., 2007 and van Ours & Vodopivec, 2008), provided data on the exit rate from re-employment. A hazard ratio of less than 1 indicates that the treated (with reduced maximum entitlement to unemployment benefits) is favoured. That is, the conditional exit rate from re-employment into unemployment is lower for persons who had a lower maximum entitlement to unemployment benefits at the time the job was found. The evidence is inconclusive; one study reported results indicating a negative effect and two studies reported results indicating a positive effect. The total number of spells used was 532,073. Pooled results showed a negative non-significant effect. The random effects weighted hazard ratio was 0.99 (95% CI 0.97 to 1.02, p=0.64). There were no statistically significant heterogeneity of effects among studies (τ2=0.00, Q= 0.36, df=2, p=0.84). Although the p-value of the Q-statistic is notoriously underpowered to detect heterogeneity in small meta-analyses, the estimated τ2 is 0.00 and I2 is 0%, implying that heterogeneity among these three studies is not present. The forest plot is displayed in Figure 4.2.
Forest plot, exit from re-employment, hazard ratio
Three studies provided data on the log wage ratio in the re-employment job (Barbanchon, 2016; Card et al., 2007 and Nekoei & Weber, 2015).
The evidence is inconclusive; one study reported results indicating a positive effect and two studies reported results indicating a negative effect.
The weighted average was a wage ratio of 1, indicating no difference between treated and control. The random effects weighted mean wage ratio was 1.00 (95% CI 0.99 to 1.01, p=0.089). The forest plot is displayed in Figure 4.3. There is some heterogeneity between the studies; the estimated τ2 is 0.00 (the more exact value of the estimated τ2 is 0.0001) and I2 is 64% as displayed in figure 4.3.
Forest plot, wage ratio in re-employment job
The I2 indicates some degree of heterogeneity; however, this appears to be mainly due to the sensitivity of the I2 statistic to the precision of the primary studies effect sizes (Rücker, Schwarzer, Carpenter, & Schumacher, 2008). As indicated by the value of τ2, it is probably ignorable.
4.3.3 Sensitivity analysis
Sensitivity analyses were planned to evaluate whether the pooled effect sizes were robust across study design and components of methodological quality. Due to the fact that we found no randomised controlled trials, we could not evaluate the impact of study design. For methodological quality, we carried out sensitivity analyses for the Confounding, Other bias, and Selective reporting components of the risk of bias checklists, respectively. We examined the robustness of conclusions when we excluded studies with risk of bias scores of 4 on Confounding, Other bias, and Selective reporting. Sensitivity analysis was further used to examine the robustness of conclusions in relation to the quality of data (outcome measures based on weekly or monthly data collection). Finally, sensitivity analyses were used to examine robustness of conclusion when we removed the study analysing an extension of the maximum entitlement to unemployment benefits.
The results are provided in Table 4.5 and displayed in a forest plot in Section 13.1.
Sensitivity analysis - results
There were no appreciable changes in the results either due to exclusion of studies with scores of 4 on the Confounding, Other bias, or Selective reporting components of the risk of bias checklists. Further, there were no appreciable changes in the results following removal of the study based on monthly data or the study analysing an extension of the maximum entitlement to unemployment benefits.
The overall conclusion that the hazard rate significantly increases when reducing the maximum entitlement to unemployment benefits does not change.
5 Discussion
5.1 SUMMARY OF MAIN RESULTS
This review focused on the effect of reducing the maximum entitlement to unemployment benefits. The available evidence does suggest that there is an effect, although the effect is small. We found a statistically significant effect of reducing the maximum entitlement to unemployment benefits. The effects were measured by hazard ratios in five studies and one study reported mean difference in non-employment duration. In the context of hazard ratios (the ratio of two hazard rates), the hazard is the rate within a short time interval at which the unemployed individual finds a job, conditional on staying unemployed. In other words, the probability of finding a job in that short time interval is the hazard rate. The weighted average effect (using the five studies reporting hazard ratios) measured as a hazard ratio is 1.10, which translates into an increase of approximately 10% in the exit rate from unemployment into employment. The effect thus favoured the treated in the sense that reducing the maximum entitlement to unemployment benefits increases the exit rate to work. The one study that reported mean difference in non-employment duration supported this result.
Interpretation of the result of a 10% increase in the exit rate from unemployment into employment would ideally involve a measure of the average hazard rates for the comparison. However, none of the five studies reported such rates. Some of the studies displayed figures of the average hazard rates over the entire unemployment period. Using these figures, we were able to estimate that the relevant hazard rates (depending on the elapsed duration, all papers reported decreasing hazard rates over the spell) lie in the interval 0.02-0.14, i.e., the conditional probability of finding a job in a short time interval (a week or a month depending on the unit of analysis in the primary studies) lies between 2% and 14%. Thus, the hazard rates have increased with 10% to the interval 0.022-0.154, i.e., the conditional probability of finding a job in a short time interval has increased to 2.2-15.4% solely due to a shorter benefit entitlement period.
The interpretation of a hazard ratio greater than one is that a treated unemployed person who has not yet found a job by a certain time has a higher chance of finding a job at the next point in time compared to someone in the control group.
There is an alternative interpretation of the hazard ratio that may be intuitively easier to understand. The hazard ratio is equivalent to the odds that an individual in the group with the higher hazard reaches the endpoint (finds a job) first.
Stated another way, for any pair of unemployed people, one from the treatment group and one from the control group, the hazard ratio is the odds that the time to find a job is less in the unemployed from the treatment group than in the unemployed from the control group. The probability of finding a job first (P) can easily be derived from the odds or hazard ratio (HR) of finding a job first, which is the probability of finding a job first divided by the probability of not finding a job first: HR=odds= P/(1- P); P= HR/(1+ HR) (Spotswood et al. 2004). A hazard ratio of 1.10 therefore corresponds to a 52% chance of the treated unemployed person finding a job first. The lower and upper 95% confidence interval corresponds to a 51- 54% chance of the treated unemployed person finding a job first.
Concerning secondary outcomes, we analysed the effect of reducing the maximum entitlement to unemployment benefits on the subsequent exit rate from the re-employment job and the wage ratio in the re-employment job. Only three studies could be used in each of these analyses. The overall impact on the exit rate from the re-employment job of shortening the maximum duration of unemployment benefit entitlement, obtained using hazard ratios, was 0.99 and the overall wage ratio was 1.00. There is a lack of evidence to support the hypothesis that reducing the maximum entitlement to unemployment benefits has an impact on the quality of the job, measured as the exit rate of re-employment and wage ratio.
5.2 OVERALL COMPLETENESS AND APPLICABILITY OF EVIDENCE
In this review, we included in total seven studies in the data synthesis. This number is relatively low compared to the large number of studies (41) meeting the inclusion criteria. The reduction was caused by three different factors. Three of the 41 studies did not report effect estimates or provide data that would allow the calculation of an effect size. Twenty-eight studies were judged to have a very high risk of bias (5 on the scale) and, in accordance with the protocol, we excluded these from the data synthesis on the basis that they would be more likely to mislead than inform on the size of the effect of the intervention. Three further studies were excluded because of overlapping data samples.
If all the 41 studies had provided an effect estimate with lower risk of bias, the final list of useable studies in the data synthesis would have been larger 18 which, in turn, would have provided a more robust literature on which to base conclusions.
In total, 15 countries were represented by the 41 studies meeting the inclusion criteria. The seven studies used in the data synthesis covered Austria, France, Germany and Slovenia (four countries, all European).
The geographical coverage thus became narrower as studies from the US, Netherlands, Canada, Portugal, Spain, Chile, Hungary, Japan, Uruguay, Slovakia and Poland could not be used in the data synthesis. Furthermore, all the studies used in the data synthesis were restricted to a specific (narrow) age group and/or were restricted to specific (narrow) work experience levels. These restrictions of representativeness constitute a clear limitation in terms of the generalizability of the results of the review.
The narrow geographical coverage and the restrictions of participants (to specific age and/or work experience levels) may limit the applicability of the evidence and it may be difficult to translate results to expected effects in other settings. The applicability of the evidence would be greater if an effect was found across more different labour markets, such as the US labour market (liberalistic) and e.g. the Scandinavian labour markets (comprehensive welfare state institutions) and across different characteristics of the of unemployed.
It was not possible to examine the impact of a reduction of the maximum entitlement to unemployment benefits of the following moderators: gender, age, education, type of unemployment benefit, whether alternative benefits were available, and if compulsory activation was part of the system or labour market conditions. These factors are all potential moderators of the effect that policy-makers must take into account in accordance with the country's specific institutional setting in order to assess the possibility of generalizing the synthesized result to the specific population for whom a reform of existing entitlement rules is under consideration.
In attempt to obtain a clearer picture of the effect of a reduction of the maximum entitlement to unemployment benefits on the quality of the job obtained, we analysed the subsequent exit rate from re-employment and the re-employment wage ratio as secondary outcomes. Only three studies were eligible for analysis of each of these outcomes. The small number of studies reporting these outcomes makes us reluctant to draw a conclusion.
5.3 QUALITY OF THE EVIDENCE
All studies used non-randomised designs. Overall the risk of bias in the majority of included studies was high. Twenty-eight studies were judged to be at very high risk of bias.
Some of the studies judged to be at very high risk of bias based the analysis on a regression discontinuity design relying on sharp discontinuities in age and/or work experience level. In general, these studies did not provide arguments concerning agents' inability to precisely control the assignment variable near the known cut off and there was a lack of test of smoothness and density tests reported in the studies to support the designs chosen. Another issue, especially concerning US studies was the lack of relevant information in the data sets used. The data sets used in many US studies did not have information on whom among the UI eligible actually received UI and who were entitled to maximum duration.
The risk of bias was examined using a tool for assessing risk of bias incorporating non-randomised studies. We attempted to enhance the quality of the evidence in this review by excluding studies judged to be at very high risk of bias from the data synthesis, using this tool. We believe this process excluded from the data synthesis those studies that were more likely to mislead than inform.
Furthermore, we performed a number of sensitivity analyses to check whether the obtained result is robust across methodological quality, data quality and direction of change to entitlement. The overall conclusion did not change.
To check the robustness across methodological quality, the studies with relatively high risk of bias (score of 4) in Confounding, Other bias, and Selective reporting, respectively, were excluded from the analysis. To check the robustness across data quality, the one study with an estimate on monthly data was removed. In addition, the one study analysing an extension of maximum unemployment benefit entitlement was removed.
The overall conclusion that the hazard rate significantly increases when the maximum entitlement to unemployment benefits is reduced did not change. Due to the low number of studies, it was not possible to perform sensitivity analyses for secondary outcomes.
There was overall consistency in the direction of effects on the exit rate to employment in that all effects favoured the unemployed with the shortest maximum entitlement to unemployment benefits in finding a job. There was some degree of heterogeneity between studies, but as indicated by the very low value of the estimated between study variance (τ2), it may not be of high practical importance.
5.4 LIMITATIONS AND POTENTIAL BIASES IN THE REVIEW PROCESS
We believe that all the publicly available studies on the effect of reducing the maximum entitlement to unemployment benefits on employment up to the censor date were identified during the review process. However, four references were not obtained in full text.
We were unable to comment on the possibility of publication bias because there were insufficient studies included in the meta-analysis for the construction of funnel plots. Thus, it may be possible there are still some missing studies.
We believe that there are no other potential biases in the review process as two members of the review team 19 (JKS, UHP) independently coded the included studies. Any disagreements were resolved by discussion. Further, decisions about inclusion of studies and assessment of study quality were made by two review authors (ABJ, TF) independently and minor disagreements resolved by discussion. Numeric data extraction was made by one review author (TF) and was checked by a second review author (ABJ).
5.5 AGREEMENTS AND DISAGREEMENTS WITH OTHER STUDIES OR REVIEWS
As this is the first systematic review of the literature on the effects on job finding rates of changing the maximum unemployment benefit duration no directly comparable literature exists.
An early related contribution is Krueger & Meyer (2002) summarising evidence on the labour supply effects of social insurance programmes, including unemployment insurance benefits. The authors, however, do not specifically draw conclusions regarding the size of effects of changes to the maximum unemployment benefit duration, but merely state that “the programs tend to increase the length of time employees spend out of work” (p. 2327).
Closest to our work are two recent contributions by Tatsiramos & van Ours (2014) and by Schmieder & von Wachter (2016). However, none of these reviews are systematic in their search of relevant literature and neither do any of these reviews distinguish between destinations (employment or out of unemployment). Tatsiramos & van Ours (2014) presents an overview of the results from six recent studies on the effects of the potential benefit duration, while Schmieder & von Wachter (2016) present a selected sample of five US studies and eight European studies. Different from our approach, the overviews in both Tatsiramos & van Ours (2014) and Schmieder & von Wachter include estimates of the effect of the potential benefit duration on unemployment duration, which does not imply a similar effect on job finding. The effect of the potential benefit duration on e.g. non-employment duration need not be the same as the effect on unemployment duration (see e.g. Schmieder & von Wachter (2016) for further discussion). In fact, studies reporting both estimates, typically report substantially larger estimates of the effect on unemployment duration than of the effect on non-employment duration (see e.g. Lalive (2007), Schmieder et al. (2012) and Barbanchon (2016)).
Importantly, our systematic review also differs from the reviews by Tatsiramos & van Ours (2014) and Schmieder & von Wachter (2016) by not including in the data synthesis estimates from studies that were judged (transparently) to have too high risk of bias and more likely to mislead than inform on the effect size of the intervention.
Besides not relying on a systematic search approach, the contributions by Tatsiramos & van Ours (2014) and by Schmieder & von Wachter (2016), thus, are not directly comparable to our review in two important respects concerning the studies forming the basis of the conclusions. Tatsiramos & van Ours (2014) and Schmieder & von Wachter (2016) include in their data synthesizes studies, which we have excluded from our review due to not considering the effect on job finding rates. Both studies also include studies which we have excluded from our data synthesis due to too high risk of bias.
In addition, although both reviews present estimates from all the included studies, it is not fully transparent how they synthesized them in order to reach the conclusions.
Schmieder & von Wachter (2016) report the median marginal effect on duration, the range and the mean marginal effect on duration (after two outliers at the top and bottom have been removed, p. 15) and conclude that recent studies (US and European) point to an only moderate negative labor supply effect of the potential benefit duration (p. 14).
Tatsiramos & van Ours (2014) state that: “An extension of potential benefit duration leads to an increase in actual unemployment duration of about 20% of the original benefit duration extension” (p. 299) and conclude that the effects on unemployment duration of changes to potential benefit duration are substantial. Note that this conclusion is based on an extension of potential benefit duration and it is not evident that they should be symmetric to effects of reductions.
The available evidence analysed in our systematic review also suggests an effect of changing the maximum entitlement to unemployment benefits on job finding rates, although the size of the effect is small. As such, the conclusion from this systematic review regarding the magnitude of the effect of the intervention may seem inconsistent with the conclusion in the review in Tatsiramos & van Ours (2014). However, it should be kept in mind that the apparently different conclusions concerning the magnitude of the effects are obtained based on very different inclusion criteria concerning outcomes and substantially different approaches and statistical methods.
6 Authors' conclusions
6.1 IMPLICATIONS FOR PRACTICE AND POLICY
Search theory suggests that shortening the benefit eligibility period may reduce the share of long and unproductive job searches as a shorter maximum eligibility period will tend to accelerate job searching from the beginning of the unemployment spell.
In this review, we have found evidence that a reduction in the maximum entitlement to unemployment benefits results in an increase in job finding. Thus, the theoretical suggestion of an effect of a shorter benefit period on accelerated job finding rates has been confirmed empirically, although the impact is small.
The effect of shortening the maximum entitlement to unemployment benefits was measured by hazard ratios. The overall impact of shortening the maximum entitlement to unemployment benefits corresponds to a 52% chance of the treated unemployed person finding a job first.
Overall, shortening the maximum entitlement to unemployment benefits displays a limited potential to alter the job finding rates of the affected unemployed individuals.
Whether the increased job finding rate implies a decrease in the overall unemployment level depends on whether it is caused mostly by an increase in search intensity or a decrease in reservation wages. If increases in the job finding rates are explained by decreases in reservation wages, those who have a shorter entitlement to unemployment benefits might accept jobs that do not match their qualifications, and from which they are more likely to quit in the future. If the increased job finding rates are explained by increases in the search effort, there is no reason to expect shortening the entitlement period of unemployment benefits forces unemployed individuals to find jobs that do not match their qualifications.
We found three studies that could be used for analysis of the exit rate from the re-employment job and three studies that could be used for analysis of the wage ratio in the re-employment job. Based on this low number of studies, we found no evidence to support the hypothesis that shortening the maximum entitlement to unemployment benefits has an impact on the quality of the job in terms of the exit rate from the re-employment job or the wage in the re-employment job. Whether the unemployed workers who are affected may actually be worse off, in the sense that they accept “worse” jobs, has not yet been fully investigated.
It was not possible to examine a number of factors which we have reasons to expect have an impact on the magnitude of the effect. Knowledge of whether the effect depends on labour market conditions and benefit system factors such as availability of alternative benefits, and compulsory activation may be crucial to policy-makers. The factors are all potential moderators of the effect that policy-makers need to attempt to assess in relation to the context of their country. These factors could possibly have been investigated with more studies from different labour markets and with a sufficiently low risk of bias. The results of this review, however, conclude that across a (small) number of countries there is a small positive effect of shortening the maximum potential benefit period on job finding rates.
6.2 IMPLICATIONS FOR RESEARCH
In this review we found evidence that reducing the maximum entitlement to unemployment benefits results in an increased probability of finding work faster, although the impact is small.
By excluding from the data synthesis studies judged to be at very high risk of bias, this review aimed to enhance the quality of the evidence on the effects of reducing unemployment benefit duration. We believe this process excluded those studies that are more likely to mislead than inform on the true effect sizes. Overall the risk of bias in the studies included in the review was high. Many of the available studies were judged to be at very high risk of bias. Twenty-five studies were given a score of 5 on the Confounding item, corresponding to a risk of bias so high that the findings should not be considered in the data synthesis. Of the remaining 13 studies, three were given a score of 5 on the Other bias item, corresponding to a risk of bias so high that the findings should not be considered in the data synthesis.
Some of the studies were judged to be at very high risk of bias, based the analysis of a regression discontinuity design relying on sharp discontinuities in age and/or work experience level in the Netherlands, Canada, Portugal, Spain, Chile, Hungary, Japan, Uruguay, Slovakia and Poland (and some studies in addition accounted for time trends using a difference-in-difference approach). These studies, however, failed to deliver convincing arguments that the identification strategies were not subject to too high risk of selection around cut off. In general, the authors using regression discontinuity designs did not provide arguments concerning agents' inability to precisely control the assignment variable near the known cut off and there was a lack of test of smoothness and density tests reported in the studies to support the designs chosen. Figures or tests of smoothness should be provided to support the assumption that all relevant factors (other than the treatment) are evolving smoothly with respect to the assignment variable and, likewise, figures or tests of density should be provided to assure that the density of the assignment variable is continuous at the discontinuity threshold.
Further, the main concern in a number of studies was the lack of information in the data sets used. In particular, studies using data from the US lacked information on whom among those eligible for unemployment insurance (UI) actually received UI (for example, the UI take up rate is around 50% according to for example Farber and Valetta (2015)). Further, in most of the studies using data from the US, entitlement (to ordinary UI) was set to the maximum (26 weeks) regardless of working history, as the data set used by the researchers did not include information on work history.
As studies from the US, the Netherlands, Canada, Portugal, Spain, Chile, Hungary, Japan, Uruguay, Slovakia and Poland could not be used in the data synthesis, the geographical coverage of the evidence of the effects of reducing the unemployment benefit duration became rather narrow, covering only four countries, all of which were European.
The planned examination of potential moderators of the effect, such as gender, age and labour market conditions, was not possible due to the low number of studies in the data synthesis. If effect sizes from all the countries represented in the review had been useable in the data synthesis, additional valuable information about the heterogeneous effects of reducing the maximum duration of entitlement to unemployment benefits may have resulted.
These considerations point to the need for future studies that more thoroughly discuss the identifying assumptions and justify their choice of method by considering and reporting all relevant data and tests. Further, future studies should rely on data where all relevant information is available, in particular rely on data were there is no lack of information on who among the unemployment benefit eligible individuals actually received unemployment benefits, and who among the unemployment benefit eligible individuals were entitled to the maximum duration.
The quality of the jobs obtained, in terms of duration and income, could not be fully investigated due to limitations in the number of studies reporting such outcomes.
Further research should be directed at the possible side effects, in particular whether the unemployed leave unemployment due to a higher acceptance of low-paid employment and, in particular, whether job transitions are caused by increased search effort.
7 Acknowledgements
We thank members of the review team at SFI Campbell, the research assistants Julie Kaas Seerup, Ulrik Højmark Pedersen and Bjørn Christian Viinholt Nielsen, for their invaluable help.
We would like to thank Dr. B. C. Reeves from the Cochrane Non-Randomised Studies Methods Group for materials and training regarding the assessment of risk of bias.
The review authors are responsible for any remaining errors.
8 Methods not implemented
8.1.1 Assessment of reporting bias
We were unable to comment on the possibility of publication bias because there were insufficient studies for the construction of funnel plots.
8.1.2 Moderator analysis and investigation of heterogeneity
We planned to investigate the following factors with the aim of explaining observed heterogeneity: Study-level summaries of participant characteristics (e.g. studies considering a specific age group, gender or educational level or studies where separate effects for men/women, young/old or low/high educational level are available), labour market conditions (good/bad), type of unemployment benefit (UI or SA/UA), whether alternative benefits are available, and if compulsory activation is part of the system.
There were, however, insufficient studies for moderator analysis to be performed.
Footnotes
9 References
10 Information about this review
11 Excluded studies
|
|
|
| Alba-Ramírez (1998) | Does not analyse maximum duration entitlement. Only considers receiving benefits or not. |
| Arranz & Muro (2004) |
Does not consider changes to maximum duration entitlement, only time to exhaustion. |
| van Audenrode & Storer (1992) | Only considers differences between eligible receivers/non receivers of unemployment benefits. |
| Barron & Wesley (1981) |
Does not analyse maximum duration entitlement. Only considers receiving benefits or not. |
| Bennmarker, Carling & Holmlund (2007) | Only changes to replacement rates analysed. |
| Bennmarker, Skans & Vikman (2013) | Soft restriction, reduction of duration of passive benefits, participation in active programs (with unchanged benefit level) qualifies for another period of passive benefits. |
|
Burgess & Kingston (1981)
|
Analyses effect on compensated unemployment duration only and not total time spend in unemployment. |
| Centeno & Novo (2014) | Does not separate exits to employment and exits to other destinations than employment. |
| Chetty (2008) |
Considers variation in benefit levels between US states, not variation in potential unemployment benefit duration. |
| Fujita (2011) | No effect of changes in maximum duration estimated, only differences in exit rates between a period where extended benefits were available/not available and further separated in duration intervals (monthly). |
|
Gaure, Røed & Westlie (2008)
|
Analyses a reform that is not primarily a change in the overall length of the maximum UI duration, but rather a soft duration constraint and a mix of other changes. |
| Gritz & MaCurdy (1992) | Do not separate exhaustion effects from max entitlement effects. |
| Grossman (1989) | Data on the number of weeks of unemployment compensation received, not on the number of weeks unemployed. |
|
Ham & Rea (1987)
|
Analyses effect of remaining entitlement. |
| Holen (1977) | Data on the number of weeks of unemployment compensation received, not on the number of weeks unemployed. |
| Hunt (1995) | Does not consider individual differences in maximum duration but differences between (large) age groups: Control group (16-41 years of age) has maximum benefit duration between 4-12 months dependent on age and tenure, treated (44-48 years of age) has maximum benefit duration between 4-22 months dependent on tenure. |
| Lalive (2008) | Results for job finding only reported in footnote (and with no standard errors) Two working papers do not consider job finding either, only all destinations after exit from unemployment. |
| Lalive, van Ours & Zweimuller (2011) |
Does not consider, or separate transitions to job from other transitions. |
| Lalive, van Ours & Zweimüller (2006) | 14 percent of spells end in non-job destinations and transitions to job not analysed separately. |
| Lauringson (2011) |
Estimates effect of covariates separately for those with short and long entitlement and shows baseline separately too. |
| Leigh (1086) | Analyses replacement rate (not including zero as only unemployment insurance benefit recipients included) effects only |
| Lindner (2015) |
Analyses changes to replacement rate; from flat rate to a decrease after 90 days. Maximum duration is unchanged. |
| Gritz & MaCurdy (1992) | Does not separate exhaustion effects from max entitlement effects. |
| Meyer (1990) | Does not separate exits to employment from exits to other destinations than employment. |
|
Moffitt & Nicholson (1982)
|
Only estimates effect of extended benefits on exhaustees. |
| Maani (1989) | No limit on benefit eligibility duration. |
| Narendranathan, Nickell & Stern (1985) | Analyses effect of income while unemployed, not entitlement and duration. |
| Pérez (2003) | Does not consider changes to entitlement, only benefits or not and do not mention a time limit on benefits. |
| Poterba et. Al (1995) | Only replacement rate effects are analysed. |
| Roed & Westlie (2012) | Analyses a reform that is not primarily a change in the overall length of the maximum UI duration. It is a soft duration constraint and a mix of other changes. |
| Rogers (1998) |
Analyses extended benefits in the week the extension becomes available. |
| Schmieder, von Wachter & Bender (2012) | Analyses the sum of unemployment spells over five years (recurrent unemployment spells over five years). |
| Steiner (2001) |
About negative duration dependence and not changes in benefit duration. |
| Moffitt (1995) | Data set 1) No exit to job, only exit from UI. Data set 3) Unemployment period not defined in a proper way (p. 75). Data set 4) Dependent variable is duration of receiving UI. |
| Vodopivec, Laporsek, Dolenc & Vodopivec (20)15 | Does not analyse potential unemployment benefit duration |
|
Wunnava & Henley (1987)
|
Does not consider changes to entitlement, only ‘exhaustion effect’ on average duration. |
|
Wunnava & Mehdi (1994)
|
Unclear if exit is to job. Examines insured unemployment rates and the duration of this unemployment period, suggesting not only exits to job. |
12 Appendices
13 Analysis
14 Data appendices
1
The replacement rate is the ratio of the unemployment benefit to that of previous earnings.
2
For a 40-year-old single worker without children and with a 22-year employment record.
3
The maximum duration was also around six months in the Czech Republic, the Slovak Republic, and the United Kingdom.
4
An alternative could be attaching behavioural conditions in terms of required job search and required acceptance of job offers to benefit receipt.
5
Extensions of benefits are included as well as long as there is a comparison of short versus longer entitlement periods.
6
The number of unfilled jobs expressed as a proportion of the labour force.
7
Extensions of benefits are included as well as long as there is a comparison of short versus longer entitlement periods.
8
Members of the review team at SFI Campbell were: the research assistants Julie Kaas Seerup, Ulrik Højmark Pedersen and Bjørn Christian Viinholt Nielsen.
9
This risk of bias model was introduced by Prof. Reeves at a workshop on risk of bias in non-randomised studies at SFI Campbell, February 2011. The model is a further development of work carried out in the Cochrane Non-Randomised Studies Method Group (NRSMG).
10
See next page for an explanation of the terms selection bias and confounding.
11
The reason for this is that unemployment implies a loss of skills or that long periods of unemployment lead to a loss of self-confidence. This “genuine” duration dependence should not be confused with sorting which is another mechanism.
13
The following description of estimation is based on Lancaster, 1990.
14
One study did not report hazard ratios but provided results measured as the difference in mean duration
15
One additional study (Winter-Ember, 1998) analysed unemployed in Austria in the same period but the study was coded with too high risk of bias and was excluded from the data synthesis.
16
Number of vacant jobs per unemployed
17
The study authors use quotations marks.
18
Avoiding overlap of data samples, 15 additional studies could have been used in the data synthesis.
19
Members of the review team at SFI Campbell were: the research assistants Julie Kaas Seerup, Ulrik Højmark Pedersen and Bjørn Christian Viinholt Nielsen.
20
This risk of bias model was introduced by Prof. Reeves at a workshop on risk of bias in non-randomised studies at SFI Campbell, February 2011. The model is a further development of work carried out in the Cochrane Non-Randomised Studies Method Group (NRSMG).
21
See user guide for unobservables