Abstract
This article shows how Qualitative Comparative Analysis (QCA) can be used to understand what works to address complex policy problems at a local level, using the example of tackling high rates of teenage conceptions in England’s most deprived local authority areas. QCA is a promising method for providing evidence in situations where interventions interact with contexts, enabling causal pathways to be discerned from how sets of conditions combine with particular outcomes: in this instance, whether inequalities in conception rates do or do not narrow, compared with the England average. A wide range of survey and secondary data, sourced in collaboration with practitioners, was explored to identify conditions that might show a relationship with the outcome. Applying QCA’s process of logical reduction enabled identification of sets of cases. Two narrowing sets and three not-narrowing sets are presented, showing how there are different pathways to narrowing and not-narrowing outcomes, and how conditions often combine to have causal effect. Although based on systematic cross-case comparison, the article also demonstrates the importance of judgement and interpretation in QCA.
Keywords
Introduction
This article discusses the method of Qualitative Comparative Analysis (QCA) as an approach that is particularly apt for producing evidence about how to tackle complex policy problems that have the character of ‘wicked issues’ (Australian Public Services Commission, 2007; Blackman, 2006; Petticrew et al., 2009; Rittel and Webber, 1973). These are issues that pose significant challenges for intervention because of interdependencies between causes, wherein intervention is just one potential cause of an outcome. Understanding how interventions interact with the issue and its context means selecting a method able to deal with:
combinations of causes rather than independent effects; and
multiple causal pathways rather than one causal model.
The need for evidence about complex interventions is a particular challenge for public health and tackling health inequalities (Wanless, 2004). The UK Parliament’s Health Select Committee argued in its report of an inquiry into health inequalities that, ‘one of the major difficulties which has beset this inquiry, and indeed is holding back all those involved in trying to tackle health inequalities, is that it is nearly impossible to know what to do given the scarcity of good evidence and good evaluation of current policy’ (House of Commons Health Committee, 2009: 5). The Committee concluded that changes to the design of policy interventions could make all the difference and recommended that future initiatives adhere to a basic set of research guidelines, including piloting, randomization, controls and the use of quasi-experimental methods where randomization would be too costly.
This approach, however, reflected a particular view among the committee’s politicians, heavily influenced by dominant academic discourse in medical research, that the best evidence is from controlled experiments with random allocation. While an alternative discourse, largely in the social sciences, emphasizes methodological plurality, this is frequently contested even within the social sciences as failing to recognize the superiority of evidence from controlled trials compared to other methods (Macintyre, 2010). Indeed, a methodological hierarchy with randomized controlled trials at the top frames the evidence filtered by many systematic reviews influenced by the Cochrance evidence-based practice movement, resulting in a view that there is a lack of knowledge about interventions in public health because so many studies are not based on RCTs (Baxter et al., 2010; Dixon-Woods et al., 2006).
Among dissenting voices are Petticrew and Roberts (2003) who, while acknowledging a shortage of evidence about public-health interventions, argue that the issue is not one of too few RCTs but of methodological inaptness, or a failure to customize methods to complex real-world scenarios. The challenge is to model the world not as separate parts but as relationships between parts and between parts and wholes (Kooiman, 2003). To focus on the intervention as a cause, and not regard features of its context as also potentially causal (rather than factors to be held constant in a controlled way), is to consider just part of the story. Similarly, regarding causes and effects as independent and dependent variables is a long way from real causal pathways and how interventions really work (or not) (Petticrew et al., 2009).
Ragin (2000) makes a much more fundamental critique of the failure of policy research to take a whole-system perspective by rejecting the idea that variables can be disaggregated from cases and analysed separately as if it is the variables rather than the cases that are causal. For Ragin, it is the ‘case’ and the state of important conditions of each case that matter. All interventions are interventions in cases, which are bundles of conditions that interact together. While these conditions are often traceable across multiple cases using variables, it is not the variables that are the actors but the cases.
Ragin (2000, 2008) has developed the method of Qualitative Comparative Analysis (QCA) as a way of exploring causality among cases. This case-based rather than variable-based approach is informed by insights from the qualitative case study, but with the important addition of systematic cross-case comparison. This uses reasoning based on membership of sets and logical reduction of conditions until only those conditions that clearly differentiate between outcomes are included in explanations. The sets that are derived from this procedure represent causal pathways to outcomes. Below we illustrate the method by applying it to understanding differences in rates of teenage conception across local authority areas in England.
Teenage conceptions as a policy problem
Reducing teenage conceptions in England is an area of policy action across health, child poverty and social exclusion because of high conception rates compared to other European countries and their association with negative outcomes, including poverty, health problems for both mothers and babies, lower educational achievement and worklessness (Department for Communities and Local Government, 2007; Department for Education and Skills, 2006a, 2007). In 2004, the government introduced a national Public Service Agreement (PSA) target to reduce the under-18 pregnancy rate by 50 per cent by 2010 compared with 1998, with the aim of ‘faster progress’ in the most deprived areas (Audit Commission, 2010).
Data from the UK Office for National Statistics show that in 2010 the average conception rate in England was 35.4 per 1000 women aged under 18 (www.statistics.gov.uk). The national rate in 1998 was 46.6, so the 2010 rate is well short of the PSA target of 23.3. There has, however, been considerable variation across local authority areas: at local authority level the 2010 rate ranges from 6.2 in Rutland in the East Midlands of England to 64.7 in the London Borough of Haringey. High rates are concentrated in areas of deprivation. This presents an opportunity to learn from those areas with high deprivation where the decline has been faster.
Until a change of government following the UK general election in 2010, there was an area-based focus for reducing health inequalities in England based on 70 so-called ‘Spearhead’ areas, including teenage pregnancies. Spearheads were the local authorities in the bottom fifth nationally for three or more of the following five indicators: male life expectancy at birth; female life expectancy at birth; cancer mortality rate among under 75s; cardiovascular disease mortality rate among under 75s; and average score on the Index of Multiple Deprivation 2004 (Department of Health, 2004). An investigation by the Audit Commission (2010) identified that the gap in teenage conception rates between the Spearhead areas and the average for England increased slightly between 1998 and 2008, but it was noted that at the local area level there were both large increases in rates and notable falls. The report suggested that the relative success of local strategies could explain at least part of this variation.
Bradshaw et al. (2005) and the Department for Education and Skills (2006b) cite wide variation in local performance across areas with similar characteristics as grounds for arguing that the delivery of local policies and strategies is an important contributory factors in teenage conception outcomes. Bradshaw et al. (2005: 16) conclude that, ‘having checked the reliability of the data, controlled for deprivation and investigated the possibility of variation being explained by other sociocultural factors, the most likely reasons lie in the policies being pursued at a local level’.
Responding to this variation, a key government guidance document, Teenage Pregnancy Next Steps: Guidance for Local Authorities and Primary Care Trusts on Effective Delivery of Local Strategies, produced by the Department for Education and Skills (2006b), presented evidence about ‘what works’ to reduce teenage pregnancies. This was based on ‘deep dive’ reviews in three ‘high performing’ local authorities and three ‘statistical neighbours’ with static or increasing rates. The guidance sought to explain this variation by identifying factors that were evident in successful areas but absent in the others. Six factors were singled out: senior-level sponsorship and engagement of all key partners; provision of ‘young people-focused’ contraception and sexual health services; strong delivery of sex and relationship education in schools; targeted work with at-risk groups, in particular looked after children; workforce training on sex and relationship issues within mainstream partner agencies; and a well-resourced youth service with a clear remit to tackle teenage pregnancy and young people’s sexual health. These factors were said to be largely absent or being delivered less effectively in the comparison areas. Local areas were encouraged to review their strategies against these findings and to ensure that lessons were incorporated into planning.
The ‘deep dive’ reviews were not based on a rigorous comparative research design and, given the lack of methodological information, would not pass the standards of a systematic review even as a qualitative study. The approach, however, recognized teenage conceptions as a complex issue involving interactions between young people’s knowledge, their access to advice and support, and the influences of educational attainment, levels of emotional well-being and parental, cultural and peer influences.
Intervening in such complexity requires multi-agency working at local level. In that sense, the local area is a complex system with its level of teenage conceptions an emergent outcome of conditions in the area. A complex system can be thought of as a network of agents that are interconnected to fulfil a shared purpose or function (Plsek, 2001). At a local strategic level, three agents were important at the time of our study regarding teenage conceptions: the primary care trust (PCT), part of the National Health Service responsible for local planning and commissioning of health services; one or more local authorities in the area delivering a range of relevant public services including education, youth work and housing; and one or more local authority partnerships (LSPs), which were tasked with coordinating multi-agency strategies in local authority areas. All were required by government policy to work together to deliver the PSA for teenage pregnancies. The challenge for understanding performance in this context is to avoid reductionism to individual variables while recognizing that local decision-makers need to focus on specific ‘best practices’.
QCA generates sets of cases, posited as different pathways to outcomes that the case – a complex system – may take depending on its combination of conditions. Outcomes are the result not only of the intentions and actions of agents but also the contexts in which interventions take place (Teisman et al., 2009). This does not mean that each system or case has to be regarded uniquely, but it does mean that it is the case, and not variables that generalize across cases, which needs to be the focus of attention. This is not least because practitioners intervene in cases and not variables, despite many policy statements targeting variables, such as the teenage pregnancy rate itself. In other words, progress against this variable can only be made by changing the state of cases and their multiple conditions.
The ‘variable’ only summarizes progress across many cases, and potentially misleadingly if there is wide variation and too much significance is attached to an average that has no reality in any cases. While cases in the present example might be reduced to the individual young woman – a biological fact – it is in wider social systems that causation lies. There are many ways to frame the relevant system, and deciding on ‘the’ system is an impossible task when complex reality is one of nested and interpenetrating systems. However, we are helped in this task by the way that policy interventions define systems as, for example, local authority areas. The system is then defined along the lines of Plsek’s (2001) definition of agents with a shared purpose, and the system’s behaviour can be explored by investigating its bounded conditions, some of which (such as deprivation) are not intrinsic to the system but an outcome of interaction between the system and wider systems (such as the global financial system). The challenge becomes one of finding the conditions that matter, not through detailed qualitative narratives or isolation of ‘independent’ effects, although these may offer helpful clues, but through systematic comparison.
Buijs et al. (2009: 45)
helpfully sum up the value of QCA in this respect as not comparing single variables but
configurations of case conditions: This allows for the specification of complex and contingent causes (because it does not
centre on isolating variables), which are however not unique, but may in fact be shared
across a number of cases. This allows the researcher to develop knowledge beyond the
detailed ideographic description of unique instances.
This case-based approach to generalization supports explaining complex causation by utilizing within-case analysis and cross-case comparison (Bennett and Elman, 2006: 259). It enables what are known as ‘set-theoretic methods’ to be used, the foundation of QCA and a term that captures its approach of logical reasoning and theoretical argument based on case membership of sets (Fiss, 2009). In policy-related terms the ‘state’ of a case, such as a local area targeted for intervention, can be defined by conditions that describe such attributes as local ways of working, types of intervention and features of the local context. QCA enables their combined effects to be explored using explicit descriptors of the conditions. These combinations are pathways to outcomes that may be single or multiple for the same outcome (Rihoux and Ragin, 2009). This is a whole-system approach that traces the processes of change that lead to an outcome (George and Bennett, 2005). Specific attention is paid to contextual conditions and, reflecting a realist perspective, these are regarded as likely to interact with interventions to produce alternative pathways (Pawson and Tilley, 1997).
Using QCA to explore teenage conceptions as an outcome of causal combinations
By way of illustration, the following example is taken from a study funded by the UK National Institute for Health Research that used QCA to investigate differences across Spearhead areas in three outcomes: premature cancer and cardiovascular disease mortality rates, and teenage pregnancy rates (Blackman, 2013; Blackman et al., 2011).
QCA regards cases as combinations of conditions, which include the outcome of interest. The first tasks are therefore to define the outcome and identify the conditions that are thought to matter to it, and then to gather data on these conditions for each case. In policy-related research, the selection of conditions for analysis should be informed by both formal and tacit knowledge about the topic, derived from relevant literature and co-working with practitioners who participate jointly in identifying conditions that are agreed as possible candidates for the ‘theory of change’ (Weiss, 1998).
The outcome for each Spearhead area was defined as having one of two states:
a narrowing teenage pregnancy rate gap compared to the national average; or
a gap that had not been narrowing.
This was based on a combination of approaches to judge the trend, including visualization in graphs for each case and a calculation of the absolute and relative differences between 2005 and 2009 (this end year cannot be updated further because of a change in local authority boundaries effective from 2010). Our analysis is therefore looking for quite short-term effects based on assessments of local conditions made for 2005.
All Spearhead PCTs in England and their partner local authorities and LSPs were invited to participate in the study. To gather much of the data needed, they were requested to complete questionnaires in teams of at least three people comprising the teenage-pregnancy coordinator (a nationally defined role), a public-health professional or local authority officer with relevant responsibility, and a sexual health lead professional from the NHS. Respondents were not aware at the time they completed the questionnaires of how their areas had been classified against the teenage-pregnancy outcome used in the study. Thirty-one out of 70 Spearhead areas returned the questionnaires, with complete returns from 27. The response rate reflected a shortage of time in many areas to complete the exercise, but a Mann-Whitney test using the five variables that determine Spearhead status showed no significant difference between the areas participating in the study and the non-responders. The teams were asked to provide assessments for current conditions in their area (2008) and for three years ago (2005), and to adopt a whole-systems view across preventative and treatment services and NHS, local authority and voluntary services, justifying their answers with examples and supporting documentation. To take account of the publication delay with teenage conceptions data and the time it can take for interventions to have an impact, the assessments for 2005 were used in the analysis and compared with the 2009 difference between the local teenage conception rate and the national rate, binarized as narrowing or not narrowing.
The questionnaire design was informed by a range of academic and policy literature. 1 The questionnaire was intended to cover the range of strategic-level decisions and conditions in Spearhead areas that could impact on progress with tackling teenage conceptions and was divided into two sections. 2 The first section focused on approaches to policy and practice and included eight statements against which respondents were asked to assess their areas using detailed scaled descriptors of ‘basic’ to ‘excellent’ practice covering: identifying, understanding and targeting teenage conceptions; processes of planning and commissioning services; strategic partnership working across sectors; partnership working on the ground; processes for community and young people’s involvement; workforce planning and development; contraception and sexual health services; and other interventions. The second part of the questionnaire focused on ways of working, such as approaches to prioritization, monitoring and joint working; types of intervention, such as in school or community settings; and the local context of the Spearhead area, such as features of local leadership, culture and partnerships.
Other conditions required gathering secondary data for each Spearhead area, including organizational performance assessment ratings made by national audit bodies for both the PCTs and local authorities; deprivation scores, crime rates, migration data and a measure of local ‘liveability’ (Collinge et al., 2005); health services data including PCT expenditure, the proportion of primary-care practices classified as outliers on national performance measures, the proportion of single-handed practices, and the number of general practitioners (GPs) per 100,000 population; a wide variety of educational attainment data; and demographic data from the 2001 population census.
Practitioners were consulted at a series of regional workshops about the structure, content and phrasing of the questionnaires and the content of the secondary dataset. They were mainly public-health professionals and local government officers. Following this feedback, the Department of Health’s National Support Teams (NSTs) for teenage pregnancy and health inequalities provided further detailed comments. This helped to ensure that all relevant conditions from a practitioner standpoint were considered and that the questionnaire was interpretable with results likely to be actionable.
QCA allocates cases to sets, which are shared configurations of conditions, enabling possible causal pathways to be identified. It involves making set-theoretic logical arguments, the clarity of which is clearer if case conditions are dichotomized as ‘present’ or ‘absent’. Dichotomization is not just a simplifying technique but reflects practical decision-making realities that are often binary ‘do this or do that’ scenarios (Blackman, 2012). We therefore used dichotomous measurement in what is known appropriately as the ‘crisp-set’ QCA technique rather than the more complicated ‘fuzzy set’ QCA that uses continuous and ordinal values (Ragin, 2000). In other words, it is the presence or absence of a ‘quality’ that we focus upon, using the same assessment across all cases. As De Meur et al. (2009: 149) explain: ‘in short, dichotomization allows us, through simplification (the operationalization of conditions), to conduct a rigorous comparison of a limited number of cases that present combinations of internally complex characteristics’.
The data available were often ordinal or continuous, so dichotomization becomes part of the analysis process. The majority of the questions in the questionnaire used scales and the remainder had categories with some yes/no answers. The secondary data comprised continuous or categorical variables. The data were coded for analysis using SPSS and QCA software developed by Ragin. 3 In order to produce crisp sets (i.e. sets of cases with the same profile of conditions being either present or absent) we recoded continuous and ordinal variables as binary. Our thresholds for dichotomization were based on where a change in relationship with the outcome occurred, a process requiring judgement and discussion. For example, the ‘narrowing’ outcome was clustered in the lower range of IMD scores, which provided the basis for binarizing the scores into higher and lower. While this was not always clear cut – there were for instance some cases with higher IMD scores that also had narrowing outcomes – it was possible for two researchers independently to agree where a transition occurred between an outcome being common and being rare. Robustness checks were also undertaken to determine how binarization thresholds affected the findings, and the thresholds we used were found to be robust.
Only a relatively small number of conditions were found to have any patterned relationship with the outcome indicators and these could all be dichotomized based on the above process. The binary variables were imported from SPSS into QCA software, enabling us to explore the effects of different states in combination. Some other conditions, which were without a patterned relationship but thought on the basis of existing knowledge to be important, were also tested in combination. Some conditions were contextual, such as higher deprivation being present or absent in a case, while others were practice-based, such as a ‘good practice’ descriptor of how services were commissioned being assessed as present or absent.
Results
Our results suggested a number of pathways both to ‘narrowing’ and ‘not narrowing’ the teenage conceptions gap; these were discovered using set-theoretic logic to identify plausible configurations of conditions from dichotomized data tables. These were shared with practitioners, using the findings as a ‘tin opener’ for them to help us develop accounts of causality in more detail. Not only did this provide a valuable knowledge exchange opportunity, it also enabled us to incorporate insights from practice into our explanations so that these could be grounded in practitioners’ worlds.
Nine conditions were included in the QCA. A full description of these conditions is given in Table 1. Table 2 shows the output from running the QCA procedure. This reduced the ‘causal’ combinations among the 27 cases to five sets using, in different combinations, four of the conditions (i.e. only these conditions were needed to differentiate between cases with narrowing and not-narrowing outcomes, configuring into five pathways). This is a ‘parsimonious’ solution in QCA that allows ‘counterfactual’ cases to be included in the process, which are combinations of conditions that are logically possible but not found in the empirical cases (Ragin, 2008). ‘Intermediate’ solutions – those that are still based on logical reduction but retaining conditions where theoretical or substantive knowledge suggests they contribute to an explanation – are often preferred because they can be more interpretable than parsimonious solutions. This was not the case with this analysis, where the parsimonious reduction produced findings that are consistent and with no contradictory cases. Of the five sets, two have a teenage-pregnancies gap that was narrowing and three have a gap that was not narrowing.
Teenage conceptions: conditions and descriptors associated with narrowing gaps.
Boolean minimization of the truth table identifying five sets (1=present; 0=absent).
Considering the narrowing sets first, what is striking about the first of these sets (set 1 in Table 2) is that there is just one sufficient condition for the narrowing outcome to be present: when the case has a higher proportion of BME groups in its population. This is not, however, a necessary condition because in the second narrowing set, set 2, this condition is not present. Instead we have a combination of lower numbers in drug treatment, a higher proportion of under 18s, and a ‘basic’ standard of commissioning. This combination is sufficient for a narrowing gap to be present, without the apparent advantage of a higher proportion of BME groups, but none of the conditions on their own are necessary: what matters is the combination, and this is sufficient for a narrowing outcome. It is also striking in set 2 that the relationship with commissioning is counter-intuitive; it is a basic standard of commissioning rather than the higher standard of good or exemplary commissioning that is associated with a narrowing outcome.
Turning to the not-narrowing sets, there are three configurations sufficient for this outcome. The first, set 3, is a combination of a lower proportion of BME groups in the population and a lower proportion of under 18s. The second, set 4, is a combination of lower BMEs and higher numbers in drug treatment. The third, set 5, is a combination of lower BMEs and a good or exemplary standard of commissioning. It is not surprising to see the presence of a lower proportion of BMEs associated with a not-narrowing gap given the make-up of the two narrowing sets. This condition is necessary for the outcome of not-narrowing to be present in any of the three configurations, but it is not sufficient on its own.
Discussion
The result for the BME condition is striking. In the narrowing set where this is a single sufficient condition there is something about having a higher proportion of BME groups in the population to start with that appears to bring about a narrowing of the gap. If we return to the individual cases, five out of the eight local areas in this set were among those with higher teenage conception rates at baseline, but this is not enough to explain away the higher BME effect as one of higher rates coming down faster. This conclusion is further supported by the presence of lower BME representation in the populations of areas with not-narrowing outcomes because these spanned a variety of initial teenage conception rates.
There is evidence that teenage conception rates have been relatively high among young people from some BME backgrounds (French et al., 2005; Teenage Pregnancy Unit, 2005). While no evidence could be identified from the research or policy literature of faster progress with reducing rates among these groups, our results suggest this may be what has been happening. We can only speculate about the reasons: increased awareness of the risk of sexually transmitted infections and rising educational aspirations and achievements are possibilities. What is strange is that the issue has not received more attention. We note, for example, that the last government’s review of England’s national teenage pregnancy strategy has no consideration of ethnic factors (DCSF/DH, 2010).
Other interesting conditions that appear to be causal are the proportion of the local population in drug treatment and the proportion of under 18s. The former could be regarded as more than just an indicator of the level of substance misuse and to be tracing risk-taking generally, including teenage sexual behaviour, given that risky behaviours are known to cluster across domains (Coleman, 2002; Seamark and Gray, 1998). The under 18s condition is more difficult to interpret. A higher proportion of under 18s in the local population is associated with a narrowing gap and may mean that these areas are more likely to provide services for this group that divert young people from risk-taking, including local sexual health provision. Unfortunately a services indicator of this type is not available so this explanation has to be speculative. What we can say is that there is something about a higher proportion of under 18s that appears to narrow the teenage conceptions gap, although only in combination with lower numbers in drug treatment and a basic standard of commissioning.
The counter-intuitive result for basic commissioning is intriguing. It is, however, important to note that none of the Spearhead areas used the option of assessing their commissioning as ‘less than basic’ so this is about having basic practice in place, which is associated with the narrowing outcome in set 2. The surprising aspect is that this condition is not ‘good’ or ‘exemplary’ practice. In fact, in set 5 we see this associated with a not-narrowing outcome. This is a similar finding to other work by the authors regarding inequalities in premature cancer mortality across Spearhead areas (Blackman et al., 2011). We argue that certain conditions are ‘bureaucratic’ in the sense that they involve a strong emphasis on process: planning, coordination, and monitoring compliance with ‘best practice’. They entail considerable effort devoted to meetings, plans and paperwork especially if done to excess. This leads us to a theory that attributes of this kind may distract effort from a focus on the outcomes.
The idea that being better than basic in this respect might be dysfunctional found support among many of the practitioners in our workshops. It is echoed in Travers’ (2007) study of ‘the new bureaucracy’ which he argues grew in the UK and other countries with the rise of an audit culture as governments, from the 1980s, took less direct control of local services and exercised control instead through incentives and sanctions attached to performance assessments, taking time from ‘normal work’ without adding anything (Clarke, 2006; Parker and Bradley, 2004; Travers, 2007). Seddon (2005, 2008) has explored this issue, arguing that an audit culture focuses systems on compliance with targets and prescribed processes rather than learning how to match capability to outcomes. Achieving good or exemplary performance with processes of commissioning is likely to entail an opportunity cost of time and resources that could otherwise be focused on ‘normal work’ to achieve the outcome. That ‘normal work’ in set 2 appears to have something to do with effective diversion from risky behaviours in areas where under 18s make up a higher proportion of the local population and services may therefore have the critical mass needed for this kind of impact.
Conclusions
Although QCA is an exploratory technique, it is concerned with identifying causal pathways and a number of issues in this respect should be considered. The first is that it is based on associations and relies on substantive and theoretical arguments to justify these as causal. Secondly, we used a relatively short time period between assessments of conditions and the outcomes postulated as caused by combinations of these conditions. This is better than simply taking a snapshot of the gap for a single time point and was a practical decision based on 2005 being the base year, the year for which respondents were asked to make their assessments of local conditions, while 2009 was the latest year for which local teenage conception data were available. We are therefore looking at short trends and assuming that these will either continue into the future and/or are affected by the state of causal conditions in 2005. Thirdly, the crisp-set QCA used in this study reduces quantitative and qualitative data to binary attributes, with this dichotomization based to a large degree on judgement. This is judgement that, nevertheless, makes clear distinctions between what does and does not matter that are practically useful and actionable. QCA is not alone as a method that involves simplifying assumptions and has the advantage that these are made transparent.
A fourth issue is that although QCA was developed for small N studies, when the total population is not included (as in our case) it is possible that the addition of missing cases would change the results. This might be regarded simply as an empirical reality (Spitzlinger, 2006). We did nevertheless carefully compare the cases for which we obtained survey data with the non-responders using an extensive set of available secondary data and found no significant differences.
The role of practitioners was important in the study, from assisting with identifying and selecting conditions for analysis to interpreting results. The experiential knowledge of practitioners about ‘their’ areas was used alongside the systematic analysis of conditions, trends and inter-relationships (Head, 2008). Returning to individual cases is an important dimension of QCA studies, either to understand anomalies or to develop general explanations. For example, although it was not possible within the scope of this study to explore further the services and risk diversion aspect that we suggest was traced by the drug-treatment and under-18 conditions, the study creates a framework for doing this. We would expect to find better services for young people in the areas with lower numbers in drug treatment and a higher proportion of under 18s in the local population, as well as commissioning that is ‘good enough’ but not so good that it distracts from delivering these services.
The configurational and set-theoretic approach of QCA is an alternative to the logic of the variable as causal agent (Byrne, 1998; Ragin, 2008). In other words, there is a fundamental distinction between QCA and conventional variable-based approaches. QCA works with a particular conception of causality based on multiple conjunctural causation, by which is meant outcomes emerging from distinctive combinations of causes that are revealed in types of case and not independent variable effects (Berg-Schlosser et al., 2009). This is a non-linear, non-additive and non-probabilistic conception that stresses diversity, complex combinations of conditions and equifinality: different paths can lead to the same outcome. The method provides a basis for the qualitative consideration of complex policy problems based on like-with-like comparisons between cases, improving the robustness of case-study research by using a systematic approach.
Footnotes
Acknowledgements
The views expressed in this report are those of the authors and do not necessarily reflect the views of the NHS or Department of Health.
Funding
Notes