Abstract
Objectives
To compare the cost effectiveness of adding a pancreatitis-associated protein (PAP) assay to common immunoreactive trypsinogen (IRT) and DNA cystic fibrosis (CF) newborn screening strategies.
Methods
Using data collected on 553,167 newborns, PAP cut-offs were calculated based on non-inferiority of the detection rates of classical forms of CF. Cost effectiveness was considered from the third-party payer's perspective using only direct medical costs, and the unit costs of PAP assays were assessed based on a micro-costing study. Robustness of the cost-effectiveness estimates was assessed, taking the secondary outcomes of screening (ie. detecting mild forms and CF carriers) into account.
Results
IRT/DNA, IRT/PAP, and IRT/PAP/DNA strategies had similar detection rates for classical forms of CF, but the strategies involving PAP assays detected smaller numbers of mild forms of CF. The IRT/PAP strategy was cost-effective in comparison with either IRT/DNA or IRT/PAP/DNA. IRT/PAP/DNA screening was cost-effective in comparison with IRT/DNA if relatively low value was assumed to be attached to the identification of CF carriers.
Conclusions
IRT/PAP strategies could be strictly cost-effective, but dropping DNA would mean the test could not detect CF carriers. IRT/PAP/DNA strategies could be a viable option as they are significantly less costly than IRT/DNA, but still allow CF carrier detection.
Introduction
Newborn screening (NBS) for cystic fibrosis (CF) is widespread and has been shown to be cost-effective.1,2,3 Most programmes use immunoreactive trypsinogen (IRT) assays on dried blood specimens 4 and DNA analysis, 5 in either a two- or three-tier protocol (IRT/IRT, IRT/DNA, or IRT/IRT/DNA). DNA analyses detect both classical and mild forms of CF, as well as CF mutation carriers. There is some disagreement over whether identifying CF carriers is of greater importance than the possible psychological effects.6–11
Several studies have confirmed the clinical relevance of pancreatitis-associated protein (PAP) in CF NBS, and the possibility of performing PAP assays on the same samples as used for the initial IRT test has been investigated.12–18 A study based on theoretical data from the Quebec region showed that an IRT/PAP strategy could be viable in terms of costs per detected case, although detection rates of classical forms of CF would be lower compared with those from IRT/DNA strategies. 3 The study used various IRT and PAP cut-off values, acknowledging that the cost-effectiveness results depended on the choice of cut-offs. More generalizable cost-effectiveness findings could be derived if optimal combinations of IRT and PAP cut-offs were identified.
We assessed the cost-effectiveness of IRT/DNA, IRT/PAP, and IRT/PAP/DNA strategies using data prospectively collected on two thirds of all French newborns in 2010. PAP cut-offs were calculated based on non-inferiority of the detection rates of classical forms of CF in IRT/DNA and IRT/PAP strategies. 18
Methods
Using data collected in a prospective clinical study of 553,167 newborns,
18
we compared the following screening strategies.
IRT/DNA. (Current practice in France) IRT was assayed on blood eluted from screening cards. At concentrations ≥ 55 µg/L, the assay was repeated on the same card. If the mean value of the duplicate IRT measurements was at least 65 µg/L, DNA analysis was performed (Elucigene™ CF30). If one or two mutations were detected, sweat tests were performed. If no mutations were detected, a new blood sample for IRT assay was collected on day 21, from newborns with IRT ≥ 100 µg/L and a sweat test was performed if IRT ≥ 40 µg/L. IRT/PAP. To maximize comparability between strategies, IRT/PAP was assessed based on the same 55 µg/L IRT cut-off as that used with IRT/DNA, although the IRT cut-off was set at 50 µg/L in the clinical study.
18
PAP was assayed in duplicate in all newborns with IRT ≥ 50 µg/L, (ELISA kit, Dynabio, France). Referral of newborns for sweat tests involved cut-offs based on non-inferiority of the detection rate of the classical forms of CF.
18
Optimum PAP cut-off values were then identified at 1.8 µg/L in all cases where the IRT values ranged between 50 µg/L and 100 µg/L, and at 0.6 µg/L in those where the IRT values were above 100 µg/L. IRT/PAP/DNA. This strategy was simulated using the data collected in the clinical study, using the cut-offs adopted in the IRT/DNA and IRT/PAP strategies. Because the cut-off used in the IRT/DNA strategy did not require collecting DNA on the 238 newborns with 55 µg/L < IRT < 65 µg/L, the distribution of the DNA mutations in these newborns was assumed to be similar to that observed in the 647 newborns with IRT ≥ 65 µg/L. The missing data cannot have affected the detection rates or total costs because all 238 newborns were found to be true negatives.
The primary endpoint for assessing the effectiveness of the screening strategies was the detection of classical forms of CF. The detection of mild and heterozygous forms of CF were considered secondary outcomes. Because the PAP cut-offs were designed to prevent inferiority of IRT/PAP compared with IRT/DNA, these two strategies gave similar detection rates of classical forms. 18 The effectiveness of IRT/PAP/DNA was assessed using the cut-offs described above.
Costs of the screening strategies were assessed from the perspective of the third-party payer (the French national health insurance system). Only direct medical costs were taken into account. Except for the PAP assay, the cost analysis involved measuring the physical quantities of the resources individually consumed (IRT, DNA, PAP, sweat tests, consultations with a pediatrician in the case of newborns with CF), combined with a monetary assessment based on the tariffs of the French national health insurance system. 19 Because no tariff is available for PAP assays, a micro-costing study was conducted based on direct observations of the time, equipment and consumables (disposables and reagents) required.19,20 Technicians and biologists were consulted to identify the equipment used, professional qualifications, organizational constraints, etc. for laboratories. Cost calculations were based on the purchase price of equipment and supplies (with or without a 25% price reduction as compared with the price list) and on the standard costs of the technicians, secretary, and physicians involved. As is standard, a 5% discount was applied to account for the opportunity costs of investment in equipment. The overhead costs were provided by the Accountancy Department of the hospital at which the PAP measurement process was analyzed. We did not include the cost of premises, because of the great variability among regions.
Average costs were estimated as a function of the sample throughput (duplicate assays), ranging from one sample per year to an annual level of activity requiring a full-time technician performing PAP measurements. Two options were tested for the allocation of equipment costs: i) equipment dedicated to PAP assays regardless of the activity levels, and ii) equipment assigned to PAP depending on the activity levels. The possibility of having to repeat PAP assays due to sampling problems or unexpected results was rated at 2% after consulting the technicians and biologists involved. Additional costs resulting from repeated steps applied to the cost of technician time, equipment, disposables and reagents were assessed. A criterion defined by average cost stability (±1% variation) despite variations in the annual activity rates was used as a marker of perennial laboratory activity. Sensitivity analyses were conducted, in which several key parameters were varied (allocation of equipment, repeat rate, discount rate).
The cost-effectiveness analysis compared, for each CF NBS strategy, the total costs of screening with the expected outcome of this resource allocation (detection of classical forms). As the same newborns were included in the IRT/DNA, IRT/PAP, and IRT/PAP/DNA strategies, the cost data could not be compared using standard statistical methods. To deal with the correlation between data, bootstrap methods were used. 21 Variability in total costs and statistical significance of the differences in total costs between strategies were then assessed, based on bootstrap analyses, with 10,000 replications to construct an empirical distribution of the data obtained. As the study sample covered 69.7% of all French newborns in 2010, the confidence intervals relating to the detection rates and costs were calculated using the finite population correction factor, to reduce the standard error by a value equal to [(N-n)/(N-1)]1/2 (where N and n are the size of the 2010 French newborn cohort and that of the study sample, respectively).
Robustness of the cost-effectiveness estimates was assessed, taking the secondary outcomes of CF NBS (ie. mild forms and carriers) into account. Neglecting these outcomes would amount to assuming that they have no value. The cost-effectiveness analysis was therefore repeated, assuming that detecting these outcomes yielded either a negative or a positive value in comparison with the value attributed to the detection of a newborn with CF. Assuming the value assigned to the detection of a newborn with CF to be 1, values ranging from −1 to 1 (with a step of 0.1) were attributed successively to the secondary outcomes. To assess in each case whether one strategy was more effective than another, detection rates weighted by the values assigned to the primary and secondary outcomes were used.
Results
Of the 553,167 newborns in the clinical study, 252 were subsequently excluded (Figure 1).
18
The cost-effectiveness study was therefore conducted on data from 552,915 newborns.
Flow chart of newborn CF screening based on IRT/DNA and IRT/PAP strategies.
CF was identified in 95 cases (74 classical, 12 mild, nine cases of meconium ileus).
18
Of the 12 mild cases, all had normal or borderline sweat test results except one, who had an R117H mutation.
8
Using the IRT/DNA strategy, 68 newborns with classical forms of CF were detected (91.9% detection rate, 95% CI: 87.1%–94.7%), 12 with mild forms, five with meconium ileus, and 165 CF carriers (Figure 1A). With the IRT/PAP strategy, 69 newborns with classical forms were detected (93.2%, 95% CI: 88.6%–95.7%), five with mild forms, and six with meconium ileus (Figure 1A). Using the IRT/PAP/DNA strategy (Figure 1B), 66 newborns with classical forms, five with mild forms, six with meconium ileus, and 165 carriers were identified. The detection rate of classical forms (89.2%, 95% CI: 84.0%–92.5%) was statistically similar to the rates obtained with the IRT/PAP and IRT/DNA strategies.
Flow chart of newborn CF screening based on the simulated IRT/PAP/DNA strategy.
While a full-time technician could perform an estimated maximum of 28,160 PAP assays/year (four 96-well microplates/day, 80 wells used for PAP assays, corresponding to 40 newborns tested), the average costs of PAP measurements stabilized at an annual activity of 1480 or 1640 newborns tested with dedicated equipment and 920 or 1040 newborns tested if the equipment was shared (Figure 2) (depending on whether price discounts were assumed to be obtained on equipment and supplies). At these levels of activity, a 10% variation in volume was associated with a 3–4% variation in average costs with dedicated equipment, or 2–3% with shared equipment.
Average costs depending on the annual rates of PAP assays conducted on shared versus dedicated equipment.
Average cost of PAP measurements associated with annual volumes consistent with average cost stability in the case of shared equipment versus a dedicated PAP-only platform.
Note: The MUCOPAP kit (13.73€) accounted for 98% of the cost of disposables and reagents and 40%–60% of the average costs, depending on the hypotheses adopted about the production and prices.
Because all three strategies (IRT/DNA, IRT/PAP, IRT/PAP/DNA) yielded similar detection rates for classical forms, the cost-effectiveness analysis could be restricted to a cost-minimization analysis. The IRT/PAP strategy required significantly lower resources than IRT/DNA, and was cost-effective if each PAP assay cost less than 55 euros (at this cost the 95% CI difference in total costs was: −3,206.9 € + 17,029.8 €). For secondary outcomes (mild and heterozygous forms of CF), the IRT/PAP strategy remained cost-effective in 88% of cases assessed, as significantly higher or similar weighted detection rates were associated with lower screening costs (Table 3). The cost-effectiveness of this strategy in the remaining cases was unclear (corresponding to a slightly negative value attached to the detection of mild forms whereas a highly positive value attached to the detection of CF carriers), as IRT/PAP yielded significantly lower weighted detection rates than IRT/DNA, while being less costly. The use of an IRT cut-off of 50 instead of 55 µg/L in the IRT/PAP strategy would not have affected the findings of the study (two additional meconium ileus detected, 4.7% increase in total costs).
The IRT/DNA and IRT/PAP strategies: cost of screening and diagnosis and the corresponding outcomes.
Cost calculations were based on the tariffs applied by the French national health insurance system for IRT (1.7€), DNA (135.79€), sweat tests (13.5€), and consultations with a pediatrician (25€). The unit cost of 17.65€ for PAP assays was calculated based on the micro-costing study.
Assessments based on the annual incidence of CF carriers of 1 newborn per 38.6, given the annual incidence of CF of 1/5,950 newborns observed in the present study.
Robustness analysis of the cost-effectiveness of IRT/PAP and IRT/DNA as CF screening strategies.
Taking the value (the utility) attached to the diagnosis of a newborn with CF to be equal to 1, attributing a value of −1 to the identification of a newborn with a mild form means that the disutility attached to the diagnosis of a mild form equals the utility attached to the detection of a case of CF. A value of 0 means that no value is attributed to the detection of a mild form.
Taking the value (the utility) attached to the diagnosis of a newborn with CF to be equal to 1, attributing a value of 0.5 to the identification of a CF carrier means that the utility attributed to the detection of a CF carrier is half of that attributed to the detection of CF.
Note: Results based on values of less than 0 corresponding to the detection of a CF carrier or more than 0 corresponding to the detection of a mild form are not presented. They would only have contributed to increasing the number of cases for which the IRT/PAP strategy gave significantly higher weighted detection rates.
Discussion
Our results show that IRT/PAP strategies have the potential to be strictly cost-effective. In comparison with IRT/DNA, the IRT/PAP strategy was cost-effective even with PAP cost values up to three times higher than the 17.65€ adopted here. When the secondary outcomes of CF NBS (detection of mild and heterozygous forms) were taken into account, IRT/PAP was still cost-effective in most of the cases assessed. For the IRT/PAP/DNA strategy simulated based on the data collected, 18 the slightly (but not statistically significant) lower detection rate of classical forms of CF obtained was in line with the findings from two large-scale prospective studies, in which IRT/PAP and IRT/PAP/DNA were compared.14,16 IRT/PAP/DNA was not cost-effective compared with IRT/PAP, but was cost-effective compared with IRT/DNA, if a relatively low value is attached to the identification of CF carriers.
These results confirmed that average costs per detected newborn with CF could be lowered with IRT/PAP-based strategies. 3 In addition, IRT/PAP-based strategies could be strictly dominant strategies, being as effective and significantly less costly than either IRT/DNA or IRT/PAP/DNA, if PAP cut-offs are set based on non-inferiority. Comparing the cost of CF NBS between studies is difficult because of the differences between the cut-off values used to classify newborns depending on their CF risk,22–24 and between the various national healthcare systems and health insurance pricing practices. The average cost per newborn screened using the IRT/DNA approach in our study was similar to that reported in a previous French study, 25 although it was significantly lower than that reported in other studies.26–29 Higher unit costs (DNA, IRT and sweat tests, pediatricians’ consultations) would not affect our findings, as they are based on comparisons between options that were valued using the same unit cost data. In particular, the 20% difference in the total costs between the IRT/DNA and IRT/PAP strategies was in a similar range to those reported in previous studies (differences of 15% or 45%).28,29
This study has several limitations. First, the prospective clinical study on which it is based was not designed to assess either the IRT/IRT strategy, although it may not be cost-effective, 29 or the IRT/PAP/DNA strategy. By adopting reasonable assumptions about the distribution of DNA mutations in newborns, it was possible to assess the cost-effectiveness of a simulated IRT/PAP/DNA strategy based on data from the prospective clinical study. Another limitation is the use of a 30 gene mutation panel for DNA testing, although much larger mutation panels are available. However, achieving a 100% detection rate following DNA testing would result in IRT/DNA and IRT/PAP/DNA yielding similar detection rates to those obtained with IRT/PAP (94.59% and 93.24% respectively, see Figure 1). A further limitation is that the clinical study on which this one was based sampled newborns at age three days. Although earlier sampling would have been possible, there are questions about the PAP kinetics before 48 hours. 30 A final limitation is that the cost of the genetic counseling and analyses generated in the families of heterozygous newborns was not included in our calculations. Adding these costs would increase the difference in cost in favour of the IRT/PAP strategy, although the impact of this cost component was expected to be low (165 CF carriers). By comparison, the impact of genetic counseling in a recent study was only 0.6% of the total costs. 29
In conclusion, the detection rate of classical forms of CF obtained with the IRT/PAP strategy was similar to those obtained with IRT/DNA and IRT/PAP/DNA, but the IRT/PAP strategy required fewer resources and led to the detection of a smaller number of mild forms compared with IRT/DNA. Switching from IRT/DNA to an IRT/PAP strategy would mean giving up the possibility of identifying CF carriers, although the parents of newborns attach some value to this outcome.6,9–11 Taking this into account, IRT/PAP/DNA could be a viable option, as this strategy still makes it possible to detect CF carriers, while being significantly less costly than IRT/DNA.
Footnotes
Conflicts of interest
None.
Funding
This study was supported by the French National Health Insurance Fund (Caisse Nationale d’Assurance Maladie des Travailleurs Salariés, CNAMTS-AREDEMAG) under grant number II/6/2009/CT/96.
Research ethics
An anonymized data set was used - ethical approval was not required.
