The effect of omitting an early population-based vision screen in the Netherlands: A micro-simulation model approach

Abstract

Objective

To estimate the effect of omitting an individual screen from a child vision screening programme on the detection of amblyopia in the Netherlands. A previous study (Rotterdam Amblyopia Screening Effectiveness Study) suggested that the three screens carried out between 6 and 24 months contributed little.

Methods

We developed a micro-simulation model that approximated the birth-cohort data from the previous study, in which 2964 children had completed follow-up at age 7, and 100 amblyopia cases were detected. Detailed data on screens, referrals, and orthoptic follow-up, including the cause of amblyopia, were available. The model predicted the number of amblyopia cases detected for each screen and for the entire screening programme, and the effect of omitting screens. Incidence curves for all types of amblyopia caused by strabismus, refractive anomalies or by both were estimated by approximation of the observational data, in conjunction with experts’ estimations and the literature.

Results

We calculated mean actual sensitivity per screen per type of amblyopia, and the effect per screen. Screening at 24 months was found to be least effective. The impact on the screening programme, estimated by summing the effectiveness per screen, omitting the 24-month screen, was a reduction of 3.4% (57 vs. 59 cases) in the number of detected cases of amblyopia at age 5.

Conclusion

The effectiveness of the Dutch vision screening programme would hardly be affected by omission of the 24-month screening examination. A disinvestment study is warranted.

Introduction

In most vision screening programmes, visual acuity (VA) is measured at age 3–6 years. In the Netherlands, VA measurement is preceded by primarily orthoptic examination to detect visual disorders on four occasions between age 0 and 24 months. At 0–4 months, screening consists of inspection of the eye, pupillary reflexes, and examination of the red fundus reflex, to assess media opacities. At the three screening examinations between 6 and 24 months, tests also include corneal reflexes of a light held by the examiner (Hirschberg test), cover test, alternating cover test, eye motility, and quality of ocular pursuit movement (online appendix table).^1–3 VA is measured at 36 months with the Amsterdam Picture Chart, and at 45 months with the Landolt-C chart. ³

The four main types of amblyopia are refractive amblyopia due to anisometropia (unequal strength of glasses), strabismic amblyopia due to misalignment of the eyes, combined-mechanism amblyopia (a combination of misalignment and unequal strength of glasses) and, rarely, deprivation amblyopia due, for example to congenital cataract or retinoblastoma. Strabismic amblyopia can be noticed by the parents, but refractive amblyopia is not externally apparent, and is often not found until VA is measured.

In the Rotterdam Amblyopia Screening Effectiveness Study (RAMSES)^4,5 4624 children born in Rotterdam between September 1996 and May 1997 were followed for 7 years, through all routine vision screening examinations (0–4, 6–9, 14, 24, 36, 45, and 54–60 months) and diagnostic follow-up. Of the 3897 children still living in Rotterdam at age 7, 2964 underwent the final orthoptic study examination. Amblyopia had been diagnosed in 100 children (3.4%). Screening between 6 and 24 months of age contributed little to the detection of refractive amblyopia.^4,5

If detailed data are available, the impact of omission or introduction of a single screen can be calculated, but this is difficult if that screen is part of a programme consisting of consecutive screens. To calculate the impact of omission of parts of the early vision screening, we formulated a model, inputting these data.

Methods

We developed a micro-simulation model, programmed in MATLAB R2008b (MathWorks), to assess modification of the Dutch eye screening programme. Monte Carlo simulations with various incidence and sensitivity combinations were used to approximate the observational data. The model simulates the path followed by a subject during consecutive screens within the Dutch amblyopia screening programme, which consists of seven exams between age 0 and 60 months. VA is measured from 36 months onwards. A child can attend or not attend a screen test. If the child attends screening, the result can be positive or negative. Positive cases can comply with the referral for diagnostic follow-up. Screen positive cases who have the disease are the true positives (tp). The number of tp and the number of children with the disease not screen-detected at that time (i.e. false negatives (fn)), determine the sensitivity (tp/(tp + fn) of the test (Figure 1). For the approximation of the RAMSES data, input parameters were varied until the model predicted the detection of the cases of amblyopia actually detected by screening (i.e. not by the parents or others) at the corresponding screen for each of the four types of amblyopia. Thereafter a programme with one of the screens omitted was simulated to determine the relative effect on the detection of amblyopia.

Figure 1.

Schematic representation of the path that an individual undergoing screening follows. Indicated are the data obtained from observational studies, the disease incidence and the sensitivity per screen within the path, which are used in the model. Here, n is the number of diseased population and n_non is the number of the healthy population. The fn, fp, tn and tp are the false negatives, false positives, true negatives and true positives, respectively. Here, a and b are the fractions of true positives and false positives, respectively, that comply with the referral.

A major problem in developing this model is that the true incidence curves of the disease are unknown. It is known that amblyopia does not exist at birth, but may develop after age 3 months in cases where one eye is used less for vision. Generally, amblyopia does not develop after age 6. Refractive amblyopia may develop at an older age than strabismic amblyopia and can be treated at an older age.^6–8 The incidence curves for refractive, strabismic, combined-mechanism, and deprivation amblyopia were derived by approximation of the observational data in conjunction with experts’ estimations. Data collected during the RAMSES study, in which the prevalence of amblyopia between ages 0 and 7 years was 3.4%, ⁴ were used.

In the RAMSES study, 100 amblyopia cases were found, 73 with a positive screening and 27 with no positive screening. Of the 73 cases with a positive screening, 60 had been detected by screening, and 12 were detected by the parents but were positively screened later on. In one case, it was unclear whether the child visited the ophthalmology department after the positive test. Four children had been unsuccessfully referred. All 100 amblyopia cases had been differentiated by amblyopia type, and a further differentiation in age of detection for each of the four types was made. We used the combined data about the cause of amblyopia with the screen at which each case of amblyopia was detected (Table 1).

Table 1.

Number of tests and detected amblyopia cases in the RAMSES study differentiated to the four main causes of amblyopia and the screen at which each case was detected.^4,5

Age of positive screening	Tests	Strabismus	Refractive	Combined- mechanism	Deprivation	Unknown type	Total amblyopia
6–9 Months	3272	3	1	4	0	0	8
14 Months	3297	1	0	2	2	0	5
24 Months	2982	1	0	7	1	0	9
36 Months	2659	5	4	4	0	1	14
45–54 Months	2457	3	15	5	1	1	25
60 Months	2824	0	8	3	1	0	12
No positive screen		6	14	5	2	0	27
Sum		19	42	30	7	2	100

Note: The screening exams before 6 months of age were not included in the RAMSES study. No positive screen means that the child was detected outside of screening, for example because parents had noticed abnormalities and had gone to their General Practitioner or to the hospital.

Several orthoptists and strabismologists were asked as experts (i) to report the youngest age at which patients were diagnosed with strabismic, refractive, combined-mechanism, and deprivation amblyopia, and (ii) to estimate the highest age at which each of the four types of amblyopia could develop. For instance they were asked whether they had ever seen a patient with refractive amblyopia diagnosed at age 5 who had good VA at age 3.

By approximation of the observational RAMSES data, in conjunction with experts’ estimations and literature,^6–8 incidence curves for the four types of amblyopia were estimated, with upper and lower estimated boundaries. From the literature and expert opinion we assumed that the sensitive period for amblyopia to develop starts at 3 months (t_start) and that there is no additional amblyopia development after the age of 5 years (t_end).^6–8 The upper limit follows a negative polynomial function (equation (1)). For the lower estimate, the progression is more conservative and stagnates in time, described by a parabolic function (equation (2)).

$I(t)=\alpha +\frac{1}{\beta ·t}$

(1)

$I(t)=a·t^2+b·t+c$

(2)

where I(t) is the incidence at time t and a, b, c, α and β are variables indicated in the input function

$a=\frac{p}{-(t_{\mathrm{end}}{)}^2+(-(t_{\mathrm{start}}{)}^2+2·t_{\mathrm{start}}·t_{\mathrm{end}})}b=-2·t_{\mathrm{end}}·ac=a(-(t_{\mathrm{start}}{)}^2+2·t_{\mathrm{start}}·t_{\mathrm{end}})\alpha =\frac{p}{1-t_{\mathrm{start}}/t_{\mathrm{end}}}\beta =-\frac{1}{t_{\mathrm{start}}·\alpha }$

where t_start is the age for which the first diseased cases start to develop, t_end is the age for which the incidence rate reaches zero or is equal to zero, and p is the prevalence at the end of amblyopia development, that is when incidence rate is zero.

We simulated eight incidence curves with lower and upper estimates. A best-fit approximation for the incidence curve was performed during the fitting of the sensitivity, using the discrete quasi maximum likelihood method, comparing the observed values in the study with the predicted values (online appendix figure). In this way, the most likely incidence curve was estimated.

To calculate the mean sensitivity, we assumed that all screens using the same combination of tests (online appendix table) had the same sensitivity. The model was run with 20 different values of sensitivity between 0% and 100%. We first estimated the mean sensitivity of the 6–24 months orthoptic screens. For each incidence curve, screening was simulated using the 20 values of sensitivity for the preverbal screens. The available subjects for that specific screen examination were obtained from the incidence curve. For each screen the screening test was simulated per subject, subsequently using one sensitivity out of the range. This resulted in a number of positively screened subjects per screen. For each incidence-sensitivity combination this was repeated 4000 times. Each time the simulated number of detected subjects for the preverbal screens was equal to the number of detected subjects in the observational study, the sensitivity used was implemented in an array. After the simulation, a mean sensitivity for that incidence curve was calculated, based on the sensitivity values in the array. The mean sensitivity for screens using the VA test was obtained in a similar way.

Overall sensitivity of the Dutch vision screening programme had been calculated as 73% in the RAMSES study (73 of the 100 detected amblyopia cases had been positively screened). ⁵ For this calculation, sensitivity had been defined as the proportion of children with amblyopia who had a positive vision screening result at any point in time. Because 12 of the 73 amblyopia cases with a positive screening had been detected by the parents, but were positively screened later on, and in one case it was unclear whether this child visited the ophthalmology department after the positive test, in the current evaluation only the 60 screen-detected cases with amblyopia were used.

Using the incidence curves and the mean sensitivity per screen, we simulated the entire screening programme. With the estimated effect per detected case, we estimated the effect per screen. The effectiveness of the programme was calculated by summing the effect per screen. Starting with the scenario of the current vision screening programme, we subsequently calculated what the effectiveness of the screening programme would be after omission of the screen that was found to be least effective in the original simulation.

Results

The estimation of the incidence curves per amblyopia type resulted in four graphs each with an upper and lower estimate, based on approximation of the observational data in conjunction with the experts’ estimations and literature.^6–8 For upper and lower estimates of incidence curves, we consulted several orthoptists, who are involved in the treatment of children with amblyopia, and strabismologists. All agreed that amblyopia does not develop before the age of 3 months, or after the age of 5 years. Elston and Timms have previously shown that the lower age limit before the start of the sensitive period for development of amblyopia is at least 4–6 weeks after birth. ⁶ Basic neurophysiologic research on amblyopia performed by Hubel and Wiesel ⁷ showed that there is an upper age limit for the development of amblyopia in animals. Keech and Kutschke ⁸ found that no human subject developed amblyopia after age 6 years. The model simulated 10 incidence curves per amblyopia type which, together with the cumulative number of cases found per screen, produced the plots shown in Figure 2. The incidences of combined-mechanism amblyopia and refractive amblyopia were highest (between 30 and 40 cases per 2964 children), the incidence of pure strabismic amblyopia was lower (less than 20 cases per 2964 children) and the incidence of deprivation amblyopia was lowest (less than 20 cases per 2964 children).

Figure 2.

The estimated cumulative incidence per amblyopia type for a cohort of 2964 children. Dots represent the detected amblyopia cases in the RAMSES study by screening only (cumulative). The upper black lines represent the upper estimate and the lower grey line represents the lower estimate of the incidence curves.

The sensitivity of the total programme is different from the sensitivity per screen. If all the separate screens are combined into one programme, this will give a higher overall sensitivity. The sensitivity curves presented here are estimated per screen per amblyopia type (Figure 3). For each amblyopia type, the mean actual sensitivity per screen was calculated. The sensitivity of the preverbal screens was less than 15% per screen. The sensitivity of the VA screens was 17–26% per screen. Using the incidence curves and the mean sensitivity per screen, we estimated the effect per screen.

Figure 3.

The estimated test sensitivity per screen per amblyopia type for the six screening examinations.

The Monte Carlo simulation that best approached the observational data was used to estimate the effect of each screen. Using the estimates for mean incidence and mean sensitivity, we determined the cumulative detection over the subsequent screens per amblyopia type. From 2964 children, 59 amblyopia cases were detected by screening by the age of 5 years (Figure 4). For a screening programme with omission of the 24-month screen, more children were detected at the screens at age 3 years and later. The number of detected cases of amblyopia at age 5 would be reduced by 3.4% (57 vs. 59 cases) (Figure 4).

Figure 4.

(a) The predicted number of children detected per screen for amblyopia in the simulated RAMSES study. The squares represent the simulated present screening programme and the dots represent the effectiveness of the simulation of a reduced programme in which the screen at the age of 24 months is omitted. (b) The cumulative predicted number of children detected with amblyopia per screen for amblyopia in the simulated RAMSES study. The lines represent the present screening programme in the Netherlands (squares) and a reduced programme in which the screen at the age of 24 months is omitted (dots).

Discussion

This study shows that the effect of omission of components of a screening programme can be calculated with the micro-simulation model, provided that sufficient and detailed data are available. It confirms the suggestion, from a large prospective birth-cohort observation study, that part of the screening programme seemed to add little to the detection of amblyopia. Analysis of the data had shown that screening at 6–24 months, an age when VA cannot be measured yet, contributed little to the detection of refractive amblyopia, whereas strabismic amblyopia was detected outside of screening in half of cases. ⁴ With the micro-simulation model, the screening examination at 24 months was found to be least effective. Omitting this screen reduced the total number of detected cases of amblyopia at age 5 years from 59 to 57 (3.4%).

Our model has limitations. It is not incontrovertible to adapt some of the less well determined input parameters, like the incidence curves, to fit the detailed data from the RAMSES study, but our pragmatic approach served as a good starting point of the simulation of omission of part of the programme. For micro-simulations, accurate data on the prevalence and incidence of the disease at the time of screening and on the sensitivity of the screening methods are essential, but these data are difficult to obtain from observational studies. The age-specific incidence also varies per amblyopia type. But the curves for the four types are essential for the simulation of the effectiveness of each screening examination. We had to make assumptions to estimate these incidence curves, and for the sensitivity of the tests. To calibrate the upper and lower limits of the amblyopia incidence curves, we therefore had to use indirect derivatives such as the RAMSES data, literature, and expert opinion. If the age specific incidence of amblyopia were in fact lower than estimated, the incidence curves would be closer to the curves of detected cases in the RAMSES study, and the sensitivity of the screen would be higher. If the incidence curves were higher than estimated, the sensitivity would be lower than estimated. If children were to be tested at a later age, when they would probably be easier to test, the sensitivity of the tests would increase, leading to a higher amblyopia case detection.

It is difficult to compare our incidence curves with data in current literature. Atkinson et al. identified manifest strabismus and strabismogenic and amblyogenic refractive errors in children aged 7–9 months and found a hyperopia (≥3.5D) prevalence of 5–6% and an anisometropia and manifest strabismus prevalence of < 1% each. Untreated hyperopes developed strabismus in 21%. The children were not classified as amblyopic or not, but our incidence curves also showed an increase in strabismic amblyopia after age 6 months and a higher incidence of refractive amblyopia in comparison with strabismic amblyopia. ⁹ Williams et al. ¹⁰ reported a prevalence of 0.7–3.4%, depending on whether the children had undergone preschool screening and the definition of amblyopia.

We assumed equal sensitivities for all preverbal exams at 6–9, 14, and 24 months, as these screening examinations comprise the same tests. In the RAMSES study, however, as in real practice, these multi-component screening examinations were considered as one screen. It is possible that testing at an older age would have had a higher sensitivity, because of better cooperation and understanding of the tests. Also, if all separate screens are combined into one programme, this will give a higher sensitivity.

The low number of refractive amblyopia cases detected might be explained by the fact that strabismus causing amblyopia is often noticed by the parents, but a refractive error is not.

Complete and detailed observational data are useful to analyse the effectiveness of population-based screening programmes, but, because observational data is not dynamic, it is difficult to predict the effect of changes in a screening programme. Analysis is complex as the net effect of omission of one link in the chain is difficult to predict. Evaluation of alternative different screening strategies is difficult, as such studies are often expensive and time consuming. A micro-simulation model is dynamic, and can be used to evaluate different screening scenarios and compare their effectiveness. As our model was able to compute the mean number of detected true cases for each consecutive screen, it is well suited to evaluate a screening programme consisting of repeated screens. Further work is needed to validate and extend the model, so that it can be used to simulate screening programmes using different tests for each screen. With more input data, the model might be used as a general tool for evaluating the effectiveness of screening programmes with multiple screenings. As the input can easily be altered per disease specification, it could simulate screening for different disease types.

Further work is needed before the model is able to provide evidence for modifications of screening programmes, but it can be used as a tool to compare the effectiveness of different screening programmes, for instance in different countries in Europe, ¹¹ provided that detailed input data such as sensitivity of the tests used and background prevalence, are available. Since the completion of this microsimulation model study, we have started a disinvestment study, comparing two sequential birth cohorts of approximately 6000 children in a care region in the Netherlands. In the second birth cohort, specific eye examination during general screening between the age of 6 and 24 months was omitted. The first results of this disinvestment study confirm the predictions of the microsimulation model for the screens at age 6–9 months: the screened and unscreened groups differed little in the proportion of children referred and found to have amblyopia. On further examination, all cases of amblyopia detected were caused by strabismus, not by refractive errors, and most cases of strabismic amblyopia were found because the parents had noticed the strabismus. ¹² At age 14–24 months, referral was still mostly based on conspicuous strabismus or other visually apparent disorders noted by parents or screening physicians, but, although still a minority, specific eye screening led to more referrals than at 6–9 months. ¹³