Interlaboratory comparison of measurement capabilities for flicker and stroboscopic effects for Ecodesign conformity assessment

Abstract

Lamps based on LEDs often introduce temporal light modulation (TLM). TLM consists of fluctuations in the emitted light over time and can lead to the perception of temporal light artefacts such as flicker and the stroboscopic effect. Due to the widespread concern about the negative effects of TLM on health and safety, additional regulations (Ecodesign) have been introduced. Since these regulations entered into force, it is important for laboratories to underpin their TLM measurements with SI traceability, and to intercompare their result for validation, because their measurements determine whether an LED lamp is allowed on the European market. Therefore, an interlaboratory study of LED lamps was conducted to validate laboratories’ capabilities and methods for TLM measurements of light sources. Overall, the developed models and evaluated uncertainties perform consistently, especially for optical TLM measurements of well-controlled sources with built-in power supply.

1. Introduction

Since the introduction of LED lamps, incandescent and fluorescent light sources are being replaced by these more energy-efficient LED lamps. While LED lamps have many advantages such as the mentioned energy efficiency and prolonged lifetime, LED lamps also introduce temporal light modulation (TLM). Both pulse-width modulation drivers and suboptimal AC–DC converters are usually the root cause of the TLM.¹ TLM consists of fluctuations in the emitted light over time and can lead to the perception of the temporal light artefacts (TLA) such as flicker, stroboscopic effect and the phantom array effect.^2–5

Flicker is the perceived variation in light intensity because of fluctuations in luminance or spectral distribution observed by a static observer in a static environment.⁵ In the case where the observer is in a non-static environment, for example, there is a change in motion perception, the stroboscopic effect can be perceived.^5,6

Flicker and the stroboscopic effect can introduce negative effects on health and well-being.^7,8 These effects can be divided in neurobiological effects, such as eye strain, fatigue, headaches and migraines,^9–11 and performance or cognitive effects, such as decreased mood and comfort.^12–14 Additionally, TLM affects brain activity, with a higher brain activity being measured when the flicker or stroboscopic effect are observed.¹⁵

Due to the widespread concern on the negative effects of TLM on health and safety, the European Commission laid down Ecodesign requirements.^16–18 These requirements entered into force in September 2021 and raised interest in reliable, accurate and reproducible methods to measure TLM. According to these regulations, short-term flicker $P_{st}^{LM}$ , measured using the light flicker meter, shall not be higher than 1 at full load, in which case the average observer has a 50 % probability of detecting flicker. The stroboscopic visibility measure (SVM), expressed as $M_{VS}$ , is limited to less than 0.4 at full load as of 2024. An $M_{VS}$ of 1 represents a 50 % probability of observing the stroboscopic effect for an average observer. Additionally, the Ecodesign requirements state that for conformity assessment against these limits, a tolerance interval of 10 % is allowed. This implies that the uncertainty of TLM measurements should be better than 10 % at the given limits to ensure justified conformity assessment.^16–18

Since these EU regulations entered into force, it is important for laboratories to underpin their TLM measurements with SI traceability and to intercompare their results. These measurements play a decisive role whether an LED lamp and its associated separate control gears can enter the European market. Over time, guidance on how to measure and calculate TLM has been described in various technical reports.^3–5,19 To validate the laboratories’ capabilities and methods for TLM measurements of light sources, an interlaboratory comparison of LED lamps was conducted among eight laboratories. These laboratories included national metrology institutes, lamp manufacturers, instrument manufacturers and universities.

In this study, we compared measurements of flicker and SVM of commercially available lamps as well as of a programmable TLA reference radiator designed for verification. Most commercially available lamps used in the study have flicker and SVM values below the regulation limits. We show that most of the participants would reach consensus on classification of the LED lamps in conformance with the regulations, but there is still room for discussion.

2. Method

2.1 Structure of interlaboratory comparison

The interlaboratory comparison comprised of eight participants: Aalto University, DTU Electro, Gigahertz Optik, ICCS, LMT, RISE, Signify Netherlands B.V. and VSL. Of these participants, VSL acted as pilot and performed the data analysis of the comparison results. RISE acted as copilot and carried out lamp ageing, pre- and post-comparison measurements, and distributed the light sources to the participants. Figure 1 shows an anonymised schematic of the structure of the comparison, where the copilot’s measurements are used to link the results of all participants. Each participant measured the light sources according to their normal measurement procedure for flicker expressed in $P_{st}^{LM}$ and SVM expressed in $M_{VS}$ , following additional guidance given in the protocol and in the IEA 4E Solid State Lighting Annex interlaboratory comparison technical protocol.²⁰ The protocols use the CIE 2016 for weighting the stroboscopic visibility.⁵ The copilot measured the light sources before and after the comparison to monitor the stability during the comparison. As the monitoring measurements were performed with a non-optimised SVM algorithm and with a low sampling frequency for $P_{st}^{LM}$ , the copilot performed additional measurements on one of the sets of light sources using their normal measurement set-up, to be used as their participation result in the comparison. Then, all measurement data were collected, and the data analysis was performed by the pilot.

Figure 1

The structure of the comparison, where four sets of light sources are sent around, each measured by two of the participants. The copilot acts as a linking laboratory and measures the light sources before and after the participating laboratories. In addition, one of the sets was measured by the copilot, acting as participant, this time using a slightly different measurement set-up

2.2 Light sources

During the interlaboratory comparison, several light sources were measured: four waveforms of the Signify TLA reference radiator, two lamp types selected by the MetTLM project and five lamps supplied by the IEA 4E Solid State Lighting Annex.^20–22 While in possession of participants, the light sources were stored between 15 °C and 35 °C and at a relative humidity below 75 %. To enable parallel measurements by participants, four sets of nominally identical light sources have been used.

The Signify TLA reference radiator is a programmable LED source with preinstalled waveforms to emulate flicker and stroboscopic effect, having various visibility and perception properties.^23,24 Additionally, the temporally modulated output of the TLA reference radiator is independent of the mains impedance because of the programmable driver for the current. The TLA reference radiator needs 15 min of stabilisation time for initial warm-up and 5 min after changing waveform. The waveforms measured are waveforms 9, 14, 24 and 26. The shape of these waveforms is depicted in Figure 2. Occasionally, (about once every 30 min) the TLA reference radiator can switch off for a second, resulting in unintended temporal modulation. To mitigate this issue, participants performed repeated measurements.

Figure 2

The waveforms used in the comparison of (a) the lamps selected from the MetTLM project, (b) the lamps from the IEA 4E Annex interlaboratory comparison, (c, d) the TLA reference radiator

The lamps selected by the MetTLM project, lamps 1 and 2, needed a stabilisation time of 30 min and were measured base-up. These lamps were a combination of omnidirectional lamps with an E27 screw base and spot lights with an E14 screw base. The lamps supplied by the IEA 4E Solid State Lighting Annex, lamps 3 to 7, used a stabilisation time of at least 15 min, and the lamps were also measured base-up. All IEA 4E Solid State Lighting Annex interlaboratory comparison lamps were omnidirectional and had an E27 screw base.

2.3 Stability of the light sources

All light sources were seasoned and thereafter assessed for stability by the copilot following the stabilisation test protocol. To season the lamps, they were placed base-up in racks and powered for 24 h with a stabilised power supply. The racks were shielded from ambient light and kept at an ambient temperature of (25 ± 5) °C. The electrical test conditions used a supply voltage of 230 V AC (±0.5 %), a frequency of 50 Hz, a total harmonic distortion below 1.5 % and a power meter with a bandwidth larger than 100 kHz meeting CIE S 025 requirements.²⁵ After seasoning, the lamps were then moved to another facility where the photometric and electrical quantities were then measured every 5 min for the total duration of the stability test of 63 min, with measurements of the $M_{VS}$ (5 s), $P_{st}^{LM}$ (180 s), lamp current, active power and luminous intensity. The stability of each light source was evaluated by checking the above-mentioned parameters over time. Additionally, rolling 15 min variances of the values were compared to a threshold value of 0.5 % or 0.005 (if the absolute values were very low) to determine when sufficiently stable operation had been achieved. Figure 3 shows two examples of measured 15 min variance for lamp type 2, light source set 1 (Figure 3(a)) and lamp type 7, light source set 1 (Figure 3(b)). All lamps fulfilled the set threshold criteria although some of the older LED lamps with large heat sinks took almost one hour to fully stabilise. However, the TLM-parameters were stable within about 30 min for all light sources.

Figure 3

Two examples of measured 15-min variance (relative variation width) for the comparison light sources (a) lamp type 2, light source set 1 and (b) lamp type 7, light source set 1

2.4 Measurement methods

The participants measured the TLM properties for each of the light sources. However, $P_{st}^{LM}$ and $M_{VS}$ are not the only information reported for each light source. Additionally, each participant reported the power supply type, measurement mode, date, stabilisation time, operating position, sampling rate, current, power, voltage, temperature, humidity and corresponding measurement uncertainties. Measurement uncertainty budgets were prepared using the ‘Guide to the expression of uncertainty in measurement’.^26,27

To measure the $P_{st}^{LM}$ and $M_{VS}$ , the measurement set-up of the participants typically included a power supply, a photometer, an amplifier, filtering, digitisation and post-processing of the data.^24,28–34 Figure 4 shows a schematic of such a typical set-up. From temporal measurements of the illuminance, the $P_{st}^{LM}$ can be calculated using a set of Laplacian filters in the time domain.^34,35 These filters represent the human eye–brain response to flicker. Using the stroboscopic visibility threshold and a Fourier transform from temporal measurements of the illuminance, the $M_{VS}$ is calculated.^34,35

Figure 4

Typical measurement set-up to measure TLM

2.5 Data analysis

The calculation method used in analysis of the comparison results for flicker is outlined below; a similar approach was applied to the results for the stroboscopic effect. The results were processed by the pilot laboratory using methods from ‘Guidelines for CCPR Key Comparison Report Preparation, CCPR-G2 Rev.4, January 8, 2019’.³⁶ The copilot laboratory performed measurements of the light sources before sending them to the participants and after receiving the light sources back. In the analysis, each lamp type, as well as each waveform of the TLA reference radiator, is mathematically treated as an independent light source. Flicker measurements by the copilot are denoted by $P_{st, 0, j}^{LM}$ for each lamp or waveform j. The flicker measurement by the participants is denoted by $P_{st, i, j}^{LM}$ for each participant i, including measurement by the copilot, RISE, in the role of participant. To assess the stability of the light sources during the comparison, the ratio $R_{i, j}$ between the participants and the copilot is calculated using Equation (1)

R_{i, j} = \frac{P_{st, i, j}^{LM}}{P_{st, 0, j}^{LM}} .

(1)

This ratio is then normalised by dividing the ratio by the average of the ratio of the before and after measurements of the copilot. The normalised ratio ${\bar{R}}_{i, j}$ is calculated using Equation (2)

{\bar{R}}_{i, j} = \frac{R_{i, j}}{\frac{1}{2} \sum_{r = 1}^{2} R_{i, j, r}},

(2)

where $r = 1$ indicates the before measurement and $r = 2$ indicates the after measurement. This facilitates assessment of light source stability without revealing participants’ biases.

The uncertainty $u_{R_{i, j}} (P_{st, i, j}^{LM})$ of the ratio depends on the absolute expanded uncertainty of the measurement performed by the participant $u (P_{st, i, j}^{LM})$ , the absolute expanded uncertainty of the before or after measurement of the copilot $u (P_{st, 0, j}^{LM})$ and the absolute expanded uncertainty in the reproducibility by the copilot $u_{repro} (P_{st, 0, j}^{LM})$ , including the stability of the comparison scale and repeatability of the comparison light sources during the comparison period. In case $u_{repro} (P_{st, 0, j}^{LM})$ was underestimated by the copilot, it was calculated using Equation (3)

u_{repro} (P_{st, 0, j}^{LM}) = \frac{2 \cdot Δ P_{st, 0, j}^{LM}}{\sqrt{3}}

(3)

where $Δ P_{st, 0, j}^{LM}$ is the biggest difference between the stability measurements of the light source. Underestimation, and thus calculation of $u_{repro} (P_{st, 0, j}^{LM})$ according to this formula, was the case for (i) waveform 14, (ii) lamp 1 and (iii) lamp 3.

u_{R_{i, j}} (P_{st, i, j}^{LM}) = \sqrt{{(\frac{u (P_{st, i, j}^{LM})}{P_{st, 0, j}^{LM}})}^{2} + {(- \frac{u (P_{st, 0, j}^{LM}) \cdot P_{st, i, j}^{LM}}{{P_{st, 0, j}^{LM}}^{2}})}^{2} + {(- \frac{u_{repro} (P_{st, 0, j}^{LM}) \cdot P_{st, i, j}^{LM}}{{P_{st, 0, j}^{LM}}^{2}})}^{2}}

(4)

u_{{\bar{R}}_{i, j, before}} (P_{st, 0, j, before}^{LM}) = \sqrt{{(\frac{- 2 u (P_{st, 0, j, before}^{LM}) \cdot P_{st, 0, j, after}^{LM}}{{(P_{st, 0, j, before}^{LM} + P_{st, 0, j, after}^{LM})}^{2}})}^{2} + {(\frac{- 2 u_{repro} (P_{st, 0, j}^{LM}) \cdot P_{st, 0, j, after}^{LM}}{{(P_{st, 0, j, before}^{LM} + P_{st, 0, j, after}^{LM})}^{2}})}^{2}}

(5)

Δ_{i, j} = \frac{P_{st, i, j}^{LM}}{{\bar{P}}_{st, 0, j}^{LM}} - 1

(6)

{\bar{P}}_{st, 0, j}^{LM} = \frac{1}{2} \sum_{r = 1}^{2} P_{st, 0, j, r}^{LM}

(7)

u ({\bar{P}}_{st, 0, j}^{LM}) = \frac{1}{2} \sum_{r = 1}^{2} u (P_{st, 0, j, r}^{LM})

(8)

u (Δ_{i, j}) = \sqrt{{(\frac{u (P_{st, i, j}^{LM})}{{\bar{P}}_{st, 0, j}^{LM}})}^{2} + {(- \frac{P_{st, i, j}^{LM} \cdot u ({\bar{P}}_{st, 0, j}^{LM})}{{\bar{P}}_{st, 0, j}^{LM}^{2}})}^{2} + {(- \frac{P_{st, i, j}^{LM} \cdot u_{repro} ({\bar{P}}_{st, 0, j}^{LM})}{{\bar{P}}_{st, 0, j}^{LM}^{2}})}^{2}}

(9)

The combined uncertainty of the ratio is determined by taking the partial derivatives and can be found in Equation (4). Additionally, the uncertainty corresponding to the normalised ratio is obtained by taking partial derivatives of the absolute expanded uncertainties of the before or after measurements of the copilot and is shown in Equation (5).

After determining the performance of the copilot, the measurements of the participants are evaluated. First, the relative difference $Δ_{i, j}$ is determined in Equation (6), where ${\bar{P}}_{st, 0, j}^{LM}$ is the average over the before and after measurement of the copilot given in Equation (7), with the associated total measurement uncertainty given in Equation (8).

Additionally, the uncertainty of the relative difference can be calculated by taking partial derivatives shown in Equation (9).

Next, the average relative difference per waveform $j$ is given in Equation (10)

{\bar{Δ}}_{j} = \frac{1}{N_{part}} \sum_{i = 1}^{N_{part}} Δ_{i, j}

(10)

where $N_{part}$ is the number of participants that measured each waveform. The uncertainty associated with the average relative difference is given in Equation (11)

u ({\bar{Δ}}_{j}) = \frac{1}{N_{part}} \sum_{i = 1}^{N_{part}} u (Δ_{i, j}) .

(11)

The relative difference from the mean value for each participant $i$ for each waveform $j$ , termed the degrees of equivalence, is then calculated in Equation (12)

D_{i, j} = Δ_{i, j} - {\bar{Δ}}_{j} .

(12)

Using this equation, $D_{i, j}$ averaged over all participants becomes zero. Additionally, the uncertainty of the relative difference from the mean for each participant $i$ can be calculated in Equation (13)

u (D_{i, j}) = \sqrt{u^{2} (Δ_{i, j}) + u^{2} ({\bar{Δ}}_{j})} .

(13)

The comparison reference value (RV) of the flicker measurement $P_{st, i, j, RV}^{LM}$ is then determined in Equation (14)

P_{st, i, j, RV}^{LM} = - D_{i, j} \cdot P_{st, 0, j}^{LM} + P_{st, i, j}^{LM}

(14)

and the corresponding uncertainty of the RV is given in Equation (15)

u (P_{st, i, j, RV}^{LM}) = \sqrt{{(- u (D_{i, j}) \cdot {\bar{P}}_{st, 0, j}^{LM})}^{2} + {(- u ({\bar{P}}_{st, 0, j}^{LM}) \cdot D_{i, j})}^{2} + u {(P_{st, i, j}^{LM})}^{2}}

(15)

It should be noted that the comparison RV of each of the lamps or waveforms is the same for all participants who measured that lamp or waveform.

2.6 Outlier removal

Outliers were removed according to ‘Guidelines for CCPR Key Comparison Report Preparation, CCPR-G2 Rev.4, January 8, 2019’. In the case where the difference between the measurement and the RV is larger than three times its associated expanded uncertainty, the point is removed, and a new RV is calculated.³⁶ The biggest outlier is removed first; if outliers still remain after the RV is recalculated, then the next biggest outlier is removed, and so on until remaining outliers are the result of underestimation of the measurement uncertainty. This underestimation was assumed when (i) the measurement had a significantly lower uncertainty compared to other measurements, and (ii) the measurement would not be an outlier if it had an uncertainty comparable to the other measurements.

3. Results

3.1 Flicker results

3.1.1 Light source stability and consistency

Figure 5 shows the normalised ratios, used to assess stability and consistency of the light sources, for flicker measurements of two of the light sources. Figure 5(a) shows the normalised ratio of waveform 9 of the TLA reference radiator. The before and after measurements are nicely overlapping. This normalised ratio is representative for all the waveforms of the TLA reference radiator and for the other light sources, with the exception of the lamp type shown in Figure 5(b).

Figure 5

The normalised ratio of the flicker ( $P_{st}^{LM}$ ) measurements for (a) waveform 9 of the TLA reference radiator and (b) lamp type 1. Waveform 9 shows better overlap than lamp type 1. The other lamp types and waveforms showed an overlap similar to waveform 9, making lamp type 1 the exception

Figure 5(b) shows the normalised ratios of lamp type 1. These lamps are less stable than the TLA reference radiator waveform 9, with lamp type 1 from light source set 2 only having barely overlapping uncertainties from the before and after measurements, of which one does not even pass the normalised ratio value of 1.0. Lamp type 1 was the exception from all the other lamp types and an example of an unstable light source, which was specifically chosen by the consortium due to the high amount of flicker.

3.1.2 Degrees of equivalence

Figure 6 shows the degrees of equivalence as well as the flicker measurements and their RV of four types of light sources from those measured during the comparison. Figure 6(a) shows the degrees of equivalence of TLA reference radiator waveform 9. The figure shows that participants 1, 2, 3, 4 and 6 are non-equivalent. Figure 6(b) gives the results, RV and uncertainties for each participant for this waveform and shows that the value of participant 3 is significantly higher than the RV. Additionally, for participants 5, 7 and 8 the uncertainties are nicely enveloping both the measurement value and the RV, while for participants 1, 2 and 4 only the uncertainties are overlapping. Figure 6(c) shows that for lamp type 1 the participants have reached equivalence. Figure 6(d) shows that the uncertainty of the RV of lamp type 1 overlaps with the measurements of all participants, but the uncertainty of the measurement only envelops the RV for participant 5. Figure 6(e) shows that for lamp type 3, as for waveform 9, there also is no equivalence. This is confirmed by Figure 6(f), where there is a large variation in measurement values. Only for participant 5 is the measurement overlapped by the uncertainty of the RV. Additionally, Figure 6(f) shows that the result of participant 3 is off from the RV and is thus pushing the other laboratories out of equivalence. Figure 6(g) shows that for lamp type 6 none of the participants are equivalent. This is confirmed by Figure 6(h), where only participant 5 has their measurement overlapped by the uncertainty of the RV. Participant 3 even measures a value above 1. However, since the measured value still falls within 10 % of the limit, the lamp would still be allowed on the EU market. Furthermore, Figure 6(h) shows that the results of participant 3 is off from the RV and thus pushing the other laboratories out of equivalence. The results of all degrees of equivalence as well as the mean RV for all lamps and all participants are shown in Supplemental Information 1.

Figure 6

Degrees of equivalence and their uncertainty of the flicker ( $P_{st}^{LM}$ ) measurements for the light sources (a) waveform 9 of the TLA reference radiator, (c) lamp type 1, (e) lamp type 3 and (g) lamp type 6. The $P_{st}^{LM}$ measurements, their RV and corresponding uncertainties for light sources (b) waveform 9 of the TLA reference radiator, (d) lamp type 1, (f) lamp type 3 and (h) lamp type 6. Results before outlier removal

3.2 Stroboscopic effect visibility measure

3.2.1 Light source stability and consistency

Figure 7 shows the normalised ratio of the stroboscopic effect visibility measure for two of the light sources. Figure 7(a) shows the normalised ratio of $M_{VS}$ for waveform 9 of the TLA reference radiator. The measurements performed by the copilot before and after the interlaboratory comparison overlap nicely, showing good light source stability. Figure 7(b) shows the normalised ratio of $M_{VS}$ for lamp type 3. In this figure, the overlap is less and thus are the lamps less stable. However, even though the uncertainties from the before and after measurements of the copilot of lamp type 3 from light source set 2 barely overlap, the overlapping is an indication of adequate stability.

Figure 7

Normalised ratio of the stroboscopic effect visibility measure ( $M_{VS}$ ) for light sources (a) waveform 9 of the TLA reference radiator and (b) lamp type 3

3.2.2 Degrees of equivalence

Figure 8 shows the degrees of equivalence and the measurement and RV for two of the waveforms of the TLA reference radiator as well as of two of the lamp types. Figure 8(a) shows that participants 3 and 6 are non-equivalent for waveform 9. This is confirmed by Figure 8(b), which shows that the uncertainties for these participants are not overlapping. Additionally, Figure 8(b) shows that the measurement and RV are enveloped by the uncertainties of participants 1, 2, 4, 5 and 7. Figure 8(c) shows that all participants are equivalent for waveform 24. Figure 8(d) shows that for waveform 24 the measurement and RV are enveloped by participants 2, 4, 5, 6, 7 and 8. Figure 8(e) only shows equivalence for participant 8 for lamp type 2. Figure 8(f) shows that participant 7 measured a value significantly higher than the others. Additionally, the $M_{VS}$ is close to the limit of 0.4. While the lamp is fit for the EU market according to participants 1, 2, 3, 4, 5, 6 and 8, participant 7 would reject the lamp since the $M_{VS}$ is more than 10 % above the limit. Figure 8(g) shows that for lamp type 3, participant 7 is non-equivalent. Figure 8(h) confirms the non-equivalence, and the measurement and RV of participants 2, 4, 5 and 6 are enveloped by their uncertainties. The results of all degrees of equivalence as well as the mean RV are shown in Supplemental Information 1.

Figure 8

Degrees of equivalence and their uncertainty of the stroboscopic effect ( $M_{VS}$ ) measurements for the waveforms of the TLA reference radiator: (a) waveform 9, (c) waveform 24, and lamp types (e) 2 and (g) 3. The $M_{VS}$ measurements, their RV and corresponding uncertainties for TLA reference radiator waveforms (b) 9 and (d) 24, and lamp types (f) 2 and (h) 3. Results before outlier removal

3.3 Outlier removal

The measurements of the light sources contained several results which required removal since they were outliers, that is, the difference between the measurement and the RV was larger than three times its associated expanded uncertainty. This resulted in removal of outliers for waveform 9, waveform 24 and waveform 26, and lamps 6 and 7 for the $P_{st}^{LM}$ and in removal of outliers for lamp types 2, 4 and 7 for $M_{VS}$ . Figure 9 shows the degrees of equivalence without the outliers for $P_{st}^{LM}$ . The degrees of equivalence include not only the measurement uncertainty of the participant but also rely on the largest uncertainty of all participants. Specifically, Figure 9(a) shows the results of waveform 9 of the TLA reference radiator, where the participants are now in equivalence. Figure 9(b) shows the results for lamp type 6 after removal of two outliers. For this light source, the results are still not equivalent. Since the difference from the RV of these participants is less than three times the expanded uncertainty, the non-equivalent participant should increase their measurement uncertainty. Figure 10 shows the degrees of equivalence without the outliers for $M_{VS}$ . In Figure 10(a) the results for lamp type 2 are shown, which now show equivalence. Figure 10(b) shows the results for lamp type 4, which do not show equivalence yet. However, the difference between the measurement and RV is less than three times the expanded uncertainty. Therefore, the remaining outliers are the result of underestimation of the measurement uncertainty; the non-equivalent participants should increase their measurement uncertainty.

Figure 9

Degrees of equivalence of (a) waveform 9 of the TLA reference radiator and (b) the lamp type 6 for the flicker measurements after removal of the outliers

Figure 10

Degrees of equivalence of (a) lamp type 2 and (b) lamp type 4 of the stroboscopic effect visibility measure after removal of outliers

4. Discussion

The aim of the interlaboratory comparison was to validate participants’ capabilities for measurements of the flicker and stroboscopic effect produced by several types of light sources. While most of the participants agreed on the admission of certain lamps to the European market, the results show that the agreement is not yet unanimous, with one result falling even outside the 10 % tolerance of the Ecodesign requirements.¹⁶ Therefore, it is important to evaluate the origin of the differences between measurements of the participants.

These differences can be explained because each of the participants used their own measurement set-up with corresponding uncertainty budget for this interlaboratory comparison. Therefore, uncertainties between participants vary greatly. Participants with a lower claimed measurement uncertainty could have missed some factors affecting their measurement in their uncertainty budget determination, resulting in a poor degrees of equivalence and thus having less overlap with the RV or measurements of other participants. Additionally, the equivalence and RV could be affected by outliers in the measurements. While we took outlier removal into account, alternative removal methods could be debated.³⁶ Furthermore, the stability of the lamps could have affected the measurements. Lamp type 1 showed a stability worse than the others. This could have increased the differences among the participants.

In addition, it has been shown that applying zero padding, a method to increase the frequency resolution of a fast Fourier transform to the $M_{VS}$ calculation can make results more consistent, which not all laboratories had implemented at the time of the comparison.³⁰ Another cause for the deviations of the flicker results could be the impedance of the power supplies used to power the light source. In a preliminary test, these power supplies were shown to influence the flicker of certain lamps.³⁷ It is worthwhile to note that over all the results for the TLA reference radiator, which has a built-in power supply, are better in agreement. When looking at the result for each lamp, individual consistent outliers are the cause of an overall negative consistency. Overall, the developed models and evaluated uncertainties perform consistently, especially for optical TLM measurements against well-controlled sources such as the TLA reference radiator.

5. Conclusion

This interlaboratory comparison showed measurements of the flicker and stroboscopic effect by eight different participants. The participants measured the $P_{st}^{LM}$ and $M_{VS}$ for four waveforms of a TLA reference radiator and seven types of LED light sources. The measurements show that even though there are differences in the measured values, overall, the participants are in consensus about whether a light source can enter the European market. We see that measuring close to the threshold is still difficult, though can be measured with reasonable uncertainty in perspective of the Ecodesign requirements.

Furthermore, some participants must re-evaluate their uncertainty budget, since the uncertainty is underestimated for the influence of zero padding in the SVM model for stroboscopic effect visibility measure and differences associated with the use of different power supplies among laboratories. For optical sources such as the TLA reference radiator, that eliminates the effects of different external power supplies, the results are consistent among most participants, underpinning their capability for assessment of light sources against the Ecodesign requirements.

Supplemental Material

sj-docx-1-lrt-10.1177_14771535261417537 – Supplemental material for Interlaboratory comparison of measurement capabilities for flicker and stroboscopic effects for Ecodesign conformity assessment

Supplemental material, sj-docx-1-lrt-10.1177_14771535261417537 for Interlaboratory comparison of measurement capabilities for flicker and stroboscopic effects for Ecodesign conformity assessment by MW Kuiper, S Källberg, A Thorseth, A Klej, V Mantela, EN Madias, T Reiners, R Zuber and PR Dekker in Lighting Research & Technology

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work published in this paper is funded by the European Metrology Programme for Innovation and Research (EMPIR) Project 20NRM01 MetTLM ‘Metrology for temporal light modulation’. This project has received funding from the EMPIR programme co-financed by the Participating States and from the European Union’s Horizon 2020 research and innovation programme.

ORCID iDs

MW Kuiper

S Källberg

A Thorseth

V Mantela

EN Madias

R Zuber

PR Dekker

Supplemental material

Supplemental material for this article is available online.

References

Veitch

Martinsons

Coyne

, et al. Correspondence: On the state of knowledge concerning the effects of temporal light modulation. Lighting Research and Technology 2021; 53: 89–92.

Brown

Foulsham

Lee

, et al. Research note: Visibility of temporal light artefact from flicker at 11 kHz. Lighting Research and Technology 2020; 52: 371–376.

International Electrotechnical Commission. Equipment for General Lighting Purposes – Objective Test Method for Stroboscopic Effects of Lighting Equipment. IEC TR 63158:2018. Geneva, Switzerland: IEC, 2018.

International Electrotechnical Commission. Equipment for General Lighting Purposes – EMC Immunity Requirements – Part 1: An Objective Light Flickermeter and Voltage Fluctuation Immunity Test Method. IEC TR 61547-1:2020. Geneva, Switzerland: IEC, 2020.

Commission Internationale de l’Éclairage (CIE). Visual Aspects of Time-Modulated Lighting Systems – Definitions and Measurement Models. CIE TN 006:2016. Vienna, Austria: CIE, 2016.

Lindén

Dam-Hansen

Flicker Explained: Guide to IEC 61547 for the Lighting Industry. Lund, Sweden: Lund University, 2022.

Wald

Erwin

Annon-Eberharter

, et al. LED light can falsify pulse oximetry readings via the stroboscopic effect. Engineering and Physical Sciences in Medicine 2023; 46: 1667–1675.

Goswami

Chakraborty

Ghosh

, et al. A laboratory based study on the effect of peripheral flickering LED sources on reaction time of drivers for object recognition. Optik 2023; 273: 170428.

Wilkins

Veitch

Lehman

LED lighting flicker and potential health concerns: IEEE standard PAR1789 update. IEEE Energy Conversion Congress and Exposition, Atlanta, GA, USA, 12–16 September 2010.

10.

Wilkins

Nimmo-Smith

Slater

, et al. Fluorescent lighting, headaches and eyestrain. Lighting Research and Technology 1989; 21: 11–18.

11.

Mankowska

Grzywinska

Winklewski

, et al. Neuropsychological and neurophysiological mechanisms behind flickering light stimulus processing. Biology 2022; 11: 1720.

12.

McColl

Veitch

JA.

Full-spectrum fluorescent lighting: a review of its effects on physiology and health. Psychological Medicine 2001; 31: 949–964.

13.

Jaén

Colombo

Kirschbaum

CF.

A simple visual task to assess flicker effects on visual performance. Lighting Research and Technology 2011; 43: 457–471.

14.

Yoshimoto

Jiang

Takeuchi

, et al. Visual discomfort from flicker: effects of mean light level and contrast. Vision Research 2020; 173: 50–60.

15.

Zhao

Hou

Lin

, et al. The effect of stroboscopic effect on human health indicators. Lighting Research and Technology 2020; 52(3): 389–406.

16.

European Commission. Laying down Ecodesign requirements for light sources and separate control gears pursuant to Directive 2009/125/EC of the European Parliament and of the Council and repealing Commission Regulations (EC) No 244/2009, (EC) No 245/2009 and (EU) No 1194/2012. Official Journal of the European Union, Brussels, Belgium, 2019.

17.

Schakel

Banerjee

Bergen

et al. Guidance on the Measurement of Temporal Light Modulation of Light Sources and Lighting Systems. CIE Technical Note 012:2021. Vienna, Austria: CIE, 2021.

18.

Weinold

A long overdue end to flicker: the 2020 EU lighting efficiency regulations. Cambridge Journal of Science and Policy 2020; 1: 1–8.

19.

Commission International de l’Éclairage (CIE). Guidance on the Measurement of Temporal Light Modulation of Light Sources and Lighting Systems. CIE TN012:2021. Vienna, Austria: CIE, 2021.

20.

Coyne

Ohno

Technical Protocol: Solid State Lighting Annex IC 2023: interlaboratory comparison on measurement of temporal light modulation [Online]. Retrieved 08 October 2025 from https://www.iea-4e.org/wp-content/uploads/2023/09/IC-2023-Technical-Protocol-Final.pdf

21.

Ohno

Coyne

J-S

, et al. Interlaboratory comparison for temporal light modulation measurement. IEEE Sustainable Smart Lighting World Conference and Expo (LS24), Eindhoven, The Netherlands, 12–14 November 2024.

22.

European Association of National Metrology Institutes. Metrology for temporal light modulation [Online]. Retrieved 5 December 2025 from https://www.euramet.org/research-innovation/search-research-projects/details/project/metrology-for-temporal-light-modulation

23.

International Electrotechnical Commission Technical Committee 34. Guidance to Build a TLA (Temporal Light Artefacts) Reference Radiator for Test Set-Up Validation Purposes. Geneva, Switzerland: IEC, 2020. https://www.iec.ch/ords/f?p=103:227:414094049344731::::FSP_ORG_ID,FSP_LANG_ID:1235

24.

Klej

Jackson

Beeckman

, et al. Flicker (pstLM) and stroboscopic effect (SVM) – light measurements in photometrical laboratories. Signify developed setup and validation method. Proceedings of the 29th CIE Session, Washington, DC, USA, 14–22 June 2019.

25.

Commission International de l‘Éclairage (CIE). Test Methods for LED Lamps, LED Luminaires and LED Modules. CIE S 025/E: 2015. Vienna, Austria: CIE, 2015.

26.

EA Laboratory Committee. EA-4/02 – Evaluation of the Uncertainty of Measurement in Calibration. Geneva, Switzerland: European Co-operation for Accreditation (EA), 2022.

27.

Joint Committee for Guides in Metrology. Guide to the Expression of Uncertainty in Measurement JCGM 100: 2008. Sèvres, France: Bureau International des Poids et Mesures, 2008.

28.

Ikonen

Nordlund

Mantela

, et al. Improvement in the temporal light artefact metrics of commercial LED lamps. Proceedings of the CIE Symposium on Advances in Measurement of Temporal Light Modulation, Athens, Greece, 11 October 2022.

29.

Mantela

Nordlund

Askola

, et al. Digital implementations for determination of temporal light artefacts of LED luminaires. Lighting Research and Technology 2024; 56: 734–743.

30.

Koch

Zuber

Anti-aliasing filter effects on sampling frequency and effects of mathematical implementation. Proceedings of the CIE Symposium on Advances on the Measurement of Temporal Light Modulation, Athens, Greece, 11 October 2022.

31.

Nordlund

Validation for measurement of Temporal Light Artefacts on LED light sources. Master’s thesis. Espoo, Finland: Aalto University, 2022.

32.

Bouroussis

Tsirbas

Canale

, et al. Arbitrary light waveform generator for the validation of the light flicker measurement devices. The 46th Annual Conference of the IEEE Industrial Electronics Society (IECON), Singapore, 18–21 October 2020.

33.

Dam-Hansen

Coyne

Isoardi

. Minimising the uncertainties in the calculation of stroboscopic effect visibility measure. CIE Symposium on Advances on the Measurement of Temporal Light Modulation, Athens, Greece, 11 October 2022.

34.

Dekker

van Bloois

AL.

Facility for calibration of photometers for measurement of temporal light modulation. Lighting Research and Technology 2023; 55: 414–419.

35.

Tan

Leon

Miller

NJ.

Temporal light modulation: data processing and metric calculations. Lighting Research and Technology 2024; 56: 755–771.

36.

Consultative Committee for Photometry and Radiometry Working Group on Key Comparisons. Guidelines for CCPR Key Comparison Report Preparation. Sèvres, France: Bureau International des Poids et Mesures, 2019.

37.

Dam-Hansen

Coyne

, et al. Influence of AC power supply on light flicker measurements. IEEE Sustainable Smart Lighting World Conference and Expo (LS24), Eindhoven, The Netherlands, 12–14 November 2024.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.07 MB