Assessing ambient temperature measurements in road cycling races

Abstract

The impact of heat stress on sports performance is a highly relevant topic. Environmental factors, such as temperature and humidity, play a significant role in determining athletic performance. Cycling computers are commonly used in road cycling to capture various data, including temperature. However, it is crucial to emphasize the importance of accurately measuring ambient temperature. This study aimed to assess the reliability and validity of the Garmin Edge 830 cycling computers in measuring ambient temperature during road cycling competitions. We analyzed temperature data collected by 12 different cycling computers. We focused on assessing these devices’ reliability and validity compared to measurements obtained from an official meteorological station. The interclass correlation coefficient between the devices was 0.468 (95% CI: 0.254 to 0.626; p < 0.001). The temperature readings exhibited a significant variability across the different devices. Furthermore, when comparing the data with the readings from the meteorological station, there was no agreement, as indicated by a Lin's Concordance Coefficient (LCC) of 0.398 and a Mean Absolute Percent Error (MAPE) of 8.87%. This research raises concerns regarding the reliability and validity of cycling computer devices for measuring ambient temperature.

Introduction

The impact of heat stress on sports performance has received increasing attention. ¹ Environmental factors such as temperature and humidity have a substantial impact on athletic performance, leading to a growing body of research in this field.^2–5 Among these factors, temperature is particularly important and can present significant challenges for athletes. ⁶ In endurance sports like cycling, temperature regulation is critical as it directly affects metabolic efficiency, cardiovascular strain, and the body's ability to maintain optimal performance levels during prolonged efforts.^6–8

Currently, to measure the effects of temperature on performance, the wet bulb globe temperature (WBGT) is used to quantify thermal stress.^6,9 However, when WBGT measurements are unavailable, ambient temperature and humidity are used as alternative indicators. ⁹ In many cases, coaches only have temperature data and use this value as a reference of heat strain. For example, coaches can predict dehydration based on the ambient temperature. ⁶

In road cycling, cycling computers are expected to record data during races and training sessions to quantify internal and external load. These cycling computers are equipped with temperature sensors that allow the recording of temperature data throughout the entire route. Consequently, some studies have used these data to assess the impact on sports performance. ¹⁰ The authors suggested that the power values of the cyclists changed depending on the ambient temperature and concluded that the optimal temperature range is between 10°C and 25°C.

However, it must be taken into account that the cycle computers are placed on the handlebars of the bicycle. This placement is not the most suitable for measuring ambient temperature because the sensor is exposed to direct solar radiation, which may affect the measurement. In addition, for example, the technical specifications provided for Garmin Edge devices lack essential details such as accuracy data, sampling rate, and response time of temperature sensors. As a result, it is difficult to assess the reliability of the sensor right from the start.

In previous studies of marathon running, researchers have discovered the presence of microclimates along the competition course. ¹¹ While no specific research has been conducted on microclimates in cycling, it is reasonable to speculate that similar conditions could exist in cycling events due to their similarities with running. Accurately measuring the ambient temperature is essential to estimating the thermal stress experienced by cyclists. To ensure the reliability and validity of temperature measurements, it is important to consider both concepts. Reliability ensures consistent measurements, while validity confirms the accuracy of the device's readings. Both factors are crucial when relying on device measurements, particularly in the case of ambient temperature.

Moreover, it is important to consider that a standardized protocol must be followed when measuring ambient temperature. It is recommended to conduct temperature measurements in shaded areas, away from direct solar radiation, as it has been observed that sunlight can distort temperature readings.^12,13

Therefore, accurate temperature measurements are essential to obtaining unbiased and reliable data. The reliability and validity of measuring ambient temperature using cycling computers mounted on bike handlebars have yet to be established. Our main hypothesis is that cycling computers yield biased ambient temperature readings. Hence, the objective of this study was to analyze the reliability and validity of the Garmin EDGE 830 cycling computers in measuring ambient temperature during road cycling competitions. It was hypothesised that the Garmin EDGE 830 cycling computers are useful for measuring ambient temperature.

Methods

The data were obtained from 12 Garmin EDGE 830 (Garmin International Inc, Olathe, KS) devices used in the under-23 (U23) road cycling race in Amurrio (Araba, Basque Country, Spain). The road race took place on a circuit around the village. The researchers collected the “.fit” file, which was recorded using a cycling computer Garmin EDGE 830. All the devices were mounted in their standard location on the bike handlebar. All the data were recorded throughout the race. The temperature was recorded with an integrated sensor in a cycling computer. We analyzed the temperature data for the 2-h duration of the race. This research follows the ethical principles established by the Declaration of Helsinki and its subsequent revisions. As the University of the Basque Country ethics committee indicated, ethical approval is unnecessary for using ambient temperature data.

The official meteorological station (Euskalmet, Basque Country) was used as a reference method. This station provided temperature data at 10-min intervals. The farthest point of the circuit and the meteorological station were approximately 5800 meters apart. The altitude of the meteorological station was 230 m, and the range altitude of the circuit was between 182 m and 352 m. On the race day, the sky was clear, and the temperature ranged between 21.6°C and 24.6°C. In order to synchronize both datasets, we extracted the temperature from the device every 10 min, resulting in a total of 12 time points.

Statistical analysis

Data analysis was performed using the R programming language. The normality of the data was assessed using the Shapiro-Wilks test. Means, standard deviations (SDs), and coefficients of variation (CVs) were calculated using standard statistical methods.

Relative reliability was assessed using intraclass correlation coefficients (ICCs) with 95% CIs. ICC (two-way mixed model, single measures, absolute agreement) was selected based on guidelines published elsewhere. ¹⁴ One-way ANOVA was conducted, followed by a Bonferroni pos-test to examine pairwise differences. The effect size (ES) of one-way ANOVA effects was determined using partial eta squared (ηp2).

Construct validity was assessed using independent Student's T-test to compare mean differences between the two methods. The differences’ magnitude was evaluated using a 95% Confidence Interval (CI) and Cohen's effect size (ES). Agreement was assessed using Lin's Concordance Coefficient (LCC), Mean Absolute Percent Error (MAPE) and Bland–Altman plots with limits of agreement (LoA). Homoscedasticity was assessed by visualizing Bland–Altman plots. Significance was set at p < 0.05.

Results

Reliability

The cycling computer devices recorded a temperature range between 18°C and 33°C. Figure 1 shows the average of all devices at each time point. (Figure 1).

Figure 1.

Temperature data from cycling computers during two-hour race time.

ICC was 0.468 (95% CI: 0.254 to 0.626; p < 0.001). Significant differences were observed between devices (p < 0.05; ηp2 = 0.464). All pairwise combinations were significant except for two pairwise comparisons. The mean SD for all recorded time points was 2.14 °C, ranging from 1.66 °C to 3.08 °C. The mean CV was 9.09%, ranging from 6.80% to 11.60%. The mean range between the minimum and maximum recorded temperatures at each time point was 5.7 ± 0.86°C (min = 4°C; max = 9°C). Table 1 presents the descriptive data of 12 time points recorded by the device.

Table 1.

Descriptive data of 12 time points from cycling computers.

	Mean (°C)	SD (°C)	CV (%)	MIN (°C)	MAX (°C)
1	22.08	2.11	9.55	19.00	25.00
2	21.00	2.22	10.55	18.00	24.00
3	21.08	1.98	9.37	19.00	24.00
4	22.42	2.07	9.21	20.00	25.00
5	22.25	2.34	10.52	19.00	26.00
6	23.25	2.09	9.01	21.00	26.00
7	24.92	2.19	8.80	22.00	28.00
8	23.50	2.15	9.16	21.00	26.00
9	25.42	1.93	7.59	23.00	29.00
10	25.17	2.04	8.10	22.00	28.00
11	25.58	2.11	8.24	23.00	28.00
12	28.08	2.11	7.51	25.00	31.00

Note: Mean: mean of all cycling computers at each point; Max: maximum temperature registration from cycling computers; Min: minimum temperature registration from cycling computers; SD: standard deviation; CV: coefficient of variation.

Validity

We observed significant mean differences between the cycling computers and the meteorological station (p = 0.006, ES = 0.255). The LCC was 0.398 (95% CI: 0.32 to 0.472), and MAPE was 8.87%. The agreement can be observed in Figure 2 and Figure 3. The mean bias was 0.55 ± 2.36°C (Lower LoA = -4.07; Upper LoA = 5.18). On Figure 3

Figure 2.

Bland-Altman analysis to compare the agreement between cycling computers and the official meteorological station.

Figure 3.

The cycle computers and the meteorological station temperature recorded at 12 different time points.

Discussion

Accurately measuring ambient temperature is essential for estimating the athlete's performance in heat situations.^6,10 This study represents the first analysis of the reliability and validity of ambient temperature recordings using cycling computers. The findings revealed that the current recording method using the Garmin EDGE 830 is inaccurate.

Our findings revealed that these cycling computers recorded a wide range of temperatures at the same time point, indicating a low ICC and significant differences between variables. This lack of reliability presents a challenge when comparing data among cycle computers. Given that cycling performance is highly sensitive to environmental conditions, particularly temperature, effective thermoregulation strategies are essential for optimizing endurance and preventing heat-related impairments. ⁶ Studies have shown that even moderate increases in ambient temperature can exacerbate cardiovascular strain, reduce muscle blood flow, and accelerate the onset of fatigue.^7,15 Thus, with a valid device for measuring ambient temperature, coaches could make decisions based on ambient temperature.

Despite participating in the same race, if we look at the temperature data, it may seem that each cyclist participated in a different race. For instance, in a study conducted by Valenzuela et al. 2022 ¹⁰ cycling computers were used to examine the impact of ambient temperature on power output. However, based on our study, the results seem to indicate that these devices may not be valid.

The results show lack of validity, as evidenced by low LCC and a high MAPE. Regarding agreement, although the mean bias is low, the LoAs are considerable. The temperature accuracy is set at 0.1°C below our mean bias. ¹² Moreover, visual analysis revealed the presence of heteroscedasticity, as the mean bias increases with higher ambient temperatures. The discrepancy between the meteorological station and cycling computers may be attributed to external ambient variables that could influence the measurements. The lack of accuracy could be attributed to direct solar radiation, as observed in other studies. ¹² A solution to the radiation issue could involve relocating the temperature sensor on the bike, for example, underneath the saddle. Furthermore, the discrepancy between the weather station and the EDGE devices could be caused by local microclimates. Similar to the marathon scenario, ¹¹ there may be microclimates along the course that make it challenging to compare with the weather station.

This lack of agreement presents challenges in understanding the influence of temperature on performance and hinders data comparison. Nonetheless, using cycling computers for temperature measurement remains prevalent among scientists and trainers. ¹⁰ However, these cycling computers seem to not be valid for quantifying WBGT during a race. It is crucial to consider that temperature accounts only for 10% of the WBGT equation. ¹³ As highlighted by Lemke and Kjellstrom 2012, ¹³ a one-degree change in air temperature roughly corresponds to a one-degree change in WBGT. Another current method for assessing heat strain involves considering temperature and humidity. ⁹ Nevertheless, these cycling computers are not suitable yet for accurately assessing heat strain.

Our study found that temperature sensors on cycle computers, or their placement, may encounter issues when calculating ambient temperature. One proposed solution to this problem is for various companies to produce a small, portable weather station that can be placed in an optimal location on the bicycle. This idea has already been suggested for measuring thermal stress using external body temperature sensors. ¹⁶ The authors emphasize the importance of combining both measurements to predict an athlete's thermal stress based on ambient temperature.

The main limitation of this study is the “black box effect”, where there is a lack of knowledge regarding the specific mechanisms by which the Garmin EDGE 830 cycling computers record temperature. Understanding this process is crucial for comprehending the observed lack of accuracy. For instance, while we observed a 1°C accuracy in the device temperature output, we do not have information regarding the sampling frequency or the type of sensor used.

Another limitation is the range of actual temperatures measured during the race. The presence of heteroscedasticity highlights the importance of analyzing accuracy under lower and higher temperature conditions. Additionally, since the official meteorological station reported data every 10 min, increasing the sampling frequency may lead to slightly different outcomes. Finally, the research was conducted only for one race and 12 devices. While this approach has facilitated the comparison with the same meteorological station, it is worth noting that the circuit's characteristics could impact the measurements. To tackle this issue, considering the potential invalidity of the sensors, it is advisable for future research to adopt the methodology utilized by Kosaka et al. 2018 ¹¹ which compared the mobile weather station with the cyclo-switches. Moreover, it would be beneficial to include a wider array of devices from different manufacturers to enhance the sample size of device. Lastly, the sample size was limited, and despite efforts to reach out to more devices, it was not feasible. As a result, the anticipated statistical power was not attained. Nonetheless, the findings demonstrate the behaviour of these devices, which should be checked in future research.

Conclusions

In conclusion, this research raises concerns regarding the reliability and validity of the Garmin EDGE 830 cycling computer devices for measuring ambient temperature. Coaches and scientists should exercise caution when utilizing data from these devices, considering the observed discrepancies with the official meteorological station. Future research should clarify whether the results found in this study are generalizable to other models and brands of cycling computer devices. Also, future research should prioritize identifying optimal locations for bicycle temperature sensors to enhance data accuracy and validity by comparing with mobile weather stations.