Test-retest reliability and validity of the Stages mountain bike power meter

Abstract

BACKGROUND:

Stages device is an affordable power meter used by several professional and competitive cyclists. However, few studies have investigated its validity and reliability.

OBJECTIVE:

The aim of this study was to analyze the validity and the reliability of the Stages mountain biking power meter.

METHODS:

Twenty-six cyclists (age: 35 $\pm$ 7 years, VO ${}_2$ max: 58 $\pm$ 7 ml $\cdot$ kg ${}^{-1}$ $\cdot$ min ${}^{-1}$ ) completed six testing sessions within a 4-week period. Cyclists performed an incremental exercise test, a 5-min time trial (TT), and a 20-min TT in different days using the Velotron cycle ergometer attached with the power meter.

RESULTS:

The power output of the Stages was significantly lower during each step of the incremental exercise test (11.8 to 16.4%) and performance testing ( $\sim$ 18%) compared with the cycle ergometer. Bland Altman plots presented high bias and limits of agreements when comparing the power output of the Stages and Velotron. In general, high coefficient of variation was found for the power and cadence of the Velotron and Stages, except for the Velotron’s power during the 20-min TT.

CONCLUSIONS:

The interchangeable use of the power from the Velotron and Stages is not recommended due to the observed high within-subject variation.

1. Introduction

Cycling power output (PO) has traditionally been measured using cycle ergometers in laboratory conditions; however, with the development of several portable power meters, the PO can be measured during cycling training, races, and testing in the field [1, 2, 3, 4]. In this regard, cycling is a sport where PO can be measured on the road, on off-road terrain, track cycling and BMX via portable devices attached to the bicycle.

Olympic cross-country is a modality in mountain biking that depends on the capacity of the cyclist to produce high PO in a variety of up-hill climbs during races [3, 5]. The cyclists and their bicycles need to be light since the peak power output (PPO) relative to body mass was previously strongly correlated with cross-country mountain biking performance [6]. The majority of the portable power meters are heavy and add 300 to 500 g in added weight to the bicycle [7]. Stages device (Stages Cycling, Saddleback Ltd., UK) is a power meter that measures torque via strain gauges in a tiny box weighing the only 20 g that is attached to the left crank-set of the bicycle. This seems to be an interesting advantage compared to the other models designed for cross-country MTB races. The device is not complicated to use since it only needs a bicycle computer to pair with the power meter via ANT $+$ or Bluetooth. The power meter does not need calibration before each use and the user can easily change the battery when necessary. Finally, the power meter is more affordable for the cyclists due to the lower price (i.e. $\sim$ 600 EUR) compared to the “gold standard” SRM (i.e. $\sim$ 1500 EUR) [7]. However, as the power meter only measures the torque generated from the left side, the device under-reports the symmetries related between both legs.

A literature survey revealed one study case related to the comparisons between the Stages MTB (Shimano XT M8000) and the SRM portable power meter during mountain biking up-hill climbs [7]. This study indicated that both power meters provided a reliable means of recording mean PO and cadence though peak power values were less reliable. In addition, significant differences were found between the two systems for mean PO ( $p<$ 0.001), with the Stages power meter under-reporting power by 8 $\pm$ 1 % compared to the SRM [7]. Recently, Miller et al. [8] reported no differences ( $p>$ 0.05) between Stages (SRAM X9), Quarq and Power tap during treadmill riding ( $n=$ 4) at high or low cadences. However, dissimilarities for both power and cadence were apparent during actual cross-country mountain bike riding ( $n=$ 8). Therefore, we do believe that a study with a large sample size with more controlled laboratory conditions simulating cycling performances could provide new insights into the reliability and validity of the Stages MTB power meter.

One of the most traditional tests to determine the PO of the physiological marker determinants of endurance performance is the graded exercise test (GXT) performed on cycle ergometers [9]. In addition, cycling endurance performance is frequently measured by a range of short, medium, and long time trials (TT) [10]. A number of cycle ergometers have been developed and assessed including the Velotron Dynafit Pro cycle ergometer (RacerMate Inc, WA, USA). The Velotron is a reliable and valid ergometer that was compared over a range of exercise intensities and durations with the calibration rig [11]. Therefore, the Velotron may be used as a valid criterion to evaluate PO during GXT and cycling performance during TT [11, 12, 13]. In this regard, the aim of this study was to analyze the validity of the Stages mountain biking power meter with a cycling GXT. In addition, the study aimed to verify the test-retest reliability and validity of the power meter during performance tests.

2. Methods

2.1 Participants

Twenty-six male mountain bikers (mean $\pm$ SD; age: 35 $\pm$ 7 years, mass: 76 $\pm$ 9 kg, height: 177 $\pm$ 5 cm, VO ${}_2$ max: 58 $\pm$ 7 ml $\cdot$ kg ${}^{-1}$ $\cdot$ min ${}^{-1}$ ) volunteered to participate in this study. All participants gave their written informed consent to participate in the study, which was approved by the local research ethics committee dated 19.12.2016, # 61343516.0000.0118. All cyclists were classified as trained according to a previous recommendation and competed regularly in mountain biking competitions [14]. All participants were familiar with the test procedures used in this study had previously participated in a reproducibility study which utilized similar test procedures.

2.2 Study design

The study was a repeated, measured, controlled trial where cyclists completed six testing sessions within a 4-week period. In the first session cyclists performed a GXT. The second session was used as a familiarization, and the athletes performed a 5-min TT followed by a 20-min TT separated by 20-min of rest. The following sessions were in a randomized order where participants performed a 5-min TT or a 20-min TT. All physiological and performance assessments were completed on a Velotron Dynafit Pro (RacerMate Inc, WA, USA) cycle ergometer calibrated in accordance with the manufacturer’s instructions. The Velotron ergometer has previously been shown to provide valid and reliable power measurements during steady state output in comparison to a gold standard calibration rig [11]. Prior to testing each participant was fitted to the ergometer in a position to replicate his own racing bicycle. The positions were recorded and repeated for the subsequent session. During each session, the Velotron was attached with the Stages mountain biking power meter paired (via ANT+ protocols) to a Garmin (Edge 520) cycle computer and recorded data at $\sim$ 1 Hz. The Stages mountain biking power meter is a tiny box with a total weight of 20 g already attached to the left crank by the manufacturer (Stages Cycling, Saddleback Ltd., UK). The Stages contains strain gauges that calculate the torque, and consequently, the instantaneous PO is a simple product of the torque and the cadence. We used the model Stages mountain bike power meter Shimano XT M8000 (G1) with a crank length of 175 mm. The Stages was calibrated in accordance with the manufacturer’s instructions. In the 24 hours before each testing session, participants were instructed to prepare as if it was a competition and to avoid strenuous physical activity and any potential performance altering supplements prior to the trials. Throughout all the tests, the ambient temperature of the laboratory was controlled at $\sim$ 20 ${}^{\circ }$ C with a relative humidity of $\sim$ 50%. For the TT tests, no verbal encouragement was provided and water was ingested ad libitum.

2.3 Maximal incremental exercise test

Cyclists initially completed a 10-minute warm-up at a self-selected intensity. The Velotron ergometer was then set to the isokinetic mode for the test to ensure that power output remained constant regardless of any fluctuations in pedal cadence. Cyclists were requested to maintain a preferred cadence throughout the test. The test commenced at 100 W and was increased by 25 W every minute until volitional exhaustion. The PPO was determined as the last completed stage plus the fraction of time spent in the final non-completed stage multiplied by 25 W [15]. For the PO mean comparison between Velotron and Stages, the power output was selected from the beginning of the test up to a complete power of 300 W. Mean PO for each exercise step was downloaded from the Stages and used for subsequent data analysis.

Oxygen uptake (VO ${}_2$ ) and heart rate (HR) were continuously measured using a gas analyzer (Quark PFTergo, Cosmed, Rome, Italy) calibrated in accordance with the manufacturer’s instructions. Maximal oxygen uptake (VO ${}_{2\max }$ ) was determined as the highest 30-s mean value recorded and maximal HR (HR ${}_{\max }$ ) as the highest individual value. The workloads corresponding to ventilatory thresholds 1 and 2 (VT ${}_1$ and VT ${}_2$ , respectively) were also identified. VT ${}_1$ was determined using the criteria of an increase in both VE $\cdot$ VO ${}_2^{-1}$ and PETO ${}_2$ with no concomitant increase in VE $\cdot$ VCO ${}_2^{-1}$ [16]. VT ${}_2$ was determined using the criteria of an increase in both the VE $\cdot$ VO ${}_2$ ${}^{-1}$ and VE $\cdot$ VCO ${}_2^{-1}$ and a decrease in PETCO ${}_2$ [16].

2.4 Time-trials

The first TT session was used as familiarization and cyclists performed a 5-min TT and a 20-min TT, respectively, with a passive rest interval of 20-min between the trials. The following sessions were performed in a randomized order separated by at least 48 hours. The tests were using the same cycle ergometer that allowed the cyclists to manually shift the gears attached on the handlebar while pedaling. For each test, the Velotron was connected to a laptop computer interfaced with a projector that displayed the computer-generated image of the 3D course profile in front of the cyclist (Interactive 3D, RacerMate, Seattle, WA, USA). A flat terrain without wind was modeled through the proprietary software. During the TT, participants were able to view their progress over the course on a computer screen and were provided with information on elapsed time and gear selection, to avoid any pacing effects.

2.5 Statistical analysis

Simple descriptive statistics are shown as means $\pm$ SD. Two purposely made Microsoft Excel spreadsheets were used to determine the validity and test re-test reliability of the data retrieved from the Stages [17]. The data from the Stages was analyzed as both raw values and also as log-transformed values in order to reduce any effects occurring through non-uniformity of error in the data. Reliability between TTs was analyzed as the raw unbiased typical error and also expressed as the coefficient of variation (CV) with accompanying confidence intervals ( $\pm$ 95% CL). The utilized spreadsheet also provided the intraclass correlation (ICC type 3:1) between repeated trials. The validity was analyzed by Pearson’s r correlation to determine the strength of the relationship between Velotron and Stages from each test step. For multiple comparisons, two-way analysis of variance (ANOVA) with repeated measures was also performed to determine any significant differences between PO from the Velotron and Stages at each exercise mode (GXT, 5-min TT, and 20-min TT) with subsequent post-hoc comparisons (Bonferroni) when necessary. The bias and limits of agreement (LoA) of the mean PO differences between Velotron and Stages were defined using the method of Bland and Altman [18]. All significance based analyses were performed using SPSS statistical software, version 20 for Windows (SPSS Inc, Chicago, IL, USA) with alpha set at 0.05.

Table 1

Mean power output, coefficient of variation, and intra-class coefficient of correlation from Stages and Velotron during the 5-min time trial

	Stages		Velotron
	Power (W)	Cadence (RPM)	Power (W)	Cadence (RPM)
TT1	241 $\pm$ 50	106 $\pm$ 11	295 $\pm$ 56	106 $\pm$ 11
TT2	249 $\pm$ 46	105 $\pm$ 10	301 $\pm$ 48	105 $\pm$ 10
Mean	245 $\pm$ 47*	105 $\pm$ 10	298 $\pm$ 51	105 $\pm$ 10
CV (%)	6.2 (4.9–8.6)	3.1 (2.4–4.3)	5.7 (4.5–7.9)	3.1 (2.4–4.3)
ICC	0.99 (0.93–1.00)	0.99 (0.92–1.00)	0.98 (0.90–1.00)	0.99 (0.92–1.00)

PO: power output; TT: time trial; CV: coefficient of variation; ICC: intra-class correlation coefficient. *: Significant different to Velotron ( $P<$ 0.05).

Table 2

Mean power output, coefficient of variation, and intra-class coefficient of correlation from Stages and Velotron during the 20-min time trial

	Stages		Velotron
	Power (W)	Cadence (RPM)	Power (W)	Cadence (RPM)
TT1	202 $\pm$ 36	99 $\pm$ 10	245 $\pm$ 37	100 $\pm$ 10
TT2	203 $\pm$ 37	98 $\pm$ 10	246 $\pm$ 37	99 $\pm$ 10
Mean	202 $\pm$ 36*	98 $\pm$ 10	245 $\pm$ 37	99 $\pm$ 10
CV (%)	4.1 (3.2–5.7)	4.3 (3.4–5.9)	2.4 (1.9–3.3)	4.0 (3.1–5.5)
ICC	0.99 (0.95–1.00)	0.97 (0.82–1.00)	1.00 (0.98–1.00)	0.98 (0.97–1.00)

PO: power output; TT: time trial; CV: coefficient of variation; ICC: intra-class correlation coefficient. *: Significant different to Velotron ( $P<$ 0.05).

Figure 1.

Stages and Velotron power distribution during time-trials.

Figure 2.

Bland-Altman plots of power output between Stages and Velotron.

3. Results

The Velotron and Stages mean $\pm$ SD for the test-retest mean PO and cadence, CV, and ICC during the 5-min TT and 20-min TT are presented in Tables 1 and 2, respectively. There were no significant differences between Velotron TT1 vs. TT2, and between Stages TT1 vs. TT2 for the 5-min TT and 20-min TT, respectively. The PO from Velotron was significantly higher than Stages for the both TT ( $p=$ 0.0001). High CV scores were found for the PO and cadence of the Velotron and Stages, except for the Velotron’s PO during the 20-min TT and the cadence during the 5-min TT. High values of ICC were found for the mean PO and cadence between both TT under the Velotron and Stages, respectively.

The mean values in each step of the GXP were significantly ( $P<$ 0.001) lower for the Stages during the test in a range of 15 $\pm$ 8 W in the beginning to 35 $\pm$ 14 W at the last step (300 W). In addition, LV1, LV2, and PPO were significantly lower for the Stages power meter (194 $\pm$ 47 W; 263 $\pm$ 40 W; 332 $\pm$ 43 W; $P<$ 0.001) compared to the Velotron (228 $\pm$ 44 W; 304 $\pm$ 36 W; 381 $\pm$ 48 W; respectively). Furthermore, strong $r^2=$ 0.999 between the Stages and Velotron was found. Also, an equation to calibrate PO from Stages to that of the Velotron can be used ( $y=0.905x-8.373$ ; SSE $=$ 3.5 W).

The absolute and normalized PO during each 30 s of the 5-min TT and each 2 min of the 20-min TT for the Velotron vs. Stages are shown in Fig. 1A–D. The PO in each step over the both TT was significantly ( $P<$ 0.001) lower for the Stages ( $\sim$ 18%) than that produced using the Velotron. However, the normalized PO showed similar values between the devices except for the beginning of the TT and in the 16 ${}^{\text{th}}$ minute of the 20-min TT ( $P<$ 0.01). In addition, the mean PO from the Stages was significantly lower during the 5-min TT and 20-min TT than the Velotron (Tables 1 and 2, respectively) and respectively the Bland-Altman plots showed very high bias and LoA for all variables compared (Fig. 2A–E).

4. Discussion

The main finding of this study was that the PO of the Stages MTB power meter was significantly lower during each step of the GXT (11.8 to 16.4%) and performance testing ( $\sim$ 18%) compared with the cycle ergometer. The Bland Altman plots presented a high bias on comparison of the PO of the Stages and Velotron. In general, high CV scores were found for the PO and cadence with the Velotron and Stages, except for the Velotron’s PO during the 20-min TT. Moreover, high values of ICC were found for the mean PO and cadence between both TT under the Velotron and Stages, respectively. Therefore, the interchangeable use of the PO from the Velotron and Stages, or vice versa is put in doubt due to the high within-subject variation that might be found.

A previous study reported a CV of 5.5% for the mean PO of the Stages MTB power meter across three consecutive days on an off-road terrain for the same subject [7]. The high CVs found by Hurst et al. [7] was related to the MTB terrain where vibrations and changes in cycling position (i.e. seating vs. standing). In addition, the cumulative fatigue of the consecutive days could provide for a less reliable PO. Recently, Granier et al. [19] investigated the reliability of the Stages road cycling device in eleven males during GXT and sprint exercise. The authors found CV values of 3.8% and 4.2% for the GXT and the sprints, respectively. Similarly, the results of the present study showed high CV values of 6.2% and 4.1% for the 5-min TT and 20-min TT, respectively. The reasons for the high CV are unknown but certainly not related to the “off-road conditions” due to a more controlled environment in the laboratory. The high CV associated with the Stages might be related to the lower level of experience of the mountain bikers and consequently affects the mean PO and pacing strategy during the performance test. This is supported by the lower intensity sustained during the 5-min TT ( $\sim$ 80% of the PPO) compared with a time to exhaustion at PPO ( $\sim$ 4-min) in well-trained cyclists [20]. The reliability of a power meter sets the limits for its ability to track the potential worthwhile changes in cycling performance [21]. Thus, the reported CV for the Stages presented a considerable within-subject variation, and as such would limit the devices ability to track worthwhile changes in cycling performance for professional cyclists.

The Stages produced significantly lower power in all the subjects during each step of the GXT. The mean bias in PO over the 100–300 W range was substantially lower for the Stages, ranging from 16.4% to 11.8%. Despite the Stages differences with the cycle ergometer, it reported strong linear relationships with the Velotron PO (Fig. 2). Given the strong correlations and low standard error of estimate between the measured PO, a correction formula for each individual’s test data could be used interchangeably between the Stages and the Velotron. In addition, during the performance tests, the Stages device was around 18% lower than the Velotron. However, when the PO was normalized by the mean PO both devices presented similar percentage values. Therefore, the normalized PO displayed a pacing strategy of a “U” shape and reverse “J” shape [22] for the 5-min TT and 20-min TT, respectively.

The reason for the large bias range between Stages and Velotron is unknown, but it is presumably associated with the technology to measure the PO. While PO of the Velotron is calculated as a combination of the known resistance from eddy current braking, and any change in speed/inertia to the flywheel; Stages records power using a meter on the left crank arm that measures torque and cadence. The tiny box attached to the left crank contains strain gauges and an accelerometer. Once torque and cadence are collected they are multiplied together to determine the power of a single stroke. Therefore, PO is multiplied by two to provide the estimative for both legs. In fact, Stages power meter is limited by the measured torque and cadence in a single crank arm. This could enhance the errors of the PO for cyclists with larger asymmetries between the legs. Previous data reported that cyclists present an imbalance in bilateral force production from 5–20% [23]. Thus, although the leg asymmetries can indeed affect PO calculation using the Stages devices, a weak link was found between leg asymmetries pedal forces and 4-km TT and 40-km TT [24, 25].

5. Conclusions

The Stages MTB power meter (G1) is a simple to use lightweight device designed to monitor PO with cyclists. While the Stages may provide some estimation of PO it lacked validity compared to the Velotron cycle ergometer, and reported poor performance retest reproducibility (CV $\sim$ 5%). The poor reliability of the device renders it unsuitable for monitoring the small (1–2%) changes in performance power for professional cyclists. Given the large between-trials CV of the Stages, we would suggest that any small changes in PO of less than 5% identified by the Stages should be interpreted with caution. However, while the Stages might not be appropriate for monitoring small changes in power with professional cyclists it may provide useful power information for competitive cyclists or for those expected to make large improvements in their power following training.