Utility of video analysis and expert modelling for technique development in novice sport climbers: A randomized controlled study

Abstract

Sport climbing is a multifaceted sport that also requires appropriate techniques to optimize movements. As augmented feedback is known to facilitate motor learning, we investigated the utility of adding video analysis and expert modelling to standardized verbal feedback for the acquisition of three climbing-specific techniques (drop knee, heel hook and high step). Twenty-six novice climbers (12 women) completed two testing sessions before and after a training intervention that consisted of three coached climbing sessions targeting the three techniques. Participants were randomly assigned to a control group, which solely received standardized verbal feedback or an experimental group that additionally received standardized video analysis and expert modelling using the Dartfish tablet application. Video recordings were subsequently evaluated by two climbing experts on a 7-point scale. The expert scores were higher in the experimental than in the control group for the high step (causal total effect (CTE) 0.38, 95% confidence interval (CI) [0.06, 0.68]). Improvements for the drop knee (CTE = 0.12, 95% CI [−0.24, 0.48]) and heel hook (CTE = −0.05, 95% CI [−0.42, 0.31]) were similar in both groups. For the drop knee, we, however, observed a positive causal direct effect (CDE = 0.38, 95% CI [0.07, 0.68]), which was comparable to that observed for the high step but also a negative causal mediator effect via the perceived difficulty (CME = −0.26, 95% CI [−0.51, −0.04]). Compared to verbal feedback solely, the addition of video analysis and expert modelling might facilitate the acquisition of certain climbing techniques, such as high step, in novice climbers.

Keywords

Augmented feedback gender movement analysis skill learning

Introduction

Recreational as well as competitive sport climbing are becoming increasingly popular around the world.¹ This is also reflected in the inclusion of climbing in the 2020 Tokyo Olympics and further approval for the 2024 Paris and 2028 Los Angeles Olympic games.² Given that sport climbing, hereafter referred to simply as climbing, might be considered a predominantly physical sport requiring considerable upper body power and endurance as well as flexibility, it is not surprising that the vast majority of up-to-date climbing-related research focused on the physiological, anthropometrical and physical conditioning aspects.^3–8 However, given the complex nature of the climbing movement, it is clear that climbing success also highly hinges on the efficiency and economy of movement as well as movement coordination.⁹ Indeed, the best climber is not necessarily the strongest, but the one who is more fluent, efficient and faster in selecting and connecting appropriate movements and body positioning while minimizing jerky movements and locomotion mistakes.^9,10 Regardless of the climbing discipline, no two routes/problems are ever the same, so climbers are always faced with unique challenges and movements.

While the climbing movement is inherently varied, there are, nevertheless, some typical movement patterns, i.e. movement skills and/or techniques that are often employed by climbers and are known to facilitate climbing efficiency. The most common modern climbing techniques include drop knee, heel hook and high step.¹¹ Drop knee is one of the most efficient techniques when climbing overhanging walls since, when correctly performed, most of the climber's weight is transferred to the legs.¹¹ Efficient use of the heel hook can be useful at any angle of the wall, and when used correctly it can substitute a good hand hold by utilizing the power of the leg and the core.¹² The efficiency of the high step, or more generally using the left-right rule, which is particularly useful in vertical to slightly overhanging terrain, stems from the synergy of the opposite arm and leg interaction via a constant transfer of force and torque through the climber's body.¹¹ Learning these basic techniques seems crucial for novice climbers as they are challenging to rectify if learned and reinforced incorrectly. Furthermore, an incorrect technique often leads to increased strain and excessive loads on the fingers and upper extremities, known to be associated with increased injury risk.^13–15 Therefore, the role of a coach is critical within the early climbing technique acquisition period, to ensure that the basic techniques are learned correctly and safely. The coach must convey appropriate feedback information, which should include only key skill/movement information and not be too extensive. A trainee's visualization of the movement is therefore dependent on the coach's instructions and feedback information. The importance of appropriate feedback for movement learning facilitation has previously been demonstrated in different sports and scenarios.¹⁶ Furthermore, previous research suggests that task-related information is particularly beneficial for more complex tasks as compared to simple tasks.¹⁷ Given the complexity of climbing movement, requiring the synchronized use of all extremities to successfully perform the given task^9,18 optimizing and or augmenting feedback seems crucial to facilitate the learning process.

In this regard, the use of different behavioural interventions to facilitate climbing skill acquisition might prove beneficial as coaches often undertake skills and technique teaching/development without appropriate behavioural assistance. There have been many attempts in various sports to use behaviour analytic procedures, including consequence (e.g. verbal feedback and reinforcement), antecedent (e.g. instructions, goal setting & modelling) and other (self-talk and behavioural reinforcement) procedures to enhance skill acquisition.¹⁹ While ample evidence supports the use of various augmented feedback strategies on learning gross motor skills in various sports such as gymnastics,²⁰ volleyball,²¹ tennis²² and swimming,²³ and fine motor tasks, such as in rifle shooting²⁴ the extent to which augmented feedback can assist in technique acquisition in more complex sport-specific skills remains poorly understood, as various sources of information can have different effects on the success of the learning process.²⁵ Nevertheless, video analysis, whereby the trainee views his/her performance video before or just after the attempt and expert modelling, whereby the trainee compares his movement execution to the one done by an expert, have both demonstrated promising results across various sports.^19,26 Even though the potential of behaviour analytic technology in climbing has been discussed previously,²⁷ little empirical evidence exists regarding its utility. The only study to date on the topic in climbing was conducted by Walker and colleagues¹⁸ and aimed to investigate the role of video analysis and expert video modelling combined with verbal feedback in promoting accurate target skill performance in three novice climbers. They showed that the participants exhibited an increase in performance accuracy for selected techniques following a training period. However, the generalizability of this study's results is limited by the small sample size and the lack of a proper control group.

Accordingly, the purpose of the present study was to investigate the potential additional effect of video analysis and expert modelling relative to regularly coached climbing sessions using verbal feedback solely, on the accuracy of execution of the following three climbing-specific techniques: drop knee, heel hook and high step. We hypothesized that during the coached climbing sessions, video analysis and expert modelling would provide additional benefits for the technique development in novice climbers.

Methods

Participants

As noted in Figure 1, we initially recruited 30 participants (20 female and 10 male) from the visitors of a commercial climbing centre (Bolder Scena: 28 participants) and the students of the Faculty of Sports (University of Ljubljana; 2 participants). During the study, 4 participants (2 females from the control group and 1 male and 1 female from the experimental group – all recruited among the commercial climbing centre visitors) tested positive for SARS-CoV-2 and were unable to complete all study visits. Subsequently, only the data from the remaining 26 participants who completed all experimental sessions were used for the analysis reported in the present paper. The inclusion criteria included: (1) age between 20 and 40 years, (2) height between 160 and 185 cm and (3) climbing for ≤6 months and only very easy or easy boulders, approximately equivalent to Fb. grade 5A or less – fitting the lower grade (Level 1) the male and the female climbing group as defined by the International Rock Climbing Research Association.²⁸ They were subsequently randomized (stratified by gender) into a control group (n = 15) and an experimental group (n = 15). The participants were informed of the general aims of the study and all experimental procedures before providing written informed consent. However, they were blinded to the fact that the study involves two groups; the participants from the two groups were never mixed during the study to assure blinding regarding the two feedback approaches. The study was approved by the Commission for Ethical Questions in Sport at the Faculty of Sport, University of Ljubljana (02/2022). All experimental sessions were conducted at the Bolder Scena climbing centre using a portable angle-adjusting climbing wall (cf. supplemental material for details and graphical outlines on the boulder problems setup).

Figure 1.

The study outline and flow chart.

Study design

As depicted in Figure 1, the present randomized controlled study comprised three phases: (1) the pre testing phase, (2) the intervention phase and (3) the post intervention phase. Each participant took part in five climbing sessions: one session in Phases 1 and 3, and three sessions in Phase 2. The sessions were evenly distributed across a period of 15–17 days ensuring that participants rested for at least 48 hours between the sessions. At most, two participants (both from the same group) completed a session simultaneously, but they were not allowed to observe each other during their attempts. All sessions for all participants were conducted by the same coach that has been involved in the sport of climbing for 16 years and has 9 years of coaching experience. Each session started with a 15-minute standardized and structured warm-up comprising static and dynamic body weight and elastic band exercises performed under the supervision of an expert coach. During the main part of each session, all participants climbed six different boulder problems and always performed three repetitions of each. Two completely symmetrical boulder problems, consisting of two moves, were set for each of the three targeted skill tasks (drop knee, heel hook and high step; cf. supplementary material). Each problem was specifically set in a way that resulted in easier movement completion when appropriately using the targeted skill than without and/or reliant only on physical capacity. The same problems were used throughout the experiment. The order of the six boulder problems was randomized for each climbing session.

Phases 1 and 3 (Pre and post intervention testing): During these two phases, the participants had to attempt the specific problems after a one-time live demonstration, which was performed by the expert coach without any verbal, video or other feedback. The participants were instructed to rest for 30 seconds between each of the three attempts at the same problem with a 10-minute rest between each block of repetitions.

Phase 2 (intervention phase): During this phase, the participants in the control group received standardized verbal feedback following every attempt. This feedback consisted of clear pre-prepared standardized statements which aimed to identify key mistakes that the participants made during the attempt and provide advice on how to correct these mistakes in the subsequent attempt. In addition to the standardized verbal feedback, the experimental group participants were upon each repetition provided with video analysis and expert modelling that was conducted together with the coach. Both, video analysis and expert modelling were conducted using a tablet device with the Dartfish Express (www.dartfish.com version 7.5.11229), a video application that has previously been used in sports performance and sports medicine research and enables video recording and analysis as well as expert video modelling via side-by-side playback of different videos at modified speeds.^18,29,30 All participants were familiarized with the whole process of video analysis and expert modelling using the employed Dartfish Express setup before the first experimental session by the expert coach. The experimental group participants first watched a muted video of their climbing on a tablet device (e.g. video analysis) and subsequently watched the same muted video side-by-side with a video of an expert climbing in the same problem (e.g. expert modelling). The first expert modelling video was played at normal speed while during the second viewing the video was played in slow motion (4x slow down) with the coach pausing the video recordings at the key moments where movement errors/inefficiencies were identified to provide additional instruction/feedback for the next attempt. Finally, a third viewing of the video was again conducted at normal speed, for participants to revise their initial mistakes in real speed. The videos of an expert climbing on all six problems were preloaded onto the device at the beginning of the study. Each expert video was pre-recorded with the same female expert climber, aged 26 years, height 173 cm, executing each move of the problem with 100% accuracy and fluency. The expert is an active member of the Slovene national climbing team and has been involved in the sport for 19 years achieving excellent results in national and international climbing competitions (e.g. 5th place in the bouldering senior world cup). Each participant's attempt was recorded, and movement was synchronized with the experts’ climbing videos within the application to enable direct side-by-side comparisons of the expert's and participants’ movement execution during each repetition.

Measurements and materials

Basic anthropometric measures and indexes (body height, body mass, arm span to height ratio – ape index), as well as climbing history/ability, were obtained before Phase 1. All sessions were continuously recorded using a Sandberg USB face-recognition webcam which was set on a tripod at a fixed position during all the sessions.

Expert scores

Two climbing experts with multiple years of climbing and coaching experience rated the execution of each attempt for each session and each participant. A 7-point scoring system (ranging from 0 – very poor execution to 6 – perfect execution) with a detailed definition of each score was employed (cf. supplementary material). As we aimed to assess the isolated (single) movement (either of the three techniques) per se, we opted to use the above custom-designed scoring as opposed to the recently developed tool from Taylor and colleagues³¹ which serves as an excellent tool for climbing movement assessment during prolonged (route) climbing tasks. We explained and discussed in person all the definitions with the two experts, where videos from sessions 3 and 4 were used to show examples of each score for the three techniques. The two experts then scored 48 attempts (16 attempts of each skill) selected at random from the second session. The inter-rater reliability, as measured by the intraclass correlation coefficient for consistency (ICC (3.2)), was good (ICC = 0.84 95% confidence interval (CI) [0.61, 0.93], 0.83 95% CI [0.58, 0.93], 0.94 95% CI [0.86, 0.98] for high step, drop knee and heel hook, respectively). We then discussed the reasons for the differences in scores with the two experts and gave further explanations of the definitions where necessary. Both experts then independently scored the execution of each move during the first and the final session using the proposed scoring method. The experts were blinded in terms of group designation and the number of respective sessions (the attempts were given to them in random order). The subsequent inter-rater reliability was excellent (ICC = 0.96 95% CI [0.95, 0.98], 0.90 95% CI [0.85, 0.93], 0.95 95% CI [0.92, 0.97] for high step, drop knee and heel hook, respectively).

Perceived difficulty

Immediately after each attempt, and before receiving feedback, the participants were asked to give their subjective opinion about the difficulty of the climb on a scale from 1 to 5 (1 – very easy and 5 – very difficult).

Statistical analysis

The sample size was determined based on the assumption that the difference between the post intervention and the pre testing measurement in the experimental group will exceed the difference in the control group on average by 5, and assuming that the standard deviations at baseline and the end of the study, assumed to be equal in both groups, are 4 and 3, respectively. With the correlation between the pairs of measurements equal to 0.2, 14 participants per group were determined to be necessary to obtain 80% power, using the two-sided alpha equal to 0.05. To account for potential drop-out, 15 participants per group were enrolled in the study. The power of the achieved sample size (13 per group) is 78% under the same assumptions as reported above. Note that our assumptions roughly equate to the effect size (Cohen's d) of about 1.

Data are reported as mean (standard deviation – SD) or median (interquartile range – IQR) for normally and non-normally (and ordinal) distributed continuous variables, respectively. The normality assumption was verified with the Shapiro-Wilk test. For categorical variables, we report the frequency (and percentage, %). The differences between the groups at baseline were tested with an independent samples t-test, Mann-Whitney test or chi-squared test with continuity correction as appropriate. The analysis was performed using R (version 3.5.3).³² A p-value of equal or less than 0.05 was considered statistically significant. All reported p-values and confidence intervals are two-sided.

The expert scores for the three repetitions of each problem were summed to obtain a score on a scale ranging from zero to 18 with larger values exhibiting better execution of the skill. Inter-rater reliability (assumed fixed) was estimated by calculating ICC (3.2) using the R package psych. The scores of the two experts were then averaged for the rest of the analysis.

Linear mixed-effects models, considering expert scores and perceived difficulty for each skill as dependent variables in separate models, were used to estimate the changes between the post intervention and pre testing measurements, i.e. the causal total (intent-to-treat) effect of the intervention (which we define as the addition of video analysis and expert video modelling) on the expert scores and the causal total effect of the treatment on the perceived difficulty. The models contained group (experimental, control), side (left, right), time (pre intervention testing, post intervention testing), and group and time two-way interaction as fixed effects. Random intercept by participants ID was included to account for multiple measurements within the same individual. The square root transformation was applied for the expert scores to stabilize the variance and assure that the functional form of the model was correctly specified, which was verified using the methodology proposed by Peterlin et al.³³ (we performed a sensitivity analysis using the non-transformed outcomes reaching to the same conclusions, however, the relevant goodness-of-fit statistics were slightly better when using the square root transformation, results not shown). The model estimated effect sizes (Cohen's d) and adjusted partial eta-squared (aka epsilon squared) with their respective confidence intervals are reported in Supplementary Tables 3.1 (expert scores) and 3.2 (perceived difficulty). It was specified a priori, that only the following three contrasts will be considered: pre testing vs. post intervention for the control group, pre testing vs. post intervention for the experimental group and the difference between pre testing and post intervention for the experimental vs. the control group. To evaluate the magnitude of the observed effects, we calculated the appropriate confidence intervals for the observed effects and compared them with the standard error of the measurement (SEM) that was estimated as proposed by Weir³⁴; the relevant SEM (based on the square root transformed data) was 0.22, 0.32 and 0.26, for high step, drop knee and heel hook, respectively. The analysis was performed with R package lme4. The contrasts with their respective confidence intervals were obtained by using the R package glht. Effect size measures were obtained with the R package effect size.

We used mediation analysis³⁵ to disentangle the total causal effect of the intervention on the outcome (expert scores) by assuming that the perceived difficulty (the mediator) is ignorable given the observed intervention and pre intervention confounders, which is a standard (untestable) assumption in this analysis³⁶ (note that the strong ignorability of the treatment assignment (in our case the intervention) is in our case fulfilled due to randomization). We used model-based inference, fitting separate models for each skill. The mediator model included perceived difficulty as the dependent variable and group, time side, age, and group and time two-way interaction as fixed effects and random intercept by participant ID as a random effect. The outcome model included expert score (using square root transformation) as the dependent variable and the same fixed and random effects as the mediator model with an addition of perceived difficulty and time and perceived difficulty two-way interactions. The correct specification of the design matrices in both models was checked using the previously mentioned methodology by Peterlin et al.³³ The estimands that were obtained from these analyses were the average causal mediation effects (CME), average causal direct effects (CDEs) and causal total effects (CTE), see³⁷ for more details; beside the point estimates, quasi-Bayesian Monte Carlo simulation confidence intervals are also reported (based on 2000 simulations). A similar analysis was performed to see if the effects vary across age and gender (in separate analyses) by including appropriate higher-level interaction terms with age and gender, respectively in the mediator and outcome model. The mediation analysis was performed using the R package mediation.³⁵

Results

There were no significant differences between the two groups in terms of age, gender, body mass, height, ape index, past climbing experience (duration in months), frequency of climbing in the last half-year (times per week) or 30 days (number of climbing sessions) and the hardest completed climb (Table 1).

Table 1.

Baseline characteristics for the participants that completed the study. The data are mean (SD) for normally distributed variables (Shapiro-Wilk p-value > 0.05), median {interquartile range – IQR} for ordinal variables and non-normally distributed variables (Shapiro-Wilk p-value < 0.05), or frequency [%] as appropriate. Hardest climb is denoted based on the IRCRA reporting scale (Draper et al., 2016).

	All (n = 26)	Control group (n = 13)	Experimental group (n = 13)	p-value
Age [years]	27.3 (4.5)	27.8 (4.4)	27.0 (4.6)	0.61
Student of the faculty of sports [yes]	2 [7.7]	0 [0]	2 [15.4]	0.48
Gender [Male]	9 [34.6]	5 [38.5]	4 [30.8]	1.00
Height [cm]	174.1 (6.7)	174.4 (7.5)	173.8 (6.2)	0.82
Weight [kg]	70.0 (11.5)	71.6 (11.2)	68.3 (11.9)	0.47
Ape index	173.9 (9.4)	173.0 (9.8)	174.8 (9.4)	0.63
Climbing experience [months]	2 {1–4.9}	4 {1–5}	1 {1–2}	0.27
Climbing frequency in the last 6 months [times per week]	1 {0–1}	1 {0.5–1}	1 {0–1}	0.62
Climbing frequency in the last 30 days [days]	2 {1–3}	3 {1–4}	1 {0–3}	0.10
Hardest climb [IRCRA scale]	8 (2)	8 (1)	8 (2)	1.00

Estimation of the total causal effect

The expert scores did not differ significantly between the control and experimental group during pre testing (Table 2). As outlined in Figure 2, the expert scores were significantly higher following the training phase in both groups, but more so in the experimental than control group for the high step (CTE = 0.38, 95% CI [0.06, 0.68], p = 0.017, d = 0.55, note that the point estimate exceeds the relevant SEM) and, albeit not statistically significant, for the drop knee (CTE = 0.12, 95% CI [−0.24, 0.48], p = 0.52, d = 0.15), but not for the heel hook where the improvement was similar in both groups (CTE = −0.05, 95% CI [−0.42, 0.31], p = 0.75, d = 0.07).

Figure 2.

The difference between pre testing and post intervention for the test scores (square root transformation) and perceived difficulty in the control and the experimental group. The estimates are from the model and are adjusted for the side (left, right). The horizontal lines represent the lower and upper bounds of the 95% confidence interval.

Table 2.

Expert scores (square root transformation) and perceived difficulty at pre testing and post intervention. Data are mean (SD). Differences between post intervention and pre testing and the corresponding 95% CI and p-values are from the model and are adjusted for side (left, right).

	Control (n = 13)			Experimental (n = 13)			Experimental vs. Control
	pre testing	post intervention	post vs pre testing [95% CI; p-value] (Cohen d)	pre testing	post intervention	post vs pre testing [95% CI; p-value] (Cohen d)	post vs pre testing [95% CI; p-value] (Cohen d)
Expert scores
Drop knee	2.57 (0.83)	3.20 (0.87)	0.64 [0.38, 0.90; <0.001] (1.13)	2.51 (0.75)	3.27 (0.67)	0.75 [0.49, 1.01; <0.001] (1.33)	0.12 [−0.24, 0.48; 0.52] (0.15)
Heel hook	2.16 (1.04)	2.83 (0.91)	0.67 [0.40, 0.93; <0.001] (1.17)	2.25 (1.20)	2.86 (0.95)	0.61 [0.34, 0.87; <0.001] (1.06)	−0.05 [−0.42, 0.31; 0.75] (0.07)
High step	2.30 (0.79)	2.76 (0.86)	0.46 [0.24, 0.68; <0.001] (0.95)	2.17 (0.80)	3.00 (0.80)	0.83 [0.61, 1.06; <0.001] (1.73)	0.38 [0.06, 0.68; 0.017] (0.55)
Perceived difficulty
Drop knee	3.64 (1.18)	2.30 (1.23)	−1.34 [−1.68, −1.00; <0.001] (1.82)	3.13 (1.13)	2.36 (0.90)	−0.77 [−1.11, −0.44; <0.001] (1.05)	0.56 [0.09, 1.03; 0.019] (0.54)
Heel hook	4.59 (0.72)	3.37 (1.25)	−1.22 [−1.58, −0.86; <0.001] (1.56)	4.41 (0.81)	3.57 (1.25)	−0.84 [−1.20, −0.47; <0.001] (1.07)	0.38 [−0.12, 0.89; 0.13] (0.35)
High step	3.99 (0.89)	3.13 (1.26)	−0.86 [−1.24, −0.48; <0.001] (1.04)	3.85 (1.02)	2.94 (1.06)	−0.92 [−1.29, −0.54; <0.001] (1.11)	−0.06 [−0.58, 0.47; 0.83] (0.05)

The participants in both groups perceived that the climbs got significantly easier, more so in the control group for the drop knee (CTE = 0.56, 95% CI [0.09, 1.04], p = 0.019, d = 0.54) but similarly for the high step (CTE = −0.06, 95% CI [−0.59, 0.47], p = 0.83, d = 0.05) and the heel hook (CTE = 0.38, 95% CI [−0.12, 0.89], p = 0.13, d = 0.35; Table 2, Figure 2 and supplementary material).

Mediation analysis

For the high step, there was practically no causal mediator effect (CME = 0.01, 95% CI [−0.09, 0.12]), but a moderate and statistically significant positive causal direct effect (CDE = 0.37, 95% CI [0.07, 0.66]). For the drop knee, there was a statistically significant negative causal mediator effect (CME = −0.26, 95% CI −0.51 – −0.04), and a strong positive and statistically significant causal direct effect (CDE = 0.38, 95% CI [0.07, 0.68]) with the point estimate exceeding the relevant SEM. For the heel hook, there were no substantial causal mediator or CDEs (Table 3).

Table 3.

Results of the mediation analysis for each skill. CME: average causal mediation effect; CDE: average causal direct effect; point estimates, 95% confidence intervals (CI) and p-values.

	CME [95% CI] (p-value)	CDE [95% CI] (p-value)
High step	0.01 [−0.09, 0.12] (0.844)	0.37 [0.07, 0.66] (0.011)
Drop knee	−0.26 [−0.51, −0.04] (0.022)	0.38 [0.07, 0.68] (0.013)
Heel hook	−0.08 [−0.22, 0.02] (0.130)	0.02 [−0.35, 0.38] (0.930)

Understanding the results of the two causal analyses

Although exploratory in nature, the below results explain the causal indirect path (or a lack thereof) from the intervention to the expert score (outcome) via the perceived difficulty (mediator). Table 4 outlines the summary of the causal total models for expert scores (CTM), outcome models (OM) and mediation models (MM), which were used to conduct mediation analysis; for the brevity of the presentation, we only report the coefficient estimates that are relevant for explaining the main results presented previously, with complete results presented in the supplementary material.

Table 4.

Point estimates of the regression coefficients (implying only association and not causality), 95% confidence intervals (shown only for exploratory purposes), and effect sizes (Cohen's d) for the causal total models for expert scores (CTM), outcome model (OM) and mediation models (MM); some estimated coefficients (intercept, side, group and in some models (OM and MM) age) are omitted for brevity, see supplementary information for complete results. T: time (reference category: pre testing); G: group (reference category: control group); PD: perceived difficulty. The results with d > 0.5 are shown in bold for easier reference.

	CTM^a	OM^a	MM^b
	Estimate [95% CI] (d)	Estimate [95% CI] (d)	Estimate [95% CI] (d)
High step
T	0.46 [0.24, 0.68] (0.95)	−0.13 [−0.75, 0.49] (0.09)	−0.86 [−1.23, −0.49] (1.04)
G*T	0.38 [0.06, 0.68] (0.55)	0.38 [0.09, 0.67] (0.59)	−0.06 [−0.58, 0.47] (0.05)
PD		−0.30 [−0.45, −0.15] (0.81)
PD*T		0.11 [−0.05, 0.26] (0.29)
Drop knee
T	0.64 [0.37, 0.90] (1.13)	−0.02 [−0.50, 0.47] (0.02)	−1.34 [−1.67, −1.01] (1.82)
G*T	0.12 [−0.24, 0.48] (0.15)	0.38 [0.08, 0.68] (0.56)	0.56 [0.10, 1.03] (0.54)
PD		−0.46 [−0.59, −0.34] (1.43)
PD*T		0.02 [−0.13, 0.16] (0.05)
Heel hook
T	0.67 [0.40, 0.93] (1.17)	−1.02 [−2.01, −0.05] (0.45)	−1.22 [−1.58, −0.87] (1.56)
G*T	−0.05 [−0.42, 0.31] (0.07)	0.06 [−0.28, 0.41] (0.08)	0.38 [−0.11, 0.88] (0.35)
PD		−0.49 [−0.72, −0.26] (0.87)
PD*T		0.32 [0.11, 0.54] (0.67)

The dependent variable was the expert score.

The dependent variable was perceived difficulty.

As implied by the outcome models, larger perceived difficulty was associated with lower expert scores for all three skills. In two skills, high step and drop knee, this effect was similar during the pre testing and during post intervention, however for the heel hook, this negative association was present only during pre testing and not during the post intervention. For this reason, even though we observe some association between the intervention and the perceived difficulty as suggested by the mediation model, the perceived difficulty does not seem to be an important mediator for this skill. For the high step, no association between the intervention and the mediator was noted. Therefore, even though the participants with a higher perceived difficulty would be expected to get lower expert scores, this was not important since changes in the perceived difficulty were similar in both groups. This was however not the case for the drop knee where we did establish the causal path between the intervention and the perceived difficulty (the decrease in the perceived difficulty was larger in the control than in the experimental group) explaining the indirect path.

The role of gender and age

We explored how uniform the effects reported thus far are across various ages and sex (see supplementary material). The results for the high step and heel hook showed that the CDE and CME are similar for different values of age and both genders. For the drop knee, a similar positive CDE was observed for different values of age (in fact this effect was very similar as observed for the high step), however, the negative causal mediator effect was increasing with age: for the youngest participants, this mediator effect was practically negligible (thus observing positive causal total effect for the younger participants), but for the older participants this effect was large, completely diminishing the positive direct effect (thus not observing the causal total effect). Furthermore, while the CDEs were similar for males and females for this skill (and comparable, if not even larger, than those observed for the high step), the negative CME was only observed in males but not in females.

Discussion

This is the first randomized controlled study to evaluate the potential additional effects of video analysis and expert modelling on climbing technique development in novice adult climbers during the expertly coached climbing sessions. We focused on three important and frequently employed climbing techniques: drop knee, heel hook and high step, known to play important roles in modern climbing techniques and can optimize climbing movement and fluency.^9,11 We aimed at estimating the causal total effect of adding video analysis and expert modelling to expertly coached climbing sessions using verbal feedback only (intervention). The primary outcome was the rate of learning as measured by the post intervention and pre testing differences between the expert scores based on a scoring system, which we developed for the present study and was shown the excellent inter-rater reliability. The secondary outcome was the participants’ perceived difficulty when executing the skill. We included this outcome since it could potentially be a mediator in the underlying causal mechanism, henceforth we also refer to it as the mediator, as discussed later. Our study showed a significant causal total effect of the intervention on the expert score only for one of the skills, the high step, while for the other two skills, no significant causal total effect was found. In particular, the participants in the experimental group improved significantly more (in terms of the expert scores) than the participants in the control group for the high step but not for the other two skills.

While many potential sources might explain this, task complexity and task difficulty are probably among the prominent ones. Although the two represent different constructs they are often mistakenly used interchangeably.^38,39 Task complexity represents the characteristics or cognitive demands of a task, while task difficulty refers to a person's subjective judgment of the task complexity.³⁹ Note that in the causal inference setup under the randomized controlled experiment, these two constructs play very distinct roles: while both can affect the effectiveness of the treatment as will be argued later, only task difficulty can be a mediator in the underlying causal mechanism.

Task complexity is believed to influence the effectiveness of information sources.⁴⁰ Magill and Schoenfelder-Zohdi⁴¹ argue that multiple sources of task-related information are redundant for simple tasks because single sources provide adequate information for the development of cognitive representations and overt performance. However, numerous sources of task-related information are likely beneficial for more complex tasks.⁴² Indeed, Laguna⁴² showed that complex tasks benefited more from a combination of model demonstrations and knowledge of performance practice than simple tasks. Unfortunately, as there are no established criteria in climbing to objectively measure the task complexity, we do not have the necessary data to adequately address its role by conducting appropriate analyses. Establishing such criteria for different climbing-related skills could prove highly beneficial for a more in-depth investigation of similar questions in the future (i.e. the currently available grading scales,²⁸ do not, in our opinion, fit this purpose).⁴³ The problem that was set for practising heel hook was probably physically the most difficult, requiring the most strength, especially core strength. Therefore, (a lack of) strength was an important limiting factor for measurable improvement in this technique. Since the two groups followed the same training protocol and therefore had a similar training load across the intervention, physical improvements were likely similar between groups which explain the lack of observing a causal total or direct effect (more on that later) for this skill. While the other two techniques might have different physical/technical underpinnings (dynamic power, flexibility, etc.), we believe that the reasons for observing divergent results for the two skills were not primarily due to the task complexity but more related to the task difficulty.

The relationship between task difficulty and the outcome of the learning process has only recently been established in the literature.³⁸ In particular, Nawaz and colleagues³⁸ showed that if students perceive the task difficulty as easy or hard, it may lead to poorer learning outcomes, while medium or moderate task difficulty may result in better learning outcomes. The U-shaped relationship between motor learning and task difficulty has also been observed⁴⁴ indicating that too challenging and/or too easy tasks impair motor learning trajectory. Based on our climbing and coaching experience, we observed that climbers, especially beginners, often struggle to improve (in a particular route, a boulder problem or in learning a new skill) simply because they believe that something is too difficult. We measured the task difficulty by asking the participants about their opinion on how difficult it felt for them to execute a certain problem (the perceived difficulty), which allowed us to formally evaluate its role in the underlying causal mechanism. While our study was not designed to establish a causal effect of the task difficulty on the outcome, there are some interesting clues about this in our data (albeit only on the level of association). Namely, the results of our study suggest that for the high step and drop knee, there is a negative association between the perceived difficulty and the outcome, which is in line with previously discussed findings of Nawaz³⁸ and Akizuki⁴⁴ et al. For the heel hook, we only observed this association during pre testing but not post intervention. These results are important since they suggest that task difficulty could be a potential mediator for the drop knee and high step but not so for the heel hook. Our results indicate a significant causal total effect of the intervention on the task difficulty for the drop knee. In particular, the reduction in perceived difficulty during post intervention and pre testing was greater in the control than in the experimental group for this skill. This was observed to some extent also for the heel hook (although the effect was much smaller than observed for the drop knee and was statistically insignificant) but not for the high step. The link between the expert modelling and the participants’ belief about their ability to perform the modelled task at a certain level seems well established.^45,46 However, there is a lack of research addressing the direct link between expert modelling and task difficulty. Despite our exploratory effort in this direction further, well-designed studies are warranted to address this.

The conducted mediation analysis, formally addressing various links between the intervention and the outcome, either directly or indirectly via the perceived difficulty, showed that there is an important CME (reducing the efficacy of the treatment) but also an important positive CDE for the drop knee, which might explain the lack of causal total effect for this skill. For the other two skills, no important CME was observed, hence the causal total and direct effects were similar. Our results suggest that for the high step and the heel hook the effect of the intervention does not substantially depend on age and gender. For the drop knee, however, the negative causal mediator effect was only observed for males and not for females and it was increasing with age (for the youngest participants, this mediator effect was practically negligible, but for the older participants this effect was large, completely diminishing the positive direct effect). Hence, these results suggest, that while the CDE of the intervention for the drop knee is substantial for all ages and both genders, for males and older participants this positive effect is completely diminished by the negative effect that the intervention had on their perceived difficulty and subsequently on the learning trajectory. One potential explanation for observing divergent CMEs for males and females as well as for older participants could be related to the fact that we have used a young female expert for the expert model. Namely, according to the model-observer similarity hypothesis⁴⁶ and similarity-attraction hypothesis⁴⁷ the effectiveness of modelling is at least partly moderated by the degree to which observers perceive a model to be similar to them, which especially holds for participants, whose prior knowledge as well as self-efficacy and perceived competence are still low (e.g. beginners) since they are assumed to be particularly affected by model-observer similarity.⁴⁸ Of the various characteristics of the model, gender is presumed to be the most important, since it is the first thing that is noticed.⁴⁹ Hoogerheide et al.⁵⁰ observed that the gender of both model and observer can matter in terms of affective variables experienced during learning which was also observed in our study. A potential factor explaining why this mediation effect was observed for the drop knee and not for the high step could be related to the differences in the skillsets required to correctly perform the two skills. The problem that was set for practising the high step required the largest amount of coordination and explosive power: the problem was set in such a way that a lot of force had to be quickly generated from the legs while propelling the force in the right direction to move the body towards the final hold in one quick, complex dynamic movement. In contrast, the problem that was set for practising the drop knee required more static body control, where the movement and the power originated from the hips, also requiring significant hip and ankle flexibility. It might be, that it was difficult for the male participants (who are in general less flexible and prefer initiating the movement from their arms) to relate to our female model when executing the drop knee, while this was probably less problematic for the high step since when performing a dynamic movement, they could better relate to the model. While this remains speculative as our study was not designed to directly address this issue future work should explore the potential effects of the expert model's age and gender on the CME.

Collectively, the above-discussed analyses suggest that (1) there is a positive CDE of the intervention for two skills (high step and drop knee) which does not strongly depend on age and sex, however (2) for the drop knee, there is also a negative causal mediator effect observed for males (thus resulting in a lack of causal total effect of the intervention in this group of participants) but not for females (thus observing a positive causal total effect for this group of participants) and the other two skills independent of age and sex (thus observing a significant causal total effect for the high step independent of age and sex). It is important to note that the significant effects discussed thus far exceeded the standard error of the measurement (SEM), but were not extremely large: the relevant standardized effect size measures (Cohen's d) were about 0.5 but are similar in size to other comparable studies.⁵⁰ On the other hand, very large improvements were observed in the accuracy of the execution of each skill in both groups following the intervention. In this case, the relevant standardized effect size measures were larger than 1 (cf. supplementary material). This resulted in large improvements even in the control group, potentially leaving less scope for further improvements with the addition of video analysis and expert modelling. However, as the study was not designed to study the effect of each intervention separately (but only relative to each other), we obviously cannot attribute all this effect to the two interventions. Namely, previous work¹⁷ indicated that repeating certain climbing movements in novice climbers can quickly lead to improvements in climbing technique/economy even without any external inputs. Designing an experiment which would evaluate the effect of the coached climbing session in comparison with the true control group would undoubtedly be worthwhile in the future. However, defining the true control group could prove challenging since several different definitions of what the true control group actually is are possible and not all of these definitions are equally important in the practical applications.

From an applied angle, the effectiveness of the coached training session depends strongly on the coach, through his or her experience, rhetoric skills and competency in teaching others. In the present study, all sessions were led by a single coach with multiple years of experience in climbing-related training who provided standardized feedback comprising clear pre-prepared statements aimed to ensure only relevant and understandable info was conveyed back to the participants. Using a single highly experienced coach, however, prevents us from extending our findings to others, especially less experienced coaches, where the effect of adding video analysis and expert modelling could result in more (or less) substantial improvements. Furthermore, the results of the present study support the use of video analysis and expert modelling when training novice climbers – at least for some techniques – and in that respect extend preliminary observations by Walker and colleagues,¹⁸ suggesting that such augmented feedback manner can facilitate climbing technique acquisition in novice sport climbers. Although significant benefits were only observed for the high step, these data collectively warrant further investigation of the utility and applicability of these combined approaches in climbing, particularly in more complex and prolonged tasks as well as with more advanced (better-trained) individuals.⁹ Finally, from a coaching perspective, it is important to bear in mind the observed potential negative effect of perceived difficulty and try to influence the trainee's perception in this regard. In particular, the fact that task difficulty can be a mediator when learning some skills in climbing is highly important, as it suggests that the coach needs to take this into account when using video analysis and expert modelling trying to counterbalance its potentially negative effect on the perceived difficulty (and therefore reduced rate of learning the new technique).

Methodological considerations

Whilst this is the first study to date assessing the additive effects of video analysis and expert modelling, to regular coaching, on climbing technique development, there are a few methodological considerations we would like to acknowledge. First, to evaluate the quality of the execution of each skill, we derived a 7-point scoring system that showed excellent inter-rater reliability. A similar approach of attaching expert scores to the performance of some skill has been used in sports science literature across various sports.^18,51 For example, Born and colleagues⁵¹ derived a 6-point scoring system to evaluate the execution of each stroke in tennis, considering the sum over five different attempts of each stroke as an outcome to be used in the analysis. Also, as mentioned previously we opted not to use the recently developed tool from Taylor and colleagues³¹ as we aimed to assess the isolated (single) movements as opposed to the prolonged climbing movement. Second, an important strength of our study is that the experts who rated the climbing accuracy were not aware of the group assignment as well as the session which they were evaluating. This is crucial to assure that their opinions were not biased toward the goals of the study.⁵² This contrasts with the approach used by Walker and colleagues¹⁸ where the experts were not fully blinded, which may have introduced expert bias and thus limited their conclusions. Third, the duration of the learning phase was rather short - three supervised training sessions. While the data indicate that it was sufficient to augment movement execution accuracy and discern some independent positive effects of the tested interventions, a longer training cycle might have resulted in divergent outcomes and should be assessed in future work. Furthermore, the potential effect of within-session variability in the technique execution by the participants on the learning outcomes also needs to be considered although, based on the conducted sensitivity analysis, showing no important within-session differences and/or variability, this did not seem to influence the outcomes of the present study. Fourth, while these data apply to novice climbers, the utility of video analysis and expert modelling for technique refinement and optimization in more experienced climbers remains to be established. It is also of note that, we used a highly experienced coach with multiple years of experience, so our results in terms of the overall improvements observed for both groups cannot be generalized to other coaches, especially those with less experience. Fifth, in our analysis, the participants were recruited from two sources: the attendees of the commercial climbing centre and the students of the Faculty of Sports. While it might be possible that the rate of learning would differ for the two groups, the lack of data about the latter group (only 2 participants) prevents us from studying this in more detail. However, the results based on a subset analysis excluding the two Faculty of Sports students, led to the same conclusions as the complete data analysis, suggesting that this was not a major factor. The fact that only one female expert model was employed also needs to be considered when interpreting the present findings. Finally, as noted earlier, the lack of a second “true” control group with no coaching intervention and no verbal feedback, prevents us from attributing this improvement solely to the role of the coach but goes in line with the previous findings.¹⁷

Conclusions

In summary, the present study suggests the addition of video analysis and expert modelling to coached training sessions might facilitate the acquisition of certain climbing techniques, such as high step, in novice climbers and thus lead to faster and safer progress in climbing ability in this population. However, the present results also provide evidence that the potential benefit of such augmented feedback intervention can be hindered by the mediation effect of the perceived difficulty. Further work is therefore needed to better understand the role of the perceived difficulty as a mediator in the causal mechanisms and to fully elucidate the utility of video analysis and expert modelling for climbing movement optimization in more complex and prolonged tasks as well as for better-trained individuals.

Supplemental Material

sj-docx-1-spo-10.1177_17479541231152548 - Supplemental material for Utility of video analysis and expert modelling for technique development in novice sport climbers: A randomized controlled study

Supplemental material, sj-docx-1-spo-10.1177_17479541231152548 for Utility of video analysis and expert modelling for technique development in novice sport climbers: A randomized controlled study by Rok Blagus, Bojan Leskošek, Luka Okršlar, Nace Vreček and Tadej Debevec in International Journal of Sports Science & Coaching

Footnotes

Acknowledgements

The study was co-funded by the Slovenian Research Agency grant (N5-0152 and P3-0154). The authors would like to thank the dedicated participants without whom the present work would not have been possible. We would also like to acknowledge dr Edwin Avila for his assistance during the measurements and Mr Benjamin J. Narang for his valuable feedback on the initial version of the manuscript.

Conflict of interest

The authors declare no conflict of interest relevant to the present work.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Slovenian Research Agency, Slovenian Reseacrh Agency, (grant number P3-0154, N5-0152)

ORCID iD

Tadej Debevec

Supplemental material

Supplemental material for this article is available online.

References

Valenzuela

de la Villa

Ferragut

. Effect of two types of active recovery on fatigue and climbing performance. J Sports Sci Med 2015; 14: 769–775.

Lutter

Tischer

Schoffl

. Olympic Competition climbing: the beginning of a new era-a narrative review. Br J Sports Med 2021; 55: 857–864.

Sheel

. Physiology of sport rock climbing. Br J Sports Med 2004; 38: 355–359.

Giles

Rhodes

Taunton

. The physiology of rock climbing. Sports Med 2006; 36: 529–545.

Bertuzzi

Franchini

Kokubun

, et al. Energy system contributions in indoor rock climbing. Eur J Appl Physiol 2007; 101: 293–300.

Phillips

Sassaman

Smoliga

. Optimizing rock climbing performance through sport-specific strength and conditioning. Strength Cond J 2012; 34: 1–18.

Giles

Barnes

Taylor

, et al. Anthropometry and performance characteristics of recreational advanced to elite female rock climbers. J Sports Sci 2021; 39: 48–56.

Watts

. Physiology of difficult rock climbing. Eur J Appl Physiol 2004; 91: 361–372.

Orth

Davids

Seifert

. Coordination in climbing: effect of skill, practice and constraints manipulation. Sports Med 2016; 46: 255–268.

10.

Seifert

Hacques

Rivet

, et al. Assessment of fluency dynamics in climbing. Sports Biomech 2020; 29: 1–12.

11.

Lutter

Tischer

Cooper

, et al. Mechanisms of acute knee injuries in bouldering and rock climbing athletes. Am J Sports Med 2020; 48: 730–738.

12.

Schoffl

Lutter

Popp

. The "heel hook" – a climbing-specific technique to injure the leg. Wilderness Environ Med 2016; 27: 294–301.

13.

Grønhaug

. Self-reported chronic injuries in climbing: who gets injured when? BMJ Open Sport Exerc Med 2018; 4: e000406.

14.

Haas

Meyers

. Rock climbing injuries. Sports Med 1995; 20: 199–205.

15.

Jones

Asghar

Llewellyn

. The epidemiology of rock-climbing injuries. Br J Sports Med 2008; 42: 773–778.

16.

Ashy

Lee

Landin

. Relationship of practice using correct technique to achievement in a motor skill. J Teach Phys Educ 1988; 7: 115–120.

17.

Cordier

France

Bolon

, et al. Entropy, degrees of freedom, and free climbing - a thermodynamic study of a complex behavior-based on trajectory analysis. Int J Sport Psychol 1993; 24: 370–378.

18.

Walker

Mattson

Sellers

. Increasing accuracy of rock-climbing techniques in novice athletes using expert modeling and video feedback. J Appl Behav Anal 2020; 53: 2260–2270.

19.

Schenk

Miltenberger

. A review of behavioral interventions to enhance sports performance. Behav Interv 2019; 34: 248–279.

20.

Frikha

Chaari

Elghoul

, et al. Effects of combined versus singular verbal or haptic feedback on acquisition, retention, difficulty, and competence perceptions in motor learning. Percept Mot Skills 2019; 126: 713–732.

21.

Zetou

Tzetzis

Vernadakis

, et al. Modeling in learning two volleyball skills. Percept Mot Skills 2002; 94: 1131–1142.

22.

Hebert

Landin

. Effects of a learning model and augmented feedback on tennis skill acquisition. Res Q Exerc Sport 1994; 65: 250–257.

23.

Giannousi

Mountaki

Kioumourtzoglou

. The effects of verbal and visual feedback on performance and learning freestyle swimming in novice swimmers. Kinesiology (Zagreb) 2017; 49: 65–73.

24.

Mononen

Viitasalo

Konttinen

, et al. The effects of augmented kinematic feedback on motor skill learning in rifle shooting. J Sports Sci 2003; 21: 867–876.

25.

Petancevski

Inns

Fransen

, et al. The effect of augmented feedback on the performance and learning of gross motor and sport-specific skills: a systematic review. Psychol Sport Exerc 2022; 63: 102277.

26.

Boyer

Miltenberger

Batsche

, et al. Video modeling by experts with video feedback to enhance gymnastics skills. J Appl Behav Anal 2009; 42: 855–860.

27.

Fleming

Hörst

. Behavior analysis and sports climbing. J Behav Health Serv Res 2010; 1: 143–154.

28.

Draper

Giles

Schöffl

, et al. Comparative grading scales, statistical analyses, climber descriptors and ability grouping: international rock climbing research association position statement. Sports Technol 2015; 8: 88–94.

29.

Khadilkar

MacDermid

Sinden

, et al. An analysis of functional shoulder movements during task performance using Dartfish movement analysis software. Int J Shoulder Surg 2014; 8: 1–9.

30.

Vanlandewijck

Verellen

Tweedy

. Towards evidence-based classification in wheelchair sports: impact of seating position on wheelchair acceleration. J Sports Sci 2011; 29: 1089–1096.

31.

Taylor

Giles

Panackova

, et al. A novel tool for the assessment of sport climbers’ movement performance. Int J Sport Physiol 2020; 15: 795–800.

32.

R Developement Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, 2014.

33.

Peterlin

Kejžar

Blagus

. Correct specification of design matrices in linear mixed effects models: tests with graphical representation. TEST 2022. DOI: 10.1007/s11749-022-00830-1

34.

Weir

. Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM. J Strength Cond Res 2005; 19: 231–240.

35.

Imai

Keele

Tingley

, et al. Causal mediation analysis using R. In: Vinod

(ed) Advances in social science research using R, lecture notes in statistics. New York: Springer-Verlag, 2010, pp.129–154.

36.

Imai

Keele

Yamamoto

. Identification, inference and sensitivity analysis for causal mediation effects. Stat Sci 2010; 25: 51–71.

37.

Pearl

. Direct and indirect effects. In: Proceedings of the seventeenth conference on uncertainty in artificial intelligence. Seattle, WA: Morgan Kaufmann Publishers, 2001. pp.411–420.

38.

Nawaz

Srivastava

, et al. Analysis of task difficulty sequences in a simulation-based POE environment. In: International conference on artificial intelligence in education. Springer, 2020, pp.423–436.

39.

Robinson

. Task complexity, cognitive resources and syllabus design: a triadic framework for examining task influences on SLA. New York: Cambridge University Press, 2001.

40.

Magill

. Modeling and verbal feedback influences on skill learning. Int J Sport Psychol 1993; 24: 358–369.

41.

Magill

SchoenfelderZohdi

. A visual model and knowledge of performance as sources of information for learning a rhythmic gymnastics skill. Int J Sport Psychol 1996; 27: 7–22.

42.

Laguna

. Task complexity and sources of task-related information during the observational learning process. J Sports Sci 2008; 26: 1097–1113.

43.

Leskošek

Blagus

. The reliability of the difficulty grading of sport climbing routes. Kinesiol Slov 2021; 27: 5–20.

44.

Akizuki

Ohashi

. Measurement of functional task difficulty during motor learning: what level of difficulty corresponds to the optimal challenge point? Hum Mov Sci 2015; 43: 107–117.

45.

Bandura

. Self-efficacy: The exercise of control. New York, NY: W H Freeman/Times Books/ Henry Holt & Co, 1997.

46.

Schunk

. Peer models and children’s behavioral change. Rev Educ Res 1987; 57: 149–174.

47.

Moreno

Flowerday

. Students’ choice of animated pedagogical agents in science learning: a test of the similarity-attraction hypothesis on gender and ethnicity. Contemp Educ Psychol 2006; 31: 186–207.

48.

Buunk

Zurriaga

Gonzalez-Roma

, et al. Engaging in upward and downward comparisons as a determinant of relative deprivation at work: a longitudinal study. J Vocat Behav 2003; 62: 370–388.

49.

Contreras

Banaji

Mitchell

. Multivoxel patterns in fusiform face area differentiate faces by sex and race. PLoS One 2013; 8: e69684.

50.

Hoogerheide

Loyens

SMM

van Gog

. Learning from video modeling examples: does gender matter? Instr Sci 2016; 44: 69–86.

51.

Born

Nguyen

Grambow

, et al. Embedding tennis-specific teaching videos into long-term educational concepts to improve movement learning and technique performances. J Phys Educ Sport 2018; 18: 255–261.

52.

Marcus

Gorman

, et al. Rater bias in a blinded randomized placebo-controlled psychiatry trial. Stat Med 2006; 25: 2762–2770.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

31.22 MB