Construction and validation of the German coach-rating scale to assess achievement-motivated behavior in team sports (AMBIS-T)

Abstract

Achievement motivation has been identified as a relevant criterion to assess athletic talent. In the context of talent selection decisions, the popular use of self-report questionnaires has presented higher risks of social desirability. Consequently, a shift towards more robust methods such as the assessment of achievement-motivated behavior with the coach-rating scale achievement-motivated behavior in individual sports was suggested.¹ The purpose of this study was to adapt and validate this assessment from individual to team sports, involving 48 German-speaking coaches examining the achievement-motivated behaviors of their 527 youth athletes as well as the self-ratings of 250 athletes. The achievement-motivated behavior in team sports was found to display a good fit with the observed data (e.g., CFI = .98; RMSEA = .04) and to measure reliably the factors proactivity, ambition, and commitment with 12 items (test–retest reliability r_tt ≥.75; Cronbach's alpha α ≥ .80 for all scales). With the exceptions of the proactivity scale for interrater reliability, the commitment scale for concurrent criterion validity and the ambition scale for construct validity, psychometric properties were found satisfactory. The use of averaged ratings of two evaluating coaches is recommended to reduce possible bias in the assessment of achievement-motivated behaviors in team sports.

Keywords

Athlete selection commitment goal orientations self-determination talent identification volition

Construction and validation of the German coach rating scale to assess AMBIS-I

Achievement motivation is also known as “striving for excellence”² was found to play a critical role in talent development and athletic success (see reviews/meta-analysis e.g., Ref. 3, 4). As a result, the use of the assessment of achievement motivation in multidimensional talent identification programs spread across different countries in Europe, including Germany and Switzerland.^5–7 Achievement motivation has been an umbrella term for a variety of theories and constructs⁸ and can be assessed either from an inner (self-reports) or an outer perspective (e.g., coach-ratings). The empirical evidence regarding the importance of achievement motivation for predicting athletic success was based largely on explicit motives of the athletes themselves, thus from an inner perspective. According to Conroy et al.,⁹ the theories and constructs frequently examined in the scientific literature are related to achievement motivation,¹⁰ achievement goal orientation,¹¹ and self-determination.¹² Because motives can hardly be assessed via coach-rating,¹³ user-friendly self-report instruments have been designed (e.g., Sport Orientation Questionnaire¹⁴ and Sport Motivation Scale¹⁵). However, Zuber and Conzelmann¹ warned that the use of self-report measures to assess psychological characteristics for talent selection purposes was prone to the considerable methodological issue of social desirability.¹⁶ Because athletes typically strive to be selected for the higher-level group or team, the likelihood of trying to decipher which answers or which characteristic expressions might have a positive influence on their own selection and then temptations to adjust accordingly are high. Thus, the prediction of athletic success in sports could be severely biased if based solely, and even partially, on self-report measures.

To overcome this validity shortcoming, the German coach rating scale achievement-motivated behavior in individual sports (AMBIS-I) was designed and its psychometric properties checked.^1,17 This scale was based on observable achievement-motivated behaviors and thus intended to provide an alternative outer perspective to assess the achievement motivation of athletes independently of their self-reports. Achievement-motivated behavior has been conceptualized as a set of activities performed through an innate drive to reach high-performance standards.² In sport, the achievement-motivated behavior was therefore defined as “self-determined behavior in the context of competitive sports, which aims to achieve competition- or task-oriented goals and which involves a high degree of self-regulation and commitment.”¹⁷ The behaviors identified as achievement-motivated for AMBIS-I were attributed to the three factors proactivity (e.g., an athlete who stays longer after training and continues to train), ambition (e.g., an athlete who sets high goals), and commitment (e.g., an athlete who show a high level of commitment in training when facing adversity).^1,17 Thus, the design of this scale addressed the need to assess psychological characteristics via the lens of observable behavior patterns.^18,19 The AMBIS-I featured adequate psychometric properties to assess achievement-motivated behavior in individual sports for the population studied (e.g., CFI = 1.00; root mean square error of approximation (RMSEA) = .001; test–retest reliability r_tt >.70 for all three scales; differentiation with small to medium effects (0.30 < d < 0.62) between two performance groups as a measure for concurrent criterion validity)¹ and was found to predict performance levels of youth rowers longitudinally over a period of 30 months.²⁰

In contrast to the AMBIS-I target group, the recent scientific literature in talent research was predominantly focused on team sports, soccer in particular²¹ and not on individual sports. Additionally, a plethora of research studies in the field of predictive psychological and motivational characteristics have been conducted in team sports (see review in¹⁷). To date, the assessment of achievement-motivated behavior via observation has remained unexplored in team sports, but several compelling reasons suggest considerable benefits in using it. First, in individual sports, performance indicators, including rankings or personal records, play a substantial role in selection decisions. However, these are often not available in team sports.²² It can be assumed that assessments completed by coaches in team sports might be more influential in the selection decision of individual athletes, which makes essential the need for a robust measurement. Second, resource-related reasons could also play a role in the use of the AMBIS in comparison to other methods (e.g., interviews) to assess achievement motivation in team sports. There is empirical evidence showing increased burnout and stress levels of coaches in team sports, which are explained by the fact that they are also under increased pressure due to the longer playing season compared to coaches in individual sports.²³ This high pressure experienced by coaches in team sports led to the recommendation for the use of a brief instrument to be used whenever and wherever needed.

However, the direct application of AMBIS-I to team sports does not appear to be useful, as semantic modifications are required as the terminology used in individual and team sports are different (e.g., game vs. competition; nearly no draws in individual sports). For example, the item “He/she has acted annoyed when he/she did not finish the competition in first place” suggests that in certain individual sports, such as track and field, athletes have numerous opportunities per season to finish competition in the first place. In contrast, in team sports, such as soccer or rugby, athletes compete for an entire season and only one team per season wins a championship. As a second example, the item “He/she showed that he/she is not satisfied with coming in 2^nd place” is suitable in individual sports, but in team sports it is more common to talk about a defeat or a draw in a single game.

Additionally, research suggests that different motivational processes are at stake in the team and individual sports: For example, Hollembeak and Amorose²⁴ found higher levels of intrinsic motivation of athletes in individual than in team sports. Further validation of the instrument in team sports is warranted to test for additional criterion validity, as it seems essential to ensure that achievement-motivated behavior in a team sport is relevant for performance.

The present research

The purpose of the current study was to adapt the German AMBIS-I to team sports (AMBIS-T) and to assess the psychometric properties. A longitudinal study was deemed to provide the foundation to examine (a) factorial validity, (b) reliability, (c) concurrent criterion validity, and (d) construct validity of the instrument. As a positive correlation was observed between achievement-motivated behavior and athletic performance in an individual sport,¹⁷ a similar trend was assumed for criterion validity in team sports. Based on the supported construct validity of the AMBIS-I,¹⁷ it was hypothesized that the three factors proactivity, ambition, and commitment would present comparable relationships to self-rating scales assessing achievement goal orientations,¹⁴ self-determination,¹² and volition.²⁵ The first scale proactivity “… refers to getting involved in training processes on one's own initiative and for one's own sake”¹ and has shown to be related to self-reports measuring achievement goal orientation, self-determination, and volition.¹⁷ The second factor ambition “…is characterized by the absolute will to successfully pursue self-imposed goals in competitions”¹ and showed positive correlations to achievement goal orientations.¹⁷ The third factor commitment refers to readiness and willingness to perform¹ and displayed positive correlations to achievement goal orientations and volition.¹⁷ In summary, the convergent validity of the scales of AMBIS-I was hypothesized with self-determination, achievement goal orientations, and volition.

Methods

Adaptation of the items of AMBIS-I for team sports and preliminary analyses

As a first step for the adaptation of AMBIS-I to team sports, three sport scientists who were also expert coaches with high national education levels (professional training for elite sports or competitive sports) in team sports (soccer, handball, and floorball) were asked to evaluate thoroughly the extent to which the content and wording of all 10 items of the AMBIS-I can be used in team sports. The content of the items was considered relevant to team sports, with only minor wording adjustments needed to adequately address the context and terminology of team sports as discussed earlier in the introduction. Therefore, it was decided to keep the already known three factors and refrain from evaluating other factors. This also presents the advantage to maintain a certain degree of comparability between the two versions. Accordingly, in the second step, the items were adjusted to the context and terminology of team sports (e.g., “He/she has acted annoyed when he/she did not finish the competition in first place” was adapted to “He/she was annoyed when the team lost a game”).¹ Furthermore, four additional items were included in the scales ambition (one additional item) and commitment (three additional items; e.g., “He/she demonstrated high commitment during training sessions”). Indeed, these two scales were somewhat underrepresented with three (and not four) items in the AMBIS-I and because the scale commitment of the AMBIS-I seemed to be weakened by a restricted variance and a ceiling effect.¹ These four items originated from the act nomination phase (study 1) of the process of the construction of AMBIS-I and were generated by coaches at the first or second level of education (professional training for elite sports or competitive sports).¹ In step three, the same three experts reviewed the items again for their relevance and adequacy in the context of team sports. Based on these evaluations, only minor changes in wording were needed. As a result of the preliminary analysis, a 14-item questionnaire was designed and included in the main study to evaluate the quality of the coach-rating scale.

Participants and procedure

Formal ethical approval² was granted by the institutional review board of the lead author before conducting the study and written informed consent to participate was obtained from all participants and from the parents of underaged athletes. As the AMBIS-I was, at the time of this study, only available in German and as the instrument was intended to be used in talent identification programs for youth athletes, only German-speaking coaches and their athletes between 10 and 20 years of age were included. They were competing in three nationally popular and internationally competitive team sports in Switzerland (soccer, ice hockey, and floorball; see Figure 1 for an overview of the participants’ flow). The sample of coaches was recruited directly through sports clubs or via sport federations. First, the list already available from the construction of AMBIS-I¹ was used. Directors of the sport federations categorized by Swiss Olympic in the categories 1 to 3 (of 5), according to their national importance and achievement potential, provided the contact information of all their coaches at the first or second level of education (professional training for elite sports or competitive sports). As a second option, the professional network of the author was used to recruit coaches. Upon participation consent, they received a secured link to an internet-based questionnaire (LimeSurvey, Version 2.50) to rate each athlete they were coaching at that time, according to the lists provided by the coaches. A total of 48 coaches rated the achievement-motivated behaviors of their 527 athletes aged between 10 and 20. In the preliminary analysis, the assessments of two coaches were excluded from the study, as they indicated knowing their athletes for less than half a year and that did not feel certain about their ratings. As a result, the 866 assessments of 518 athletes (several athletes were rated by two different coaches; floorball: 57.0%; soccer: 37.6%; ice hockey: 5.4%) were performed by 46 coaches (11 women, 23.4%; 35 men, 76.1%) (M_age = 33.96 years, SD_age = 9.83) were included in the data analysis. They reported knowing their rated athletes for 2.14 years on average (SD = 1.77). Only 6% of coaches had not completed any sport-specific education (mainly assistant coaches in soccer), whereas 15% had completed a professional coaching education, which is the highest standard in Switzerland. On average, coaches had 9.42 years (SD = 6.40) of professional experience. To ensure the consistency of the results, the coaches were asked to complete their ratings twice at an interval of four weeks (t₂) using the same procedure.

Figure 1.

Flow chart participants.

To examine the construct validity, self-ratings of the athletes were also needed. The coaches provided the contact information of 527 athletes who then received a cover letter and were invited to complete the self-report online questionnaires via a secured link to LimeSurvey, Version 2.50. The final sample included 250 athletes (48.4% female, 51.6% male; M_age = 15.38, SD = 1.98 years; floorball: 54.2%; soccer: 40.7%, ice hockey: 5.1%) who had participated in their respective sports for 7.58 years (SD = 3.07) and whose performance levels ranged from non-competitive (62.1%) to the national and international level (22.9%). Out of these 250 athletes, 160 (64%) were rated by two coaches. All the ratings were performed by two coaches and averaged before being included in the analysis which was also recommended for the AMBIS-I.¹

Measures

Achievement-motivated behavior (coach-ratings)

The preliminary version of the AMBIS-T included 14 prototypical behaviors whose occurrences over the last 12 months were to be evaluated on a 5-point scale ranging from 0 ( = never) to 4 ( = always). The items were assumed to match with the three relevant factors, including proactivity (e.g., “He/she stayed after the end of the session and continued to train”), ambition (e.g., “He/she was annoyed when the team lost a game”), and commitment (e.g., “He/she demonstrated an “active” posture during training sessions”). As all coaches had submitted a list with the names of the athletes they were coaching at that time, it was possible to request that they provide individual ratings for each athlete: “How often did athlete A (name of the respective athlete) display the behavior mentioned below over the last 12 months?” Additionally, coaches were asked how certain they felt in their assessment of each particular athlete (0 = not at all, 1 = a little, 2 = somewhat, and 3 = reasonably) and for how long (in years) they had known each athlete respectively.

Performance criteria

As an external performance criterion to assess concurrent criterion validity, the allocation of the Swiss Olympic Cards (SOC)³ at the first measurement point (t₁) was checked. Out of this criterion, two groups were formed: The first group included regional or lower-level athletes (n = 766; 88.5%) who were categorized with lower performance potential. The second group included athletes on a national or international level (n = 100; 11.5%) who were attributed a higher performance potential.

Motivational and volitional constructs (self-rating)

To establish construct validity, athletes were asked to fill in self-report questionnaires designed to capture the constructs of achievement goal orientation, self-determination, and volition and available in German (Table 1). All used scales displayed reasonable to good internal consistencies at t₁.

Table 1.

Self-report instruments used for the construct validation of the AMBIS-T.

Questionnaire	Construct	Scale	α	# Items	Response format	Sample item (translation from German original items)
Sport Orientation Questionnaire (SOQ²⁶;)	Achievement goal orientation	Competitiveness (CO)	.88	13	1–5	I’m looking forward to competitions.
		Win Orientation (WO)	.80	6		I have the most fun when I win.
		Goal Orientation (GO)	.75	6		I try hardest when I have a specific goal.
Sport Motivation Scale (SMS; Burtscher et al.,²⁷)	Self-determination	Self-Determination Index^a (SDI)	.86	28	1–7	It gives me pleasure to learn more about my sport.
Volitional Components in Sport (VCS; Wenhold, Elbe & Beckmann,²⁸)	Volition	Self-Optimization (SO)	.92	29	0–3	I am optimistic about most things in sports.
		Lack of Initiation (LI)	.87	13		I usually only start to train properly when someone puts pressure on me.
		Loss of Focus (LF)	.85	9		In training, I often have to think about things that have nothing to do with what I’m doing.

Note. ^aThe seven subscales Intrinsic Motivation “To Know,” “To Accomplish,” and To Experience,” External, Introjected and Identified Regulation, as well as Amotivation were combined to form a Self-Determination Index (see²⁹).

Data analysis

To ensure results are comparable, the procedure for data analysis was intentionally similar to that described in Zuber and Conzelmann.¹ As the amount of missing values was low (coach-ratings: 5.04%; self-ratings: 1.36%) and the values missing completely at random (Little's MCAR test: χ2 = 363.83, df = 360, p = 0.43),³⁰ data were imputed using the full information maximum likelihood procedure for the coach-ratings and the Expectation Maximization for the self-ratings.³¹ Factorial validity was computed using Mplus Version 8³² and reliability as well as criterion and construct validity were computed using IBM SPSS Statistics 27. The level of significance was set at p < 0.05 for all analyses.

Factorial validity

Exploratory structural equation models (ESEM) were conducted to examine the internal structure and appropriateness of the items of AMBIS-T (coach-rating). The ESEM model for t₁ was estimated using the robust maximum likelihood (RML) estimator method with target rotation, which was suggested as an adequate approach when the number of factors is already known.³³ Following the recommendations of Schermelleh-Engel, Moosbrugger, and Müller,³⁴ a good fit is indicated when χ2/df ≤ 2.00; CFI and TFI ≥.97; and RMSEA, and standardized root mean square residual (SRMR) ≤.05. Akaike's Information Criterion (AIC) is consulted for the comparison of the tested models with smaller values representing better fit.³⁵ To test the generalizability of the proposed model, the model was cross-validated with the data from t₂. To estimate discriminant evidence of the instrument, the Fornell-Larcker Criterion³⁶ was calculated. This criterion requires that the average variance extracted (AVE) of one factor be higher than any squared correlation with another factor, assuming discriminant evidence at the factor level.

Reliability

To estimate the reliability of the indicators, squared multiple correlations (SMC) were computed. The reliability of the constructs was estimated by the composite reliability (CR)³⁷ and the AVE.³⁶ As cut-offs for good reliability, SMC ≥.40, CR ≥.70, and AVE ≥.50 were used. To determine the test–retest reliability, reference was made to the data at t₂. The coaches included in this partial sample were invited to participate again after four weeks. Altogether, 637 estimates were made by 36 coaches after an average time of 6.3 ± 2.02 weeks. The inter-rater reliability between head coach and assistant coach was determined separately for each training group (consisting of athletes assessed by the same two coaches), using the intra-class coefficient (ICC 2-way random effect model, absolute agreement) and then averaged over each training group. The interpretation was based on the recommendations from Koo and Li,³⁸ who classified ICCs lower than 0.50 as poor, between 0.50 and 0.74 as sufficient, between 0.75 and 0.89 as good and higher than 0.90 as excellent.

Criterion validity

To assess concurrent criterion validity, achievement-motivated behaviors between the performance groups were compared using an independent t-test and differences were interpreted according to Cohen.³⁹

Construct validity

To assess construct validity, coach-ratings on the AMBIS-T scales were compared to the self-ratings from the athletes on the theoretically derived constructs, including achievement goal orientations, self-determination, and volition via Pearson correlations. According to the recommendations of Gignac and Szodorai⁴⁰ correlations with r = .10; .20, and .30 were interpreted as small, medium, and large effects.

Results

Factorial validity

To establish the factorial structure of the AMBIS-T and to choose the appropriate number of items, 3 ESEM models with 14, 13 (one item was eliminated because of a low communality of .36)⁴ and 12 items (one item was eliminated because of high content redundancy) were modeled and tested. As shown in Table 2, all three models displayed good to excellent fit, with the 12-item version showing the best fit and the lowest AIC. From the perspective of a higher research economy with a shorter questionnaire and because of the even distribution of the items over the three factors, it was then decided to further consider the 12-item version for the cross-validation with the data at the second measurement point (t₂).

Table 2.

Goodness of fit statistics and information criteria for the models estimated.

Models	χ²	p(df)	χ²/df	CFI	TLI	RMSEA (CI 90%)	SRMR	AIC²
Good (acceptable) fit¹	-	>0.05 (.01)	≤ 2 (3)	≥.97 (.95)	≥.95 (.90)	≤.05 (.08)	≤.05 (.10)	lower
AMBIS-T-14 t₁	154.81	<0.001(52)	2.98	.98	.96	.05 (.04–.06)	.02	27674.59
AMBIS-T-13 t₁	114.92	<0.001(42)	2.74	.98	.97	.05 (.04–.06)	.02	25788.51
AMBIS-T-12 t₁	86.56	<0.001(33)	2.62	.98	.97	.04 (.03–.06)	.02	24443.70
AMBIS-T-12 t₂	135.09	<0.001(33)	4.09	.97	.94	.07 (.06–.08)	.02

Notes. ¹according to Schermelleh-Engel et al.³⁴ and Brown⁴³; ²AIC for the model t₂ is not presented as it is not comparable to the AIC values for t₁ due to different data used; CI = confidence interval.

The fit of the AMBIS-T-12 t₂ solution was not as good as in the first measurement point but this represented the data sufficiently well with some good and some acceptable fit indices, with only the χ²/df being below the acceptable level (see Table 2). Therefore, the coach-rating scale for AMBIS-T consisted of 12 items composing the three factors of proactivity, ambition, and commitment.

The corresponding items of the AMBIS-T as well as factor loadings and communalities are displayed in Table 3. The AMBIS-T was designed and evaluated for use in the German language. The original German wording of the items is available in Table 1 of the Electronic Supplementary Material 1. Factor loadings on the main factors were for all items between .55 and .90, except for three items with cross-loadings slightly higher than. 20. All items displayed good communalities (> .40). As the AVE for each factor was higher than any squared correlation with another factor, all three factors meet the Fornell-Larcker criterion and therefore displayed discriminant evidence at the factor level.

Table 3.

Standardized factor loadings, subscales and communalities of items, and factor reliability of subscales and Cronbach's alpha and retest reliabilities of subscales and total score.

Scales/items	F1		F2		F3		SMC		CR		AVE		α	r _tt
Scales/items	t₁	t₂	t₁	t₂	t₁	t₂	t₁	t₂	t₁	t₂	t₁	t₂	t₁
Total Score AMBIS-T														.81
1. Proactivity									.84	.87	.58	.64	.84	.75
P1: He/she asked the coach for additional training opportunities in order to improve even further	.72	.76				.25	.56	.61
P2: He/she was one of the first to arrive at the training location and began to independently practice technical processes	.62	.65			.22		.48	.51
P3: He/she stayed after the end of the session and continued to train	.86	.86					.69	.71
P4: He/she independently inquired about the content of missed training sessions	.76	.86					.59	.74
2. Ambition									.83	.86	.57	.62	.82	.79
A1: He/she was annoyed when the team lost a game			.86	.86			.67	.70
A2: He/she has clearly communicated before the game that he/she wants to win			.63	.65			.44	.47
A3: He/she has shown that he/she is not satisfied with a loss or a draw			.86	.90			.71	.84
A4: He/she reacted emotionally when he/she achieved his/her self-defined goals			.55	.60	.28		.45	.46
3. Commitment									.79	.81	.51	.59	.80	.75
C1: He/she also carried out a simple exercise in a concentrated manner and made few errors					.76	.79	.52	.59
C2: He/she exerted him/herself to the point of exhaustion during highly demanding exercises					.60	.61	.47	.52
C3: He/she demonstrated an “active” posture during training sessions					.75	.69	.62	.55
C4: He/she repeatedly tried to complete a task, even when he/she has been unsuccessful thus far					.64	.80	.45	.62

Note. F1–F3: standardized factor loadings; loadings < .20 are not displayed; SMC: estimated squared multiple correlation or communality of the item; CR: composite reliability; AVE: average variance extracted; α: Cronbach's alpha/internal consistency; r_t_t: retest reliability. The original items in German are available in Table 1 of Electronic Supplementary Material 1.

Reliability

All factors presented good factor reliabilities at both t₁ and t₂ (Table 3). Both the test–retest reliability (r_tt ≥ .75) and Cronbach's alpha (α ≥. 80) were for all three factors within an acceptable range. The highest retest reliability was observed with the total score r_tt = .81. The ICCs of the coaches averaged over all training groups are presented in Table 4. All ICCs except that for proactivity are in a satisfactory range (> 0.5³⁸).

Table 4.

Intraclass correlation coefficients (ICC) as the measure of agreement between head and assistant coach averaged over all training groups (n = 19) for t₁ (ICC 2-way random effects model, absolute agreement) and descriptive statistics and between-group analyses of AMBIS-t.

	Intraclass correlation	Regional level and lower (n = 766)		National level and higher (n = 100)
Scales	ICC (mean)	M	SD	M	SD	t(df)	p	d (95% CI)
Proactivity	.45	1.23	0.86	2.19	0.83	9.83(815)	<0.001	1.13 (0.90–1.36)
Ambition	.61	2.28	0.85	2.69	0.74	4.47(811)	<0.001	0.49 (0.27–0.70)
Commitment	.59	2.67	0.71	2.79	0.61	1.58(862)	0.11	0.17 (0.04–0.38)
Total score AMBIS-T	.60	2.07	0.61	2.57	0.59	7.48(841)	<0.001	0.80 (0.59–1.02)

Criterion validity

As displayed in Table 4, significant differences in the two scales proactivity and ambition, and the total score of the AMBIS-T between the two performance groups were found with medium to large effect sizes. The largest effect size lies in the proactivity scale with the higher performance group presenting much higher achievement-motivated behavior. In the commitment scale, a small difference on a descriptive level in the expected direction was observed, but there was no significant effect. The assumption of sufficient criterion validity was thus only partially confirmed.

Construct validity

Results regarding construct validity are included in Table 5, which contains the set of correlations between the coach-ratings on the AMBIS-T and the self-ratings at t₁ on the motivational and volitional measures SOQ, SMS, and VQS. All significant correlations lied within a range between r = .14 and r = .42. According to the recommendations of Gignac and Szodorai,⁴⁰ with r = .10; .20, and .30 as small, medium, and large effects respectively, these correlations can be interpreted as small to medium, with a large one between competitiveness and proactivity. Overall, proactivity and the total score presented the highest and most consistent correlations with the self-rating scales, whereas the scale commitment displayed only fewer and lower correlations with the two scales self-determination and the volitional construct lack of initiation (negatively pooled).

Table 5.

Correlations (Pearson) between coach-ratings of the AMBIS-t and the self-report questionnaires at t₁ (n = 250).

Scales	Proactivity	Ambition	Commitment	Total Score
SOQ CO	.42*	.14*	−.01	.27*
SOQ WO	.23*	.10	−.06	.15*
SOQ GO	.22*	.09	.01	.15*
SMS SDI	.20*	.17*	.18*	.23*
VQS SO	.27*	.17*	.09	.24*
VQS LI	−.17*	−.17*	−.19*	−.24*
VQS LF	−.16*	−.14*	−.11	−.19*

Note. * p < 0.05 (two-sided); CO = competitiveness; WO = win orientation; GO = goal orientation; SDI = self-determination index; SO = self-optimization; SI = self-impediment; LI = lack of initiation; LF = loss of focus.

In terms of content, the factor proactivity showed, as expected, a positive relationship with achievement goal orientation, self-determination, and volition. For the factor ambition, a higher agreement with the achievement goal orientation was expected, whereas the results suggest relationships with self-determination and volition. Results for the factor commitment were consistent with the expectation of a negative correlation with the volitional scale lack of initiation.

Discussion

The current study aimed to adapt the coach-rating instrument AMBIS-I, initially designed for assessing achievement-motivated behavior in individual sports,¹ to team sports and to provide an alternative perspective to assess the achievement motivation of athletes independently of their self-reports. Coaches in team sports typically have to evaluate a multitude of athletes and have even less performance-related information available to make such assessments in comparison to coaches in individual sports.⁴⁴ The second purpose of this study was to check the psychometric properties, including factorial validity, reliability, and criterion and construct validity.

For the adaptation, the three known factors from AMBIS-I were included after an expert consultation on their suitability for team sports. The examination of the test quality criteria suggests acceptable results, but still to be considered with caution. The AMBIS-T was found to display a good fit with the observed data and to measure the factors of proactivity, ambition, and commitment with 12 items reliably. In the area of interrater reliability, the factor proactivity was not entirely convincing. Possibly the limited observability of two behaviors included in this factor is partly responsible for the rather low agreement as the coach might arrive later or leave earlier than the athletes (P2: He/she was one of the first to arrive at the training location and began to independently practice technical processes; P3: He/she stayed after the end of the session and continued to train). Thus, the recommendation for including two coaches to complete the ratings remains appropriate, as it is assumed that two coaches can better perceive more relevant behaviors than only one coach and that, taken together, this results in a more valid judgment.

The results regarding concurrent criterion validity indicated that athletes in the better performance group were rated as being more achievement-motivated in the areas of proactivity and ambition, consistently with a line of research findings suggesting a positive correlation between motivational characteristics and performance.^3,4 The non-significant difference in a commitment does not meet the expectations, as there are also empirical findings that volitional or self-regulative aspects, which play a decisive role in the commitment factor, are positively related to performance.^45,46 This could be partly due to a ceiling effect and therefore restricted variance and means that the average expression in factor commitment is at the upper end of the scale. Due to the variance being limited, correlations were underestimated and the groups could hardly be differentiated.^47,48 This issue, already discussed in the AMBIS-I,¹ could only partly be addressed by strengthening the factor commitment and by adding three new items to mitigate the likely restricted variance. Unfortunately, two of these three new items were removed again when testing for the best model (see results for factorial validity). The variance of the factor commitment was still the most restricted but with a lower difference compared to the other scales as in the AMBIS-I. It remains to be investigated how high levels of commitment characteristics at the upper end of the scale could be more differentiated. It is also conceivable that restricted variance was one of the reasons for the low correlations observed for the construct validity of the factor commitment. However, low correlations are not per se surprising as two relevant differences between the compared constructs were obvious. First, the constructs used for validation were not identical but related (i.e., behaviors in the case of AMBIS-T and motives in the case of the self-rating instruments). Second, in the coach- and the self-ratings, two different data sources were used where evidence from other psychological contexts indicated that low correlations between self-reports and reports from others (e.g., teachers) are to be expected.^49,50

However, most results of the construct validation of the AMBIS-T converged in the expected direction. All factors displayed significant relationships with self-determination, which highlights the importance of sufficient autonomous motivation for performance in sports.⁵¹ Moreover, all factors indicated theory-compliant correlations with the volitional constructs. However, the higher correlation displayed by the factor proactivity with the achievement goal orientations than by the factor ambition was less expected. It is possible that the behaviors included in the factor ambition (e.g., showing dissatisfaction in defeat) were less important for individual athletes in team sports or were less strongly linked to internal motives, as individual contributions to success are smaller and less easily identifiable in team sports.²²

In addition to these content-related shortcomings, further limitations need to be addressed. First, the cross-validation of the model with the same sample at the second measurement point was not ideal. However, due to the limited sample size and limited access to highly qualified coaches in charge of selection decisions, this was deemed an acceptable solution. Furthermore, the instrument is, in its current form, only generalizable to the German-speaking area and to the three included team sports soccer, ice hockey, and floorball. Further research is warranted to examine adaptation, for example, into the English language, and also check whether this would also have led to changes in content due to cultural differences. Additionally, in adapting to AMBIS-T, it was not checked whether other dimensions could be relevant and potentially lead to insightful comparisons between the individual and team sports versions. However, there is also a possibility that certain behaviors relevant to team sports were not integrated. The use of qualitative interview study designs involving additional expert coaches would be promising in this regard. Also, a fundamental objective for the development of the two AMBIS instruments was to reduce the inherent risk of social desirability associated with self-ratings. Coach ratings could thus represent a promising alternative, even though the absence of any bias (e.g., recall or confirmation bias⁵²) cannot be ensured. It can be assumed that focusing on specific behavior might be less subject to response biases in comparison to other forms of assessments.⁵³ Furthermore, the recommendations (a) for selection purpose to always include ratings from two coaches¹ and (b) for using multidimensional test batteries which also includes objective tests and performance results^6,7 can be useful in reducing bias.

In summary, the AMBIS-T can be considered a successful adaptation of the coach rating scale AMBIS-I for the assessment of achievement-motivated behavior from individual sports to team sports, with the exception of the scale commitment for criterion validity and the scale ambition for construct validity. In addition to the shortcomings already addressed, further investigations examining long-term predictive criterion validity are warranted. Finally, the extent to which data source triangulation including self-reports and other perspectives assessing motivational variables could lead to improved predictions of athletic performance remains to be explored.

Supplemental Material

sj-docx-1-spo-10.1177_17479541221122430 - Supplemental material for Construction and validation of the German coach-rating scale to assess achievement-motivated behavior in team sports (AMBIS-T)

Supplemental material, sj-docx-1-spo-10.1177_17479541221122430 for Construction and validation of the German coach-rating scale to assess achievement-motivated behavior in team sports (AMBIS-T) by Claudia Zuber in International Journal of Sports Science & Coaching

Footnotes

Acknowledgments

I would like to thank the Swiss Olympic and the Swiss Federal Office for Sport for supporting and funding this research project. Additionally, I wish to acknowledge the support provided by Bryan Charbonnet for the implementation of the project and the editing support provided by Olivier Schmid.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article. This work was supported by the Swiss Olympic, Swiss Federal Office for Sport.

ORCID iD

Claudia Zuber

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Zuber

Conzelmann

. Achievement-motivated behavior in individual sports (AMBIS-I) – coach rating scale: development and preliminary validation. Ger J Exercise Sport Res 2019; 49: 410–423.

Brunstein

Heckhausen

. Achievement motivation. In: Heckhausen

Heckhausen

(eds) Motivation and action. 1 ed. Cambridge: Cambridge University Press, 2010, pp.137–183.

Rees

Hardy

Güllich

, et al. The Great British medalists project: a review of current knowledge on the development of the world’s best sporting talent. Sports Med 2016; 46: 1041–1058.

Ivarsson

Kilhage-Persson

Martindale

, et al. Psychological factors and future performance of football players: a systematic review with meta-analysis. J Sci Med Sport 2020; 23: 415–420.

Fuchslocher

Romann

Rüdisüli

, et al. Das Talentidentifikationsinstrument PISTE – Wie die Schweiz Nachwuchsathleten auswählt [The talent identification tool PISTE – how Switzerland selects junior athletes]. Leistungssport 2011; 4: 22–27.

Feichtinger

Höner

. Psychological diagnostics in the talent development program of the German Football Association: psychometric properties of an internet-based test battery. Sportwissenschaft 2014; 44: 203–213.

Vaeyens

Lenoir

Williams

, et al. Talent identification and development programmes in sport: current models and future directions. Sports Med 2008; 38: 703–714.

Elliot

Dweck

. Competence and motivation: competence as the core of achievement motivation. In: Elliot

Dweck

(eds) Handbook of competence and motivation. Paperback ed. New York: Guilford, 2007, pp. 3–12.

Conroy

Elliot

Coatsworth

. Competence motivation in sport and exercise: the hierarchical model of achievement motivation and self-determination theory. In: Hagger

Chatzisarantis

(eds) Intrinsic motivation and self-determination in exercise and sport. Champaign: Human Kinetics, 2007, pp.181–192.

10.

Atkinson

. Motivational determinants of risk-taking behavior. Psychol Rev 1957; 64: 359–372.

11.

Duda

. Motivation in sport settings: a goal perspective. In: Roberts

(ed) Motivation in sport and exercise. Champaign, IL: Human Kinetics, 1992, pp.57–91.

12.

Ryan

Deci

. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000; 55: 68–78.

13.

Furr

Funder

. Behavioral observation. In: Robins

Fraley

Krueger

(eds) Handbook of research methods in personality psychology. New York, London: Guilford, 2010, pp.273–291.

14.

Gill

Deeter

. Development of the sport orientation questionnaire. Res Q Exerc Sport 1988; 59: 191–202.

15.

Pelletier

Rocchi

Vallerand

, et al. Validation of the revised sport motivation scale (SMS-II). Psychol Sport Exerc 2013; 14: 329–341.

16.

Furr

Bacharach

. Psychometrics: an introduction. 2nd edition. Los Angeles: Sage, 2014.

17.

Zuber

Schmid

Conzelmann

. Achievement-motivated behavior in individual sports: evidence for the construct and criterion validity of the AMBIS-I coach-rating scale. J Sports Sci Med 2020; 19: 10–19.

18.

Musculus

Lobinger

. Psychological characteristics in talented soccer players – recommendations on how to improve coaches’ assessment. Front Psychol 2018; 9: 41.

19.

Den Hartigh

RJR

Niessen

ASM

Frencken

WGP

, et al. Selection procedures in sports: improving predictions of athletes’ future performance. Eur J Sport Sci 2018; 18: 1191–1198.

20.

Schmid

Conzelmann

Zuber

. Patterns of achievement-motivated behavior and performance as predictors for future success in rowing: a person-oriented study. Int J Sports Sci Coach 2021; 16: 101–109.

21.

Baker

Wilson

Johnston

, et al. Talent research in sport 1990–2018: a scoping review. Front Psychol 2020; 11: 607710.

22.

van de Pol

PKC

Kavussanu

. Achievement motivation across training and competition in individual and team sports. Sport, Exercise Perform Psychol 2012; 1: 91–105.

23.

Karabatsos

Malousaris

Apostolidis

. Evaluation and comparison of burnout levels in basketball, volleyball and track and field coaches. Stud Phys Culture Tourism 2006; 13: 79–83.

24.

Hollembeak

Amorose

. Perceived coaching behaviors and college athletes’ intrinsic motivation: a test of self-determination theory. J Appl Sport Psychol 2005; 17: 20–36.

25.

Elbe

A-M

Szymanski

Beckmann

. The development of volition in young elite athletes. Psychol Sport Exerc 2005; 6: 559–569.

26.

Elbe

A-M

. Testgütekriterien der deutschen Version des Sport Orientation Questionnaires [Psychometric properties of the German version of the Sport Orientation Questionnaire]. Spectrum der Sportwissenschaft 2004; 16: 96–107.

27.

Burtscher

Furtner

Sachse

, et al. Validation of a German version of the Sport Motivation Scale (SMS28) and motivational analysis in competitive mountain runners. Percept Mot Skills 2011; 112: 807–820.

28.

Wenhold

Elbe

A-M

Beckmann

. VKS: Fragebogen zur Erfassung der Volitionaler Komponenten im Sport [Questionnaire for recording the volitional components in sports], http://www.bisp-sportpsychologie.de/SpoPsy/DE/Diagnostikportal/Motivation/Sportlerfrageboegen/vqs_2008/vks_einfuehrung.html?nn=3014646 (2008, accessed 22 June 2015).

29.

Vallerand

. A hierarchical model of intrinsic and extrinsic motivation in sport and exercise. In: Roberts

(ed) Advances in motivation in sport and exercise. Champaign, IL: Human Kinetics, 2001, pp.263–319.

30.

Little

RJA

. A test of missing completely at random for multivariate data with missing values. J Am Stat Assoc 1988; 83: 1198–1202.

31.

Tabachnick

Fidell

. Using multivariate statistics. 6. edition, Pearson new international edition. Harlow, Essex: Pearson Education, 2014.

32.

Muthén

. Mplus user’s guide. 8th ed. Los Angeles, CA: Muthén & Muthén, 1998–2017.

33.

Marsh

Morin

AJS

Parker

, et al. Exploratory structural equation modeling: an integration of the best features of exploratory and confirmatory factor analysis. Annu Rev Clin Psychol 2014; 10: 85–110.

34.

Schermelleh-Engel

Moosbrugger

Müller

. Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods Psychol Res Online 2003; 8: 23–74.

35.

Akaike

. Factor analysis and AIC. Psychometrika 1987; 52: 317–332.

36.

Fornell

Larcker

. Evaluating structural equation models with unobservable variables and measurement error. J Marketing Res 1981; 18: 39.

37.

Bagozzi

. Specification, evaluation, and interpretation of structural equation models. J Acad Marketing Sci 2012; 40: 8–34.

38.

Koo

. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155–163.

39.

Cohen

. Statistical power analysis for the behavioral sciences. 2nd ed. Hoboken, NJ: Lawrence Erlbaum, 1988.

40.

Gignac

Szodorai

. Effect size guidelines for individual differences researchers. Pers Individ Dif 2016; 102: 74–78.

41.

Bühner

. Einführung in die Test- und Fragebogenkonstruktion [Introduction into test and questionnaire construction]. 3., aktualisierte und erweiterte Aufl. München: Pearson Studium, 2011.

42.

Field

. Discovering statistics using IBM SPSS statistics. 5th edition. Los Angeles, London, New Delhi, Singapore, Washington DC, Melbourne: Sage, 2018.

43.

Brown

. Confirmatory factor analysis for applied research. New York, London: The Guilford Press, 2015.

44.

Hanrahan

Cerin

. Gender, level of participation, and type of sport: differences in achievement goal orientation and attributional style. J Sci Med Sport 2009; 12: 508–512.

45.

Erikstad

Høigaard

Johansen

, et al. Childhood football play and practice in relation to self-regulation and national team selection: a study of Norwegian elite youth players. J Sports Sci 2018; 36(20): 2304–2310.

46.

Höner

Feichtinger

. Psychological talent predictors in early adolescence and their empirical relationship with current and future performance in soccer. Psychol Sport Exerc 2016; 25: 17–26.

47.

Vogt

. Dictionary of statistics & methodology: A nontechnical guide for the social sciences. Third edition. Thousand Oaks, London, New Delhi: Sage Publications, 2005.

48.

Wiberg

Sundström

. A comparison of two approaches to correction of restriction of range in correlation analysis. Pract Assess Res Eval 2009; 14: 1–9

49.

Renk

Phares

. Cross-informant ratings of social competence in children and adolescents. Clin Psychol Rev 2004; 24: 239–254.

50.

Conway

Huffcutt

. Psychometric properties of multisource performance ratings: a meta-analysis of subordinate, supervisor, peer, and self-ratings. Hum Perform 1997; 10: 331–360.

51.

Gillet

Berjot

Vallerand

, et al. Examining the motivation-performance relationship in competitive sport: a cluster-analytic approach. Int J Sport Psychol 2012; 43: 79–102.

52.

Althubaiti

. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc 2016; 9: 211–217.

53.

McCrae

Weiss

. Observer ratings of personality. In: Robins

Fraley

Krueger

(eds) Handbook of research methods in personality psychology. New York, London: Guilford, 2010, pp.259–272.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB