Modified Wisconsin Card Sorting Test (M-WCST): Normative data for Spanish-speaking pediatric population

Abstract

OBJECTIVE:

To generate normative data for the Modified Wisconsin Card Sorting Test (M-WCST) in Spanish-speaking pediatric populations.

METHOD:

The sample consisted of 4,373 healthy children from nine countries in Latin America (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. Each participant was administered the M-WCST as part of a larger neuropsychological battery. Number of categories, perseverative errors, and total error scores were normed using multiple linear regressions and standard deviations of residual values. Age, age², sex, and mean level of parental education (MLPE) were included as predictors in the analyses.

RESULTS:

The final multiple linear regression models indicated main effects for age on all scores, such that the number of categories correct increased and total number of perseverative errors and total number of errors decrease linearly as a function of age. Age² had a significant effect in Chile, Cuba, Ecuador, and Spain for numbers of categories; a significant effect for number of perseverative errors in Chile, Cuba, Mexico, and Spain; and a significant effect for number of total errors in Chile, Cuba, Peru, and Spain. Models showed an effect for MLPE in Cuba (total errors), Ecuador (categories and total errors), Mexico (all scores), Paraguay (perseverative errors and total error), and Spain (categories and total errors). Sex affected number of total errors for Ecuador.

CONCLUSIONS:

This is the largest Spanish-speaking pediatric normative study in the world, and it will allow neuropsychologists from these countries to have a more accurate way to interpret the M-WCST with pediatric populations.

Keywords

Modified Wisconsin Card Sorting Test neuropsychology Spanish-speaking populations pediatric population

1 Introduction

The Modified Wisconsin Card Sorting Test (M-WCST; Nelson, 1976) was developed as a more efficient, patient-friendly alternative to the Wisconsin Card Sorting Test (WCST; Berg, 1948; Grant & Berg, 1948). Since its inception, the M-WCST has become a popular tool used by neuropsychologists to assess executive functioning, or the set of higher-order cognitive functions generally associated with the prefrontal cortex. To perform well on the M-WCST, individuals must demonstrate mental flexibility as well as the ability to plan effectively, inhibit impulsive responses, and respond effectively to feedback (Nelson, 1976). Poor performance on the M-WCST is often indicative of executive dysfunction.

The M-WCST eliminates some of the aspects of the classic WCST that can be frustrating for patients, particularly those with cognitive impairments or other conditions that can make the full WCST seem lengthy and difficult (Nelson, 1976). For example, in the classic WCST, most response cards share more than one attribute with a stimulus card (e.g., color and shape), such that it can be impossible for patients to know why a match is incorrect. In the M-WCST, no response cards share more than one attribute with any stimulus card. In addition, in the classic WCST, when the sorting principle switches, the test-taker is not told that he or she needs to find a new rule; rather, he or she must deduce this after being told that a given match is incorrect. Furthermore, in the classic WCST, the examiner follows a fixed pattern of matches that will be considered correct, such that the test-taker’s first match is only sometimes correct (Berg, 1948; Grant & Berg, 1948). In the M-WCST, whatever principle the test-taker chooses first sets the first correct principle. And finally, in the M-WCST, the number of response cards is also reduced from 128 to 48, making the test substantially shorter. Overall, the efficiency and simplicity of the M-WCST means this instrument is particularly well-suited for administration to children, who can tend to become frustrated with the full WCST and refuse participation (Cianchetti, Corona, Foscoliano, Contu, & Sannio-Fancello, 2007). However, there is a paucity of research using the M-WCST with the pediatric cohort.

Although the classic WCST has been used by researchers to investigate executive dysfunction in children with a wide variety of medical and psychiatric conditions including Attention-Deficit/ Hyperactivity Disorder (Romine et al., 2004); learning disabilities (Helland & Asbjornsen, 2000); conduct disorders (Morgan & Lilienfeld, 2000); autism (Ozonoff, 1995); and temporal lobe epilepsy (Igarashi et al., 2002), to our knowledge, only three studies to date have used the M-WCST to investigate executive function in children or adolescents. In 2000, Diforio, Walker, and Kestler reported that youth with schizotypal personality disorder demonstrated relatively poorer performance on the M-WCST as compared to healthy controls. In 2013, Esposito et al. revealed that youth with obstructive sleep apnea performed worse on several key indices of the M-WCST. Finally, in an investigation of executive functioning deficits in adolescents and young adults with 22q11.2 deletion syndrome, Rockers et al. (2009) reported that participants with this genetic disorder performed significantly worse than controls on most indices of the M-WCST. Taken together, these studies do suggest that the M-WCST may be an effective tool for documenting executive dysfunction deficits in clinical samples of children. However, a lack of normative data for children means the instrument is not yet widely used in clinical settings.

Only one set of normative data for children on the M-WCST has been presented in the literature. In 2007, Cianchetti and colleagues administered the M-WCST to a group of Italian youth between the ages of 4 and 13 and compared their patterns of performance to that of children on the classic WCST. Results revealed predictable improvement in children’s scores on most parameters of the M-WCST with age. The authors noted that this is a similar pattern of performance as has been observed on the WCST, and concluded that the M-WCST should be more widely used with children (Cianchetti et al., 2007). However, cultural factors may mean that these normative values not be appropriate for use with children living other areas of the world. For example, children in Latin America may perform differently on the M-WCST as compared to their European peers because environmental variables, such as parental levels of education, may play an important role in the development of non-verbal executive functions (Rosselli & Ardila, 2003).

Although the M-WCST may be a clinically useful tool for Latin American children, a lack of culturally appropriate normative data means that the instrument is not yet widely used in the Spanish-speaking world. The development of a normative data set for Latin American and Spanish children on the M-WCST would enable clinicians to utilize this measure in both clinical and research settings. Therefore, the aim of the present is to present a set of normative data for children and adolescents from nine Latin American countries and Spain based on multiple linear regression analyses.

2 Method

2.1 Participants

The sample consisted of 4,373 healthy children who were recruited from Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, Puerto Rico, and Spain. Participants were selected according to the following criteria: a) were between 6 and 17 years of age, b) were born and currently lived in a country where the study was conducted, c) spoke Spanish as their mother tongue, d) an IQ≥80 on the Test Of Non-verbal Intelligence (TONI-2, Brown, Sherbenou, & Johnsen, 2009), and e) scored <19 on the Children’s Depression Inventory (CDI, Kovacs, 1992).

Children with history of neurologic or psychiatric disorders as reported by the participant’s parent(s) were excluded due to its effects on cognitive performance. Participants in the study were from public and private schools, and signed an informed consent. Socio-demographic and participant characteristics for each of the countries’ samples have been reported elsewhere (Rivera & Arango-Lasprilla, 2017). Ethics Committee approval was obtained for the study in each country.

2.2 Instrument administration

The M-WCST consists of four stimulus cards and 48 response cards. Each card varies in shape (cross, circle, triangle, or star), color (red, blue, yellow, or green), and number (one to four). The objective is to classify correctly the stimulus card according to certain rule until completion of a category. The test continues until all six categories are classified or until the whole volume has been used (Schretlen, 2010). The test allows for calculation of the number of categories, perseverations, and total errors.

Table 1
Final multiple linear regression models for the M-WCST Number of categories correct

Country B Std. Error t Sig. R² SD (residual)

Chile Constant 5.760 0.073 78.954 <0.001 0.244 0.949

Age 0.134 0.014 9.551 <0.001

Age² – 0.027 0.005 – 5.806 <0.001

Cuba Constant 5.729 0.085 67.206 <0.001 0.411 1.098

Age 0.229 0.016 13.992 <0.001

Age² – 0.044 0.005 – 8.136 <0.001

Ecuador Constant 4.571 0.184 24.853 <0.001 0.093 1.415

Age 0.096 0.024 3.991 <0.001

Age² – 0.019 0.008 – 2.380 0.018

MLPE 0.628 0.189 3.328 0.001

Guatemala Constant 4.025 0.117 34.355 <0.001 0.141 1.661

Age 0.268 0.047 5.739 <0.001

Honduras Constant 4.389 0.099 44.121 <0.001 0.112 1.697

Age 0.187 0.031 6.033 <0.001

Mexico Constant 4.235 0.075 56.482 <0.001 0.112 1.585

Age 0.151 0.015 10.077 <0.001

MLPE 0.447 0.105 4.266 <0.001

Paraguay Constant 5.560 0.049 113.187 <0.001 0.061 0.849

Age 0.062 0.014 4.410 <0.001

Peru Constant 4.734 0.084 56.278 <0.001 0.094 1.519

Age 0.148 0.026 5.796 <0.001

Puerto Rico Constant 5.456 0.079 69.469 <0.001 0.030 1.105

Age 0.055 0.022 2.478 0.014

Spain Constant 4.909 0.092 53.622 <0.001 0.211 1.411

Age 0.205 0.014 15.009 <0.001

Age² – 0.025 0.004 – 5.589 <0.001

MLPE 0.370 0.095 3.884 <0.001

Country		B	Std. Error	t	Sig.	R²	SD (residual)
Chile	Constant	5.760	0.073	78.954	<0.001	0.244	0.949
	Age	0.134	0.014	9.551	<0.001
	Age²	– 0.027	0.005	– 5.806	<0.001
Cuba	Constant	5.729	0.085	67.206	<0.001	0.411	1.098
	Age	0.229	0.016	13.992	<0.001
	Age²	– 0.044	0.005	– 8.136	<0.001
Ecuador	Constant	4.571	0.184	24.853	<0.001	0.093	1.415
	Age	0.096	0.024	3.991	<0.001
	Age²	– 0.019	0.008	– 2.380	0.018
	MLPE	0.628	0.189	3.328	0.001
Guatemala	Constant	4.025	0.117	34.355	<0.001	0.141	1.661
	Age	0.268	0.047	5.739	<0.001
Honduras	Constant	4.389	0.099	44.121	<0.001	0.112	1.697
	Age	0.187	0.031	6.033	<0.001
Mexico	Constant	4.235	0.075	56.482	<0.001	0.112	1.585
	Age	0.151	0.015	10.077	<0.001
	MLPE	0.447	0.105	4.266	<0.001
Paraguay	Constant	5.560	0.049	113.187	<0.001	0.061	0.849
	Age	0.062	0.014	4.410	<0.001
Peru	Constant	4.734	0.084	56.278	<0.001	0.094	1.519
	Age	0.148	0.026	5.796	<0.001
Puerto Rico	Constant	5.456	0.079	69.469	<0.001	0.030	1.105
	Age	0.055	0.022	2.478	0.014
Spain	Constant	4.909	0.092	53.622	<0.001	0.211	1.411
	Age	0.205	0.014	15.009	<0.001
	Age²	– 0.025	0.004	– 5.589	<0.001
	MLPE	0.370	0.095	3.884	<0.001

Note. MLPE: Mean level of parental education.

Table 2

Final multiple linear regression models for the M-WCST Number of perseverative errors

Country		B	Std. Error	t	Sig.	R²	SD (residual)
Chile	Constant	2.410	0.285	8.445	<0.001	0.125	3.711
	Age	– 0.366	0.055	– 6.676	<0.001
	Age²	0.058	0.018	3.240	<0.001
Cuba	Constant	2.927	0.366	7.985	<0.001	0.244	4.722
	Age	– 0.753	0.070	– 10.718	<0.001
	Age²	0.059	0.023	2.561	0.011
Ecuador	Constant	3.524	0.209	16.904	<0.001	0.067	3.617
	Age	– 0.280	0.061	– 4.624	<0.001
Guatemala	Constant	4.095	0.229	17.900	<0.001	0.118	3.181
	Age	– 0.473	0.093	– 5.076	<0.001
Honduras	Constant	5.354	0.408	13.128	<0.001	0.099	6.956
	Age	– 0.717	0.127	– 5.650	<0.001
Mexico	Constant	4.497	0.294	15.287	<0.001	0.103	4.652
	Age	– 0.420	0.044	– 9.526	<0.001
	Age²	0.034	0.015	2.328	0.020
	MLPE	– 0.997	0.310	– 3.221	0.001
Paraguay	Constant	2.118	0.220	9.623	<0.001	0.040	2.268
	Age	– 0.107	0.038	– 2.817	0.005
	MLPE	– 0.544	0.275	– 1.975	0.049
Peru	Constant	5.280	0.343	15.405	<0.001	0.091	6.188
	Age	– 0.592	0.104	– 5.699	<0.001
Puerto Rico	Constant	2.350	0.231	10.194	<0.001	0.051	3.244
	Age	– 0.211	0.065	– 3.245	0.001
Spain	Constant	2.209	0.173	12.765	<0.001	0.122	3.679
	Age	– 0.390	0.035	– 11.146	<0.001
	Age²	0.049	0.011	4.365	<0.001

Note. MLPE: Mean level of parental education.

2.3 Statistical analyses

Detailed statistical analyses used to generate the normative data for the M-WCST scores are described in Rivera and Arango-Lasprilla (2017). In summary, the scores were standardized using multiple linear regression analyses by means of a four-step procedure: 1) First, the number of categories, perseverations, and total errors scores were computed separately by means of the final multiple regression models. The full regression models included as predictors: age, age², sex, and mean level of parental education (MLPE). Age was centered (= calendar age – mean age in the sample by country) before computing the quadratic age term to avoid multicollinearity (Aiken & West, 1991). Sex was coded as male = 1 and female = 0. The MLPE variable was coded as 1 if the participant’s parent(s) had >12 years of education or 0 if participant’s parent(s) had ≤12 years of education. If predicted variables were not statistically significant in the multivariate model with an alpha of 0.05, the non-significant variables were removed and the model was run again. A final regression model was conducted ${\hat{y}}_{i} = B_{0} + B_{1} \cdot {(Age - {\bar{x}}_{Age by country})}_{i} + B_{2} \cdot {(Age - {\bar{x}}_{Age by country})}_{i}^{2} + B_{3} \cdot {Sex}_{i} + B_{4} \cdot {MLPE}_{i}$ . 2) Residual scores were calculated based on the final model ( $e_{i} = y_{i} - {\hat{y}}_{i}$ ). 3) Residuals were standardized using the residual Standard Deviation (SD_e) value provided by the regression model: z_i = e_i/SD_e. 4) Standardized residuals were converted to percentile values using the standard normal cumulative distribution function. This four-step process was applied for number of categories, perseverations, and total errors scores separately for each country.

For all multiple linear regression models, the following assumptions were evaluated: a) multicollinearity by the values of the Variance Inflation Factor (VIF), which must not exceed 10, and the collinearity tolerance values, which must not exceed the value of 1 (Kutner, Nachtsheim, Neter, & Li, 2005), and b) the existence of influential values by calculating the Cook’s distance. The maximum Cook’s distance value was related to a F (p, n - p) distribution. Influential values are considered when percentile value is equal or higher than 50 (Cook, 1977; Kutner et al., 2005). All analyzes were performed using SPSS version 23 (IBM Corp., Armonk, NY).

Table 3
Final multiple linear regression models for the M-WCST Number of total errors

Country B Std. Error t Sig. R² SD (residual)

Chile Constant 5.099 0.450 11.327 <0.001 0.244 5.853

Age – 0.849 0.086 – 9.828 <0.001

Age² 0.151 0.028 5.322 <0.001

Cuba Constant 7.583 0.970 7.818 <0.001 0.305 6.625

Age – 1.203 0.099 – 12.155 <0.001

Age² 0.081 0.032 2.493 0.013

MLPE – 2.343 0.981 – 2.388 0.017

Ecuador Constant 12.642 0.981 12.887 <0.001 0.075 7.867

Age – 0.356 0.133 – 2.666 0.008

MLPE – 3.750 1.047 – 3.580 <0.001

Sex 1.850 0.931 1.987 0.048

Guatemala Constant 15.267 0.603 25.334 <0.001 0.141 8.543

Age – 1.377 0.240 – 5.732 <0.001

Honduras Constant 12.902 0.591 21.820 <0.001 0.122 10.086

Age – 1.166 0.184 – 6.337 <0.001

Mexico Constant 14.703 0.419 35.055 <0.001 0.134 8.863

Age – 0.944 0.084 – 11.248 <0.001

MLPE – 2.648 0.586 – 4.518 <0.001

Paraguay Constant 6.248 0.485 12.884 <0.001 0.076 4.998

Age – 0.340 0.083 – 4.073 <0.001

MLPE – 1.600 0.606 – 2.638 0.009

Peru Constant 9.692 0.752 12.895 <0.001 0.154 9.055

Age – 1.025 0.156 – 6.556 <0.001

Age² 0.124 0.051 2.409 0.017

Puerto Rico Constant 6.523 0.467 13.979 <0.001 0.036 6.566

Age – 0.358 0.132 – 2.719 0.007

Spain Constant 10.201 0.545 18.724 <0.001 0.213 8.397

Age – 1.238 0.081 – 15.255 <0.001

Age² 0.141 0.026 5.384 <0.001

MLPE – 1.936 0.567 – 3.413 0.001

Country		B	Std. Error	t	Sig.	R²	SD (residual)
Chile	Constant	5.099	0.450	11.327	<0.001	0.244	5.853
	Age	– 0.849	0.086	– 9.828	<0.001
	Age²	0.151	0.028	5.322	<0.001
Cuba	Constant	7.583	0.970	7.818	<0.001	0.305	6.625
	Age	– 1.203	0.099	– 12.155	<0.001
	Age²	0.081	0.032	2.493	0.013
	MLPE	– 2.343	0.981	– 2.388	0.017
Ecuador	Constant	12.642	0.981	12.887	<0.001	0.075	7.867
	Age	– 0.356	0.133	– 2.666	0.008
	MLPE	– 3.750	1.047	– 3.580	<0.001
Sex	1.850	0.931	1.987	0.048
Guatemala	Constant	15.267	0.603	25.334	<0.001	0.141	8.543
	Age	– 1.377	0.240	– 5.732	<0.001
Honduras	Constant	12.902	0.591	21.820	<0.001	0.122	10.086
	Age	– 1.166	0.184	– 6.337	<0.001
Mexico	Constant	14.703	0.419	35.055	<0.001	0.134	8.863
	Age	– 0.944	0.084	– 11.248	<0.001
	MLPE	– 2.648	0.586	– 4.518	<0.001
Paraguay	Constant	6.248	0.485	12.884	<0.001	0.076	4.998
	Age	– 0.340	0.083	– 4.073	<0.001
	MLPE	– 1.600	0.606	– 2.638	0.009
Peru	Constant	9.692	0.752	12.895	<0.001	0.154	9.055
	Age	– 1.025	0.156	– 6.556	<0.001
	Age²	0.124	0.051	2.409	0.017
Puerto Rico	Constant	6.523	0.467	13.979	<0.001	0.036	6.566
	Age	– 0.358	0.132	– 2.719	0.007
Spain	Constant	10.201	0.545	18.724	<0.001	0.213	8.397
	Age	– 1.238	0.081	– 15.255	<0.001
	Age²	0.141	0.026	5.384	<0.001
	MLPE	– 1.936	0.567	– 3.413	0.001

Note. MLPE: Mean level of parental education.

3 Results

3.1 M-WCST number of categories correct

The final multivariate linear regression models for the ten countries-specific number of categories correct were significant (see Table 1). In all countries, the number of categories correct increased linearly as a function of age. The number of categories correct for Chile, Cuba, Ecuador, and Spain were affected by a quadratic age effect. Children from Ecuador, Mexico, and Spain whose parent(s) had a MLPE >12 years obtained higher number of categories correct than children whose parent(s) had a MLPE ≤12 years. The child’s sex did not affect the numbers of categories correct for any country. The amount of variance these predictors explained in the number of categories correct ranged from 3.0% (in Puerto Rico) to 41.1% (in Cuba).

3.2 M-WCST number of perseverative errors

The final multivariate linear regression models for the ten country-specific numbers of perseverative errors were significant (see Table 2). In all countries, the number of perseverative errors decreased linearly as a function of age. The number of perseverative errors for Chile, Cuba, Mexico, and Spain were affected by a quadratic age effect. Children from Mexico and Paraguay whose parent(s) had a MLPE >12 years made lower number of perseverative errors than children whose parent(s) had a MLPE ≤12 years. The child’s sex did not affect the number of perseverative errors for any country. The amount of variance these predictors explained in the number of perseverative errors ranged from 4.0% (in Paraguay) to 24.4% (in Cuba).

3.3 M-WCST number of total errors

The final multivariate linear regression models for the ten country-specific number of total errors were significant (see Table 3). In all countries, the number of total errors decreased linearly as a function of age. The number of total errors for Chile, Cuba, Peru, and Spain were affected by a quadratic age effect. Children from Cuba, Ecuador, Mexico, Paraguay, and Spain whose parent(s) had a MLPE >12 years made lower number of total errors than children whose parent(s) had a MLPE ≤12 years. Child’s sex affected the number of total errors for Ecuador such that boys make fewer errors than girls. The amount of variance these predictors explained in the number of total errors ranged from 3.6% (in Puerto Rico) to 30.5% (in Cuba).

The assumptions of multiple linear regression analysis were met for all final models. There was not multicollinearity (the VIF values were below 10; VIF ≤1.048; collinearity tolerance values did not exceed the value of 1) or influential cases (the maximum Cook’s distance value was 0.242 in a F_(3,384) distribution which correspond to percentile 13).

3.4 Normative procedure

Norms (e.g., a percentile score) for the different M-WCST scores by country were established using the four-step procedure described in the statistical analysis section. An example will be provided to facilitate an improved understanding of the procedure used to obtain the percentile associated with a score on this test. Thus, let us assume we need to find the percentile score for a 16-year-old Spanish girl who scored a 9 on the M-WCST number of total errors and whose parent(s) have a mean of 8 years of education (MLPE). The steps to obtain the percentile for this score are: 1) Find Spain in Table 3, which provides the final regression models by country for the M-WCST number of total errors. Use the B weights to create an equation that will allow you to obtain the predicted M-WCST number of total errors for this girl using the coding provided in the statistical analysis section. The corresponding B weights are multiplied by the centered age (= calendar age – mean age in the Spanish sample which is equal to 11.3 years), centered age² (= calendar age – mean age in the Spanish sample which is equal to 11.3 years)², and MLPE which was coded based on the 12 years of education threshold. Sex was not a significant predictor, thus, it is not included in the model. See Rivera and Arango-Lasprilla (2017) to determine the mean age of each country’s sample. Then the result is added to the constant generated by the model in order to calculate the predicted value.

In the case of the Spanish girl, the M-WCST predicted number of total errors would be calculated using the following equation: ${\hat{y}}_{i} = 10.201 + [- 1.238 \cdot ({Age}_{i} - 11.3)] + [0.141 \cdot {({Age}_{i} - 11.3)}_{i}^{2}] + (- 1.936 \cdot {MLPE}_{i})$ . The girl’s age is 16. The MLPE (8 years) is split into either ≤12 years (and assigned a 0) or more than 12 years (and assigned a 1) in the model. Since the parent(s) of the hypothetical child in the example have 8 years of education, the MLPE value is 0. Thus, the predicted value equation is: ${\hat{y}}_{i} = 10.201 + [- 1.238 \cdot (16 - 11.3)] + [0.141 \cdot {(16 - 11.3)}^{2}] + (- 1.936 \cdot 0) = 10.201 + (- 5.819) + 3.115 = 7.497$ .

2) In order to calculate the residual value (indicated with an e_i in the equation), we subtract the actual M-WCST number of total errors (she scored 9) from the predicted value we just calculated ( $e_{i} = y_{i} - {\hat{y}}_{i}$ ). In this case, it would be e_i = 9 -7.497 = 1.503. 3) Next, consult the SD_e column in Table 3 to obtain the country-specific SD_e (residual) value. For Spain, it is 8.397. Using this value, we can transform the residual value to a standardized z score using the equation z_i = e_i/SD_e. In this case, we have 1.506/8.397 = 0.179. This is the standardized z score for a 16-year-old Spanish girl who scored a 9 on the M-WCST number of total errors who has parent(s) with 8 years of education (MLPE). 4) The last step is to use the tables available in most statistical reference books (e.g., Strauss, Sherman, & Spreen, 2006) or use a trusted online calculator like the one available at http://www.measuringu.com/pcalcz/. In the online calculator, you would enter the z score and choose a one sided test and note the 100-percent of area (in case of perseverations and total errors) or the percent of area (in case of categories) after hitting the submit button to convert z scores to percentiles. In this example, the z score (probability) of 0.179 corresponds to the 43 percentile. It is important to remember to use the appropriate tables that correspond to each score (number of categories, perseverations, and total errors scores) when performing these calculations.

3.5 User-friendly normative data

The four-step normative procedures explained above offers the clinician the ability to determine an exact percentile for a child who has a specific score on the M-WCST number of total errors. However, this method can be prone to human error due to the number of required computations by hand. To enhance user-friendliness, the authors have completed these steps for a range of raw scores based on age, sex, and MLPE and created tables for clinicians to more easily obtain a percentile range/estimate associated with a given raw score on this test. These tables are available by country and type of test in the Appendix. In order to obtain an approximate percentile for the above example (converting a raw score of 9 on the M-WCST number of total errors for a Spanish girl who is 16 years old and whose parent(s) have 8 years of education) using the simplified normative tables provided in the Appendix, the following steps must be followed. (1) First, identify the appropriate table ensuring the appropriate country and score (number of categories, perseverations, and total errors scores). In this case, the table for M-WCST number of total errors for Spain can be found in Table A31. (2) Next, the table is divided based on MLPE (≤12 vs. more than 12 years of education). Since the parent(s) had 8 years of education, we will use the bottom section of the table for ≤12 years of MLPE. (3) Find the appropriate age of the child, in this case, 16 years old. (4) Next, look in the 16 years’ age column to find the approximate location of the raw score obtained on the test. Within the 16 years’ column, the score of 9 obtained by this Spanish girl corresponds to an approximate percentile of 40.

The percentile obtained using this user-friendly table sometimes could be slightly different than the hand-calculated, more accurate method (43 vs. 40) because the user-friendly table is based on a limited number of percentile values. Individual percentiles cannot be presented in these tables due to space limitations. If the exact score is not listed in the column, you must estimate the percentile value from the list of raw scores available.

4 Discussion

The WCST is a classic neuropsychological test used to measure executive functions such as planning, abstract reasoning, cognitive flexibility, concept formation, and perseverative responses (Nelson 1976; Tchanturia et al., 2012). In Latin America and Spain, the WCST is considered one of the 10 most commonly used neuropsychological tests within clinical practice (Arango-Lasprilla, Stevens, Morlett Paredes, Ardila, & Rivera, 2016; Olabarrieta-Landa et al., 2016). The modified version (M-WCST) may be a more suitable version for use with children and adolescents when compared to the classic WCST as the M-WCST is simpler, less stressful, requires less time for administration, and reduces the risk of the child refusing to perform the task (Cianchetti et al., 2007).

Recently, normative data have been obtained for the M-WCST for adult population in 12 Latin American countries (Arango-Lasprilla et al., 2015; Rivera et al., 2015), and Spain (del Pino, Peña, Ibarretxe-Bilbao, Schretlen, & Ojeda, 2016). However, as noted by Caffarra, Vezzadini, Dieci, Zonato, & Venneri (2004), there are few normative data for children and adolescents. The only normative study performed in Italian children was that of Cianchetti et al. (2007).

Therefore, the purpose of this study was to obtain normative data for the M-WCST in children and adolescents from nine Latin American countries (Chile, Cuba, Ecuador, Guatemala, Honduras, Mexico, Paraguay, Peru, and Puerto Rico) and Spain. Results revealed that several variables were associated with the performance of this test such as age, quadratic age, sex and MLPE. In general, we found that the final regression models explained between 3.0% and 41.1% of the variance for number of categories, between 4.0% and 24.4% of the variance for the total number of perseveration errors, and between the 3.6% and 30.5% of the variance for total number of errors.

Regarding the demographic variables that are related to test performance, age was significantly related to the total number of categories completed, total number of errors, and total number of perseverative errors. In addition, total categories completed increased in a linear fashion as children become older, while the total number of errors and perseverative errors decreased. This pattern of performance for both categories and errors has also been reported in normative studies with adults (Caffarra et al., 2004) and children (Cianchetti et al., 2007).

A curvilinear effect was also observed for the total number of categories, total number of errors and total number of perseverative errors scores. The curvilinear effect in the total categories was found for participants drawn from Chile, Cuba, Ecuador, and Spain. In these countries, the total number of categories increased prominently between the ages of 6 and 14, and after 15 this increase gradually declined. Cianchetti et al. (2007) also reported that the most significant increase in performance in the total number of categories was observed when comparing younger children with older youth, where performance flattening was observed. According to these authors, children between 12– 13 years of age performed similarly to adults (Cianchetti et al., 2007).

The curvilinear effect for perseverative errors was found for Chile, Cuba, Mexico, and Spain. For these countries, there was a prominent decrease in the total number of perseverations until the age of 13, followed by a slow decrease in perseverative errors in subsequent years, except for Chile and Spain in which the total number of perseverative errors increase slightly from 15 to 17 years old. With respect to total number of errors, a curvilinear effect of age was observed for Chile, Cuba, Peru, and Spain, where the total number of errors reduced prominently approximately until the age of 14, while after the age of 15 this decrease stabilizes, except for Chile and Spain in which the total number of perseverative errors increase slightly from 15 to 17 years old and from 16 to 17 years old respectively.

In this study, the MLPE was associated with the total number of categories in Ecuador, México, and Spain. The MLPE was also associated with the total number of perseverative errors in Mexico and Paraguay, and with the total number of errors in Cuba, Ecuador, Mexico, Paraguay, and Spain. With respect to the total number of categories, it was observed that children whose parent(s) had more than 12 years of education completed more categories than children whose parent(s) had less than 12 years of education. However, children whose parent(s) had more than 12 years of education made fewer errors than those whose parent(s) had an education less than 12 years. Parents’ education is not often taken into account in normative studies, but child development studies have already shown that parents’ education influences children’s cognitive development (Schady, 2011).

Sex was not associated with total number of categories, or total number of perseverative errors in any of the countries studied. These results are similar to those reported in other studies in which sex was not found to be associated with the performance of this test (Caffarra et al., 2004; Cianchetti et al., 2007). However, sex was related to the total number of errors in Ecuador, in that boys presented more errors than girls.

4.1 Implications

The results of this study are of great clinical relevance, since having validated and standardized tests to reliably measure problems of executive functioning in children and adolescents is extremely important for clinicians and researchers. Changes in executive functions are common in many developmental disorders such as autism (Robinson, Goddard, Dritschel, Wisley, & Howlin, 2009), learning disorders (Toll, Van der Ven, Kroesbergen, & Van Luit, 2011), attention-deficit hyperactivity disorders (Barkley, 1997), cerebral palsy (Bodimeade, Whittingham, Lloyd, & Boyd, 2013), and many others, such that the above data provide a highly useful tool to professionals working with these populations. Similarly, executive functions have been found to be one of the most sensitive functions to acquired brain injury in the pediatric population (Levin & Hanten, 2005) and appear to have a mediating effect on children’s adaptive behavior (Shultz et al., 2016).

4.2 Limitations

Despite the wide-ranging implications of this study, as it is the largest in the world on validation and standardization of the M-WCST with Spanish speaking populations, results should be interpreted in the light of the following limitations: This study presents normative data of the M-WCST for nine countries in Latin America and Spain. For this reason, it is not advisable to use these norms in the pediatric population of those Spanish-speaking countries where the study was not performed. Future studies should be conducted to standardize this test in other Spanish-speaking countries.

Although the norms of the present study could be used by neuropsychologists in other countries to evaluate Spanish-speaking immigrant children from the countries where the sample was collected for this study, we advise caution since other variables such as level of acculturation, bilingualism, the number of years living in the country, and so on, could influence performance. In addition, the quality of education of both the children and parent(s) is another aspect that may influence the cognitive performance of children.

It is very important to keep in mind that no clinical diagnosis should be made based solely on the M-WCST scores. This test should be integrated as part of a larger battery that evaluates executive functions in more detailed. Because there are a limited number of tests and norms in Latin America and Spain to evaluate this cognitive function, more efforts should be made in the future to have other similar tools.

Although the size of the sample was adequate in each of the countries where the study was conducted, it is very important to note that only the sample in Chile, Mexico, Paraguay, Puerto Rico, and Spain was obtained from several regions of the country, while in the remaining countries were collected from only one geographic area. Future studies should expand the sample in other geographical areas of these countries to enhance representativeness of the sample.

The children who participated part in the present study speak Spanish as their first language. Although Spanish is the first language of the majority of the population in Latin America and Spain, it is important to keep in mind there is a great cultural and linguistic richness in this region, to the point that in many of these countries, sometimes the first language of many children may be different from Spanish (e.g., Portuguese, Euskera, Catalan, Guaraní, Maya, Quechua). For this reason, caution should be used when using these norms in children whose first language is not Spanish.

Finally, it is important to keep in mind that the present study was performed with a normal healthy population. Therefore, future studies should be performed with clinical populations to establish the sensitivity and specificity of this M-WCST among children and adolescents.

5 Conclusion

The M-WCST is one of the neuropsychological tests currently used to evaluate planning, abstract reasoning, cognitive flexibility, concept formation, and perseverative responses in children and adolescents in Latin America and Spain. In this study, which was the first to collect normative data for Spanish-speaking children and adolescents, results revealed that older age, and higher MLPE influenced M-WCST scores. As a result, these variables should be taken into account when interpreting children’s results on this test. These norms presented above will provide neuropsychologists on these countries a more accurate interpretation of the M-WCST scores when used in assessment of executive functions of pediatric populations.

Conflict of interest

None to report.

Footnotes

The Appendix tables are available in the electronic version of this article: .

References

Aiken

L. S.

, & West

S. G.

(1991), Multiple regression: Testing and interpreting interactions. Newbury Park, CA:Sage.

Arango-Lasprilla

J. C.

, Rivera

, Longoni

, Saracho

C. P.

, Garza

M. T.

, Aliaga

, … & Schebela

(2015). Modified Wisconsin card sorting test (M-WCST): Normative data for the Latin American Spanish speaking adult population. NeuroRehabilitation, 37(4), 563–590.

Arango-Lasprilla

J. C.

, Stevens

, Morlett Paredes

, Ardila

, & Rivera

(2016). Profession of neuropsychology in Latin America. Applied Neuropsychology: Adult, 24(4), 318–330. doi: 10.1080/23279095.2016.1185423

Barkley

R. A.

(1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121(1), 65.

Berg

E. A.

(1948). A simple objective technique for measuring flexibility in thinking. Journal of General Psychology, 39(1), 15–22.

Bodimeade

H. L.

, Whittingham

, Lloyd

, & Boyd

R. N.

(2013). Executive functioning in children with unilateral cerebral palsy: Protocol for a cross-sectional study. BMJ Open, 3(4), e002500.

Brown

, Sherbenou

R. J.

, & Johnsen

S. K.

(2009), Test de inteligencia no verbal TONI-2. Madrid:TEA ediciones.

Caffarra

, Vezzadini

, Dieci

, Zonato

, & Venneri

(2004). Modified card sorting test: Normative data. Journal of Clinical And Experimental Neuropsychology, 26(2), 246–250.

Cianchetti

, Corona

, Foscoliano

, Contu

, & Sannio-Fancello

(2007). Modified Wisconsin Card Sorting Test (MCST, MWCST): Normative Data in Children 4–13 Years Old, According to Classical and New Types of Scoring. The Clinical Neuropsychologist, 21(3), 456–478.

10.

Cook

R. D.

(1977). Detection of influential observation in linear regression. Technometrics, 19(1), 15–18. doi: 10.2307/1268249

11.

Del Pino

, Peña

, Ibarretxe-Bilbao

, Schretlen

D. J.

, & Ojeda

(2016). Test modificado de clasificación de tarjetas de Wisconsin: Normalización y estandarización de la prueba en población española. Neurología, 62(5), 193–202.

12.

Diforio

, Walker

E. F.

, & Kestler

L. P.

(2000). Executive functions in adolescents with schizotypal personality disorder. Schizophrenia Research, 42(2), 125–134.

13.

Esposito

, Antinolfi

, Gallai

, Parisi

, Roccella

, Marotta

, & & Carotenuto

(2013). Executive dysfunction in children affected by obstructive sleep apnea syndrome: An observational study. Neuropsychiatr Dis Treat, 9, 1087–1094.

14.

Grant

D. A.

, & Berg

E. A.

(1948). A behavioral analysis of degree of reinforcement and ease of shifting to new responses in a Weigltype card sorting problem. Journal of Experimental Psychology, 38(4), 404–411.

15.

Helland

, & Asbjørnsen

(2000). Executive functions in dyslexia. Child Neuropsychology, 6(1), 37–48.

16.

Igarashi

, Oguni

, Osawa

, Awaya

, Kato

, Mimura

, & Kashima

(2002). Wisconsin card sorting test in children with temporal lobe epilepsy. Brain and Development, 24(3), 174–178.

17.

Kovacs

(1992), Children’s depression inventory, North Tonawanda, NY: Multi-Health System.

18.

Kutner

M. H.

, Nachtsheim

C. J.

, Neter

, & Li

(2005), Applied linear statistical models (5th ed.). New York: McGraw Hill.

19.

Levin

H. S.

, & Hanten

(2005). Executive functions after traumatic brain injury in children. Pediatric Neurology, 33(2), 79–93.

20.

Morgan

A. B.

, & Lilienfeld

S. O.

(2000). A meta-analytic review of the relation between antisocial behavior and neuropsychological measures of executive function. Clinical Psychology Review, 20(1), 113–136.

21.

Nelson

H. E.

(1976). A modified card sorting test sensitive to frontal lobe defects. Cortex, 12(4), 313–324.

22.

Olabarrieta-Landa

, Caracuel

, Pérez-García

, Panyavin

, Morlett-Paredes

, & Arango-Lasprilla

J. C.

(2016). The profession of neuropsychology in Spain: Results of a national survey. Clinical Neuropsychology, 30(8), 1335–1355.

23.

Ozonoff

(1995). Reliability and validity of the Wisconsin Card Sorting Test in studies of autism. Neuropsychology, 9(4), 491.

24.

Rivera

, & Arango-Lasprilla

J. C.

(2017). Methodology for the development of normative data for Spanish Speaking pediatric population. NeuroRehabilitation, 41(3), 581–592.

25.

Rivera

, Olivera Plaza

S. L.

, Quijano

M. C.

, Calderón Chagualá

J. A.

, De los Reyes Aragón

C. J.

, … & Arango-Lasprilla

J. C.

(2015). Datos Normativos del Test Modificado de Clasificación de Tarjetas de Wisconsin (M-WCST) para población colombiana. En Arango-Lasprilla

J. C.

, & Rivera

(Eds.), Neuropsicología en Colombia: Datos normativos, estado actual y retos a futuro. (pg. 123–142). Manizales, Colombia: Editorial Universidad Autónoma de Manizales.

26.

Robinson

, Goddard

, Dritschel

, Wisley

, & Howlin

(2009). Executive functions in children with autism spectrum disorders. Brain and cognition, 71(3), 362–368.

27.

Rockers

, Ousley

, Sutton

, Schoenberg

, Coleman

, Walker

, & Cubells

J. F.

(2009). Performance on the Modified Card Sorting Test and its relation to psychopathology in adolescents and young adults with 22q11. 2 deletion syndrome. Journal of Intellectual Disability Research, 53(7), 665–676.

28.

Romine

C. B.

, Lee

, Wolfe

M. E.

, Homack

, George

, & Riccio

C. A.

(2004). Wisconsin Card Sorting Test with children: A meta-analytic study of sensitivity and specificity. Archives of Clinical Neuropsychology, 19(8), 1027–1041.

29.

Rosselli

, & Ardila

(2003). The impact of culture and education on non-verbal neuropsychological measurements: A critical review. Brain and Cognition, 52(3), 326–333.

30.

Schady

(2011). Parents’ education, mothers’ vocabulary, and cognitive development in early childhood: Longitudinal evidence from Ecuador. American Journal of Public Health, 101(12), 2299–2307.

31.

Schretlen

(2010). Modified Wisconsin Card Sorting Test: M-WCST. Professional Manual. PAR.

32.

Shultz

E. L.

, Hoskinson

K. R.

, Keim

M. C.

, Dennis

, Taylor

H. G.

, Bigler

E. D.

, & & Yeates

K. O.

(2016), Adaptive functioning following pediatric traumatic brain injury: Relationship to executive function and processing speed.

33.

Strauss

, Sherman

E. M.

, & Spreen

(2006), A compendium of neuropsychological tests: Administration, norms, and commentary. American Chemical Society.

34.

Tchanturia

, Davies

, Roberts

, Harrison

, Nakazato

, Schmidt

, & & Morris

(2012). Poor cognitive flexibility in eating disorders: Examining the evidence using the Wisconsin Card Sorting Task. PloS one, 7(1), e28331.

35.

Toll

S. W.

, Van der Ven

S. H.

, Kroesbergen

E. H.

, & Van Luit

J. E.

(2011). Executive functions as predictors of math learning disabilities. Journal of Learning Disabilities, 44(6), 521–532.