Sage Journals: Discover world-class research

Abstract

Pragmatics refers to the ability to effectively use and interpret language in different contexts. Pragmatic abilities develop and refine through childhood, and they are essential for socialization, academic achievement and wellbeing. The scarcity of assessment tools in this field makes it challenging to provide a comprehensive assessment of pragmatic abilities. The Pragma test, originally developed for Finnish and also adapted into Italian, consists of a battery of tasks assessing children’s pragmatic abilities. In this study, a first evaluation of the psychometric qualities of a Norwegian adaptation of this test is presented. In addition, we investigated pragmatic development between ages 5 and 7, and explored possible gender-based differences. Altogether 119 Norwegian-speaking children participated in the study. The children were tested with the Pragma test and parents completed the Children’s Communication Checklist-2. The children were allocated into three groups: 5-year-olds, 6-year-olds and 7-year-olds. The psychometric qualities of this Norwegian adaptation supported its use as a tool for assessing pragmatics in children aged 5–7. Strong and significant growth in pragmatic competence was observed from age 5–6, subsequently flattening out between age 6 and 7, and gender differences in favour of girls were identified. These findings indicate that pragmatic ability, as measured by the Pragma test, shows similar age effects in a Norwegian setting as in Finnish and Italian contexts, paving the way for further cross-cultural and cross-linguistic studies.

Keywords

Pragmatic abilities assessment The Pragma test children Norwegian

Introduction

Pragmatic ability is essential for children’s socialization, academic achievement and wellbeing as successful communication requires one to go beyond the literal meaning of words and utterances and utilize linguistic, cognitive and social skills to construct meaning (Loukusa et al., 2018; Norbury, 2014). Pragmatics refers to the ability to effectively use and interpret language in different contexts (Levinson, 1983; Turkstra et al., 2017) and the terms pragmatics and social communication are often used interchangeably, referring to similar skills (Murphy et al., 2019). A broad definition of pragmatic competence includes the ability to make inference, initiate conversation, follow the rules for politeness and conversational exchange, provide relevant responses, produce coherent narratives, understand non-literal language and convey information through gesture, facial expressions and prosody (Fujiki & Brinton, 2009; Matthews et al., 2018; Murphy et al., 2019). In this study, the term pragmatics, referring to aspects of verbal as well as non-verbal language, is adopted.

Pragmatic communication skills emerge long before children start talking; as they can follow the gaze of another person and establish joint attention by 3–4 months and start using gestures and vocalizations for requesting, greeting and protesting before their first birthday. Gradually single words and two-word utterances are used to replace or supplement gestures; and by age 2, basic communicative use of language is commonly mastered (Alduais et al., 2022; Turkstra et al., 2017). Between the ages of 4 and 8, the ability to utilize contextually relevant information accelerates (Bosco et al., 2004; Gabbatore et al., 2021). Pragmatic competence continues to develop and refine through childhood and adolescence, making the individual increasingly able to use language as an effective tool for social interaction (Alduais et al., 2022; O’Neill, 2007). Successful communication presupposes the ability to take the perspective of others, and Theory of Mind (ToM), the ability to attribute mental status to other people and behave accordingly, appears to link to pragmatic development, although this relationship is still not fully understood (Bosco et al., 2018; Bosco & Gabbatore, 2017; Hyter, 2017; Loukusa et al., 2017). Traditionally, it has been argued that ToM emerges around the age of 4 and keeps developing during childhood and adolescence (Bosco et al., 2014). However, recently it has been found that even before the age of 2, children may display action expectations congruent with others’ belief when they are tested non-verbally (Grosse Wiesmann et al., 2020).

Being able to reflect upon and provide relevant explanations of one’s own answers is an ability that is connected to pragmatic abilities and that develops over time (Donaldson, 1986; Loukusa et al., 2017). In a study of Finnish children, Loukusa and colleagues (2017) found that even at the age of 8, children were not able to correctly explain more than 80% of their answers on questions targeting pragmatic ability. Similar findings were reported by Gabbatore and colleagues (2021) for Italian children; they found that while 5-year-olds were able to correctly explain 48% of their answers, the corresponding number for 8-year-olds was 78%.

To be able to identify pragmatic difficulties or delays, knowledge of typical pragmatic development is essential. It is well-established that pragmatic problems are core characteristics of a variety of clinical populations, for example, attention deficit hyperactivity disorder (ADHD; Carruthers et al., 2022; W. A. Helland, Helland, & Heimann, 2014), autism spectrum disorder (ASD; Félix et al., 2024; Loukusa et al., 2018), behavioural problems (W. A. Helland, Lundervold, et al., 2014) and hearing impairment (Yoshinaga-Itano et al., 2020). By investigating typical developmental patterns and collecting normative data, better support and individually targeted interventions can be provided for children who struggle (Gabbatore et al., 2019, 2021).

Studies exploring gender-based differences in emerging language skills have commonly reported girls scoring consistently better than boys (Bornstein et al., 2004; Eriksson et al., 2012; T. Helland et al., 2017; Nouraey et al., 2021; Stolt, 2023). However, studies focusing on the role of gender in pragmatic ability are sparse and mixed findings are reported. In a recent Italian study with 4–8-year-old children, Gabbatore and colleagues (2021) found no gender-based pragmatic differences as measured by the Pragma test (Loukusa et al., 2017), confirming previous findings by Longobardi and colleagues (2017) based on parental reports. Likewise, in studies with the Language Use Inventory (LUI; O’Neill, 2007) assessing children aged 18 to 47 months, pragmatic competence did not differ between boys and girls neither in a French (Pesco & O’Neill, 2016) nor in a Norwegian sample (W. A. Helland & Møllerhaug, 2020). However, significant differences in favour of girls have been reported in English (O’Neill, 2007), American (Ash et al., 2017), Polish (Białecka-Pikul et al., 2019) and Dutch (Ketelaars et al., 2010) samples.

The scarcity of assessment tools in this field makes it challenging to provide an accurate and comprehensive assessment of pragmatic abilities (Gabbatore et al., 2019). In Norway, only three tools for the assessment of pragmatics in children exist. These are all based on parent/teacher reports; the Norwegian versions of the Children’s Communication Checklist Second Edition (CCC-2; Bishop, 2011), The LUI (W. A. Helland & Møllerhaug, 2020; O’Neill, 2007) and the pragmatic profile of the Clinical Evaluation of Language Fundamentals, Fourth Edition (CELF-4; Semel et al., 2013). These assessment tools have all been developed in English and adapted to Norwegian. They have been found to provide reliable observations of children’s daily communication; however, the subjectivity of the observer is an obvious limitation. Thus, a structured test, not mediated by parent/teacher observations, presents as a valuable part of a comprehensive assessment of pragmatic competence.

The Pragma test (Loukusa et al., 2017) was originally developed in Finnish for use with children in the age range 4–8 years. Subsequently, an English version has also been developed. The test focuses on children’s abilities to understand others’ intentions and utilize contextual cues, areas that are not fully covered by other instruments. Furthermore, the test reflects real-life-pragmatic demands in a structured test situation and provides quantifiable results of strengths as well as weaknesses (Gabbatore et al., 2021; Loukusa et al., 2017). The Pragma test contains five categories of questions, each targeting different aspects of pragmatic abilities. The categories are termed (1) contextual interference with ToM demand, (2) contextual inference without ToM demand, (3) feelings recognition, (4) relevant language use and (5) false belief.

It is plausible that children growing up with different cultural backgrounds and native languages may have different communicative styles. However, mainly due to lack of appropriate assessment tools, research on the development of pragmatic abilities across different languages is scarce (Gabbatore et al., 2019, 2023). The Pragma test was recently translated into Italian by Gabbatore and colleagues (2021), and their results indicate that the test is also relevant in an Italian cultural context. Translating and adapting an existing assessment tool for use in different languages enables cross-cultural investigation of abilities assessed by the same instrument. To the best of our knowledge, no formal direct test of pragmatic ability is presently available in Norway. Thus, the aims of this study were (1) to make a first evaluation of the psychometric qualities of a Norwegian adaptation of the Pragma test (2) to investigate pragmatic development in Norwegian children aged 5–7 years, (3) to investigate responses by question type and (4) to explore possible gender-based differences in pragmatic abilities.

To assess the concurrent validity of the Pragma test, CCC-2 scores were obtained from a sub-set of children.

Method

Participants

Altogether, 119 Norwegian-speaking 5–7-year-old children (60 girls and 59 boys) took part in this study. The participants were allocated into three groups: 5-year-olds (n = 41), 6-year-olds (n = 39) and 7-year-olds (n = 39). In Norway, this corresponds to the last year in kindergarten, and the first 2 years of elementary school. See Table 1. Out of the 119 participants, 86 had corresponding CCC-2 data.

Table 1.

Participants’ characteristics.

Groups	N	Girls	Boys	Age (years; months)Mean (SD)
5 years	41	18	23	5;3 (0;36)
6 years	39	16	23	6;2 (0;35)
7 years	39	26	13	7;3 (0;32)

Procedure

The study was approved by the Norwegian Centre for Research Data. The participants were recruited from seven kindergartens (last year before starting school) and six elementary schools representing urban as well as rural districts in Vestland county, Norway. The kindergartens and schools distributed information about the study together with a consent form to parents of all children in the selected age range. Those who wanted their children to take part in the study returned their written consent, whereupon they received a copy of the CCC-2 to complete. The children were tested individually by students in speech-language therapy in a separate room in their kindergartens and schools. The test sessions were video and audio recorded for scoring. Initially, we intended to include only children who had Norwegian as their first language. Information on first language is collected as part of the CCC-2. However, checklists were missing for 33 out of the 119 children who were administered the Pragma test, meaning that information on first language was unavailable for a substantial part of the children. In response to this, we ran group comparisons (Student’s independent samples t-test) between the group who did report having Norwegian as their first language and the group with missing data on first language. We compared the groups on the total Pragma score and the explaining answers score, and no significant differences were found on either measure. Therefore, we decided to include all children irrespective of language status. However, children who did not speak Norwegian well enough to complete the assessment, were excluded (n = 1). The benefit of this solution is that it gives an inclusive and ecologically valid sample representative of children attending Norwegian kindergartens/schools.

Instruments

The Pragma test

The Pragma test (Loukusa et al., 2017) consists of a battery of tasks assessing children’s ability to understand contextual meaning, intentions, and relevant use of language. Short scenarios, supported by pictures, figurines and plastic animals, are presented verbally to the children. The Pragma test includes 39 questions in total, classified into five categories (see Table 2 for examples).

Table 2.

Examples of different question types, scenarios, and correct answers.

Question type	Scenario	Question	Example of correct answer
Contextual inference with ToM demand (n = 18)	A friend of dad’s from work is about to visit the Jones family. Adam, the son, gets his big toy train set out and starts to assemble it on the living room floor. His dad enters the room and says, ‘Don’t you think that this isn’t the best time?’[En venn fra jobben til pappa skal komme på besøk til familien Hansen. Sønnen Adam tar ut den store togbanen sin og begynner å sette den sammen på stuegulvet. Faren kommer inn i stuen og sier: «Synes du ikke dette er et dårlig tidspunkt?»]	Why does he say that? How do you know that?^a	Because his friend from work is coming over
Contextual inference without ToM demand (n = 10)	This is a picture of a village on a rainy day. The boy is crossing the street at a zebra crossing (pointing at the boy)[Dette er et bilde av en by en dag det regner. Gutten går over gaten på et fotgjengerfelt (peker på gutten)].	Why does the boy have a book over his head? How do you know that?^a	Because it’s raining
Feelings recognition(n = 5)	Billy has made a drawing of the family home. When his mum sees the drawing, she says,‘This looks like we should hang it on the living room wall’.[Bjørn har laget en tegning av familien sitt hus. Når moren hans ser tegningen, sier hun“Denne må vi henge opp på veggen i stuen”.]	How does Billy feel? How do you know that?^a	He feels good because his drawing will hang on the wall
Relevant use of language (n = 4)	Vera comes up to Maddie and is angry. Maddie has told someone who Vera has a crush on, even though she had promised not to tell anyone. ‘I can’t trust you anymore’, Vera says and adds, ‘I told you not to tell anyone’. Maddie has passed on Vera’s secret and feels really bad about it.[Vera kommer sint bort til Maren. Maren har fortalt hemmeligheten om hvem Vera er forelsket i, selv om hun hadde lovet å ikke si det til noen. «Jeg kan ikke stole på deg mer,» sier Vera og legger til, «Jeg sa du ikke fikk lov til å si det til noen». Maren har fortalt Vera sin hemmelighet videre, og er veldig lei seg for det.]	What could Maddie say to fix the situation?	‘Well, like you could always say you’re sorry’.
False belief (n = 2)	Vera and Tommy are visiting a farm animal park with their dad. Vera and Tommy are going to look at the sheep. Dad says, ‘I’ll go and buy a bottle of water. I’ll catch up with you afterwards’. But when Vera and Tommy go to the sheep pen, it’s empty, so they decide to go and see the cows instead.[Vera, Tommy og faren deres er på tur til en besøksgård. Vera og Tommy skal se på sauene. Faren sier: «Jeg går og kjøper en flaske vann. Jeg møter dere etterpå». Men når Vera og Tommy kommer til sauegården er den tom, så de bestemmer seg for å gå og se på kyrne i stedet.]	Where will dad look for Vera and Tommy?	From there (pointing at the empty pen)

Source: Adapted from Loukusa et al. (2017) with permission. [ ] = Norwegian version.

The explanation question was only asked if a correct answer was given.

Contextual interference with ToM demand

These questions (n = 18) assess the child’s ability to derive conclusions by connecting information from different sources (e.g. verbal or visual information, in combination with world knowledge), and to consider others’ mental states and emotions in this process.

Contextual inference without ToM demand

These questions (n = 10) focus primarily on the ability to derive conclusions by connecting information from different sources.

Feelings recognition

These questions (n = 5) assess the ability to understand feelings based on primarily verbally given contextual cues.

Relevant language use

These questions (n = 4) focus on the child’s ability to understand norms regulating language use; requiring the child to utilize world knowledge as well as contextual and social cues.

False belief

These questions (n = 2) evaluate the child’s basic ability to understand the mental states of others.

To answer correctly, the children need to understand the implied meaning of the utterances. Each question is scored as either 1 or 0. A score of 1 requires that the answer reflects an accurate understanding of the situation and addresses the targeted pragmatic aspect, failing this; a score of 0 is assigned. The maximum score equals the number of questions, leading to a Pragma maximum score of 39. In addition, 13 of the questions require the children to explain how they arrived at their correct answer. Guidelines for scoring the answers are presented in the manual (Loukusa, 2019). Depending on the performance of the individual child, it took approximately 30–45 minutes to administer the Pragma test. The test presents with good psychometric properties. Internal consistency values with Cronbach’s alpha of 0.927 and 0.794 are reported for answers and explanations, respectively. Interrater reliability (intra-class correlation coefficient, ICC) of 0.985 (answers) and 0.984 (explanations) indicates highly reliable scoring (Gabbatore et al., 2021; Loukusa et al., 2017).

Norwegian translation and adaptation of the Pragma test

Written permission for translating and adapting the English version of the Pragma test into Norwegian was given by the developer, Soile Loukusa. To provide an equivalent version of a test developed in one language into a different language, merely translating the test would not be sufficient; a more extensive adaptation process would usually be required. A two-way translation procedure in line with the guidelines given by the ‘ITC Guidelines for Translating and Adapting Tests’ (2018) was followed. A panel consisting of students in speech-language therapy (who were also educated teachers and linguists) and professors in speech-language therapy collaborated on the Norwegian translation. Only minor modifications were made to adapt the Pragma test to a Norwegian context; item 1 referring to Santa’s village in Lapland was substituted with the zoo in Kristiansand, which is more familiar to Norwegian children, and the characters were given more common Norwegian names. No modifications were made to the other test material. The Norwegian translation was back-translated into English by a native English-speaking speech-language therapist who is also fluent in Norwegian. The English back-translation, together with the Norwegian translation, was reviewed by the developer Soile Loukusa to verify that the translation corresponded to the original Finnish version. After a few minor adjustments, the final Norwegian version was approved.

The CCC-2

The CCC-2 (Bishop, 2011; Norwegian version) was used as a gold-standard in analyses of concurrent validity. The CCC-2 is a widely used parent-completed checklist, designed to screen for children at risk for language impairment and to give a quantitative estimate of pragmatic/social interactional problems in children. The CCC-2 assesses children’s communicative abilities in everyday contexts, which may differ compared to formal test situations. The instrument is standardized for the age range 4–16 years. Internal consistency for the Norwegian version ranging from 0.73 to 0.89 (Cronbach’s α) and inter-rater reliability ranging from 0.44 to 0.76 are reported (W. A. Helland et al., 2009). The checklist consists of 70 items organized into 10 subscales: (a) Speech, (b) Syntax, (c) Semantics, (d) Coherence, (e) Inappropriate initiation, (f) Stereotyped language, (g) Use of context, (h) Non-verbal Communication, (i) Social relationships and (j) Interests. Scale ‘a’ through ‘d’ assess mainly structural aspects of language (however scale ‘d’ might be considered to measure both structural and pragmatic aspects of language), scale ‘e’ through ‘h’ assess pragmatics, and scales ‘i’ and ‘j’ assess autistic traits. The raw scores are converted into scaled scores with a mean of 10 and an SD of 3 with higher scores indicating better competence. A General Communication Composite (GCC), encompassing both structural and pragmatic language difficulties, is derived by summing the scaled scores of subscales ‘a’ through ‘h’. In addition, although not included as a standard part of the CCC-2, a pragmatic composite (PC) may be calculated by summing the scaled scores of scales ‘d’ through ‘h’ (W. A. Helland & Helland, 2017).

Statistical analyses

The internal consistency of the Pragma test was measured by Cronbach’s alpha. Alpha scores between 0.70 and 0.80 are considered acceptable, with scores above 0.80 indicating a highly internally consistent scale (Tavakol & Dennic, 2011). To check the reliability of scoring, the ICC between scores assigned by two independent raters was calculated on a subsample of 24 randomly selected participants (20.2% of the total sample). An ICC between 0.50 and 0.75, between 0.75 and 0.90 and greater than 0.90 are considered moderate, good and excellent reliability, respectively (Koo & Li, 2016). Two-tailed correlation analysis (Pearson’s r) was used to assess concurrent validity between the Pragma test and the CCC-2. We correlated the Pragma total score with the GCC as well as with each of the 10 subscales measuring various aspects of communication. To investigate whether pragmatic competence developed with age, the Pragma total score as well as the scores related to each category were analysed by one-way analysis of variance (ANOVA) with Fisher’s least significant difference test (LSD) as post hoc test. Effect sizes were measured by eta square with values 0.01 to 0.05 considered small, values 0.06 to 0.13 considered medium and values 0.14 and above considered large. Gender differences were explored by Student’s independent samples t-tests. All analyses were run using SPSS version 28.

Results

Internal consistency and inter-rater reliability

The Norwegian version of the Pragma test showed a Cronbach’s alpha of 0.857 for the Pragma total score. The ICC between two independent raters measured on a subsample (n = 24) was 0.988 for the Pragma total score and 0.919 for the explanation score.

Concurrent validity

The Pragma total score correlated significantly with the composite scores of the CCC-2; GCC and PC with r = 0.360, p < 0.001 and r = 0.255 p < 0.05, respectively. Significant correlations with r ranging from 0.224 to 0.384 were also found for the subscales measuring speech, syntax, semantics, coherence, use of context and interests. See Table 3 for a more detailed overview.

Table 3.

Correlations between Pragma and CCC-2.

	CCC-2	Pearson’s r
Pragma (total)	A. Speech	0.348**
	B. Syntax	0.242*
	C. Semantics	0.253*
	D. Coherence	0.224*
	E. Inappropriate initiation	0.184
	F. Stereotyped language	0.124
	G. Use of context	0.299**
	H. Non-verbal communication	0.191
	I. Interests	0.140
	J. Social relations	0.281**
	General Communication Composite (GCC)	0.360**
	Pragmatic composite (PC)	0.255*

CCC-2: Children’s Communication Checklist Second Edition.

p < 0.01; *p < 0.05.

Performance across age groups on the Pragma test

One-way ANOVA revealed a significant effect of age (F = 31.55, p < 0.001, df =2) in the Pragma total score. When comparing adjacent age groups, the post hoc test (LSD) showed a significant difference (Bonferroni corrected; p < 0.01) between 5- and 6-year-olds. No significant difference was evident between 6 and 7-year-olds. See Figure 1.

Figure 1.

Mean Score of Pragma Total in the Age Groups 5, 6 and 7 Years.

Comparison of question types

One-way ANOVA showed a significant effect of age on four out of five question types: contextual interference with ToM demand (F = 32.41, p < 0.001, df =2), contextual inference without ToM demand (F = 16.56, p < 0.001, df = 2), feelings recognition (F = 8.76, p < 0.001, df = 2) and relevant language use (F = 5.81, p < 0.01, df = 2). As to false belief a ceiling effect was evident, and no significant effect of age was found. The post hoc test (LSD) showed that when comparing adjacent age groups there was a significant difference between 5-year-olds and 6-year-olds on the following tasks: contextual interference with ToM demand, contextual inference without ToM demand, feelings recognition, and relevant language use. As to false belief, no group differences were evident. No significant differences were found between 6-year-olds and 7-year-olds. Effect sizes ranged from small on false belief to large on contextual interference with ToM demand, contextual inference without ToM demand and Pragma total. See Table 4. The question types differed with respect to level of difficulty. Based on the percentage of correct answers within each type of question false belief, contextual inference without ToM demand and feelings recognition turned out to be the easiest ones, while contextual interference with ToM demand and relevant language use were the most demanding ones. See Table 4.

Table 4.

Mean, SD and correct answers (%) for the different question types by age groups.

Question type	Groups									Main effects
	5 years			6 years			7 years			F (2)	η²	Post hoc
	M	SD	%	M	SD	%	M	SD	%	F (2)	η²	Post hoc
ConToM(Max = 18)	5.00	2.86	27.8	8.85	3.07	49.2	10.46	3.44	58.1	32.41**	0.36	5 < 6,7**
ConNoToM(Max = 10)	5.17	1.90	51.0	6.85	1.63	65.5	7.26	1.62	72.6	16.56**	0.22	5 < 6,7**
Feelings (Max = 5)	2.59	1.22	51.8	3.41	0.97	68.2	3.44	0.88	68.8	8.76**	0.13	5 < 6,7**
Relevant (Max = 4)	1.15	0.96	28.8	1.79	1.13	44.8	1.85	0.99	46.3	5.81*	0.09	5 < 6,7**
False(Max = 2)	1.20	0.81	60	1.38	0.78	69.0	1.62	0.63	81	3.16	0.05	5 < 6,7 ns
Pragma total (Max = 39)	15.05	6.34	38.6	22.31	5.26	57.2	24.62	5.19	63.1	31.55**	0.35	5 < 6,7**

ConToM: contextual inference with theory of mind demand; ConNoToM: contextual inference without theory of mind demand; Feelings: feelings recognition; Relevant: relevant language use; False: false belief. Maximum raw score equals the number of items in each question type.

p < 0.01; **p < 0.001.

Performance on the explanation tasks

As mentioned, 13 questions included a follow-up question (given only if the child provided a correct answer) where the children were asked to explain how they arrived at their answer. Analysis of the relative frequency (number of correct explanations/total number of explanations given × 100%) showed that 5-year-olds could successfully explain 21.5% of their correct answers, 6-year-olds 39.2% and 7-year-olds 43.4%. One-way ANOVA showed a significant main effect of age group in the ability to explain answers (F = 18.22, p < 0.00, df 2). When comparing adjacent age groups, post hoc tests (LSD) showed a significant difference between 5- and 6-year-olds while again no significant difference was evident between 6- and 7-year-olds.

Gender-based differences

In the overall sample, independent samples t-tests revealed significant gender differences in the Pragma total score (t = 3.76; p < 0.001, d = 0.690) as well as in the explanation score (t = 2.10; p = 0.04; d = 0.384) with girls performing better than boys. When exploring the results of the separate age groups it was found that in the youngest group (5-year-olds) girls outperformed boys both in the Pragma total score (t = 2.13; p = 0.04, d = 0.670) and the explanation task (t = 2.22; p = 0.03, d = 0.697). No significant gender differences were found in 6-year-olds, while in the oldest group (7-year-olds) girls performed significantly better than boys in the Pragma total score (t = 2.90; p = 0.007, d = 0.968).

Discussion

In sum, the main findings of this study indicate that the Norwegian adaptation of the Pragma test, which was originally developed in Finland, presents with acceptable psychometric qualities confirming its potentials to serve as a valid tool for assessing children’s pragmatic abilities. Strong and significant growth in pragmatic competence was observed from age 5 to 6, subsequently flattening out between age 6 and 7. Furthermore, gender differences in favour of girls were identified.

The alpha value of the Pragma total score showed a highly internally consistent scale comparable to the original Finnish version (Loukusa et al., 2017), thus confirming the reliability of the Norwegian adaptation. To check the reliability of scoring, an ICC between two independent raters was calculated on a subsample of 24 children. The ICC for the Pragma total score and the explanation tasks was 0.988 and 0.919, respectively, indicating that the scoring was highly reliable. These values are also in line with those reported by Loukusa and colleagues (2017).

To derive a measure of concurrent validity for the children’s performance on the Pragma test, the CCC-2, was included in the study as a gold standard. Pearson’s correlations coefficient showed a significant and moderate correlation between the Pragma total score and the GCC of the CCC-2. Furthermore, a significant correlation was also found between the Pragma total score and the PC of the CCC-2. Better performance on the Pragma test corresponded to better performance (or fewer communicative problems) as measured by the CCC-2. These results are in line with the findings reported for the Italian adaptation (Gabbatore et al., 2021) suggesting good sensitivity of the Pragma test in identifying pragmatic difficulties in different native languages and cultural contexts. It should still be mentioned that the Pragma test showed a clearer pattern of correlation with the CCC-2 subscales foremost assessing structural aspects of language than with the subscales assessing pragmatics more exclusively. This could in part be due to the morphosyntactic and semantic demands represented by the Pragma questions. As opposed to the CCC-2, which is filled out by parents, understanding and responding to the items of Pragma requires the child to process the questions themselves. This would logically mean that Pragma should show a direct relationship with the language processing abilities of the child. Another potential reason could be that both the GCC and the Pragma total score essentially conflate subtests which assess different abilities. That is, the GCC combines measures of structural (grammatical and semantic) language and pragmatics, whereas the Pragma total score combines scores on different aspects of pragmatics – which do not necessarily measure the same construct. Nonetheless, our results indicate that the Norwegian version of Pragma is highly internally consistent, which should justify treating it as a single scale. Furthermore, the significant correlation between the PC and the Pragma total score indicates that both are measures of pragmatic language abilities. Importantly, CCC-2 and the Pragma test should be considered complimentary, and not necessarily interchangeable, instruments as they focus on similar but not entirely overlapping pragmatic competencies, and because they assess pragmatic ability from different perspectives (parents’ assessment and formal test performance).

Our results showed a significant effect of age on pragmatic abilities with performance improving with age. When comparing adjacent age groups, the youngest group (5-year-olds) performed significantly poorer than 6-year-olds. A slower but still detectable improvement in pragmatic abilities, with 7-year-olds descriptively performing better than 6-year-olds, was found; however, this improvement did not reach significance. This aligns with the results of Gabbatore et al. (2021) who did not find any significant differences when comparing 6- and 7-year-old Italian children, but contrasts with the findings of Loukusa et al. (2017) who reported significant differences between age 6 and 7 in their Finnish sample. The Pragma total score (mean) of Norwegian children is somewhat below that reported for Finnish (Loukusa et al., 2017) and Italian children (Gabbatore et al., 2021). One might only speculate what could be the reason for this. It might be due to slightly different inclusion criteria; shortcomings in translation, a stricter scoring may have been applied, or it may of course reflect cross-cultural differences in pragmatic development. In a study comparing Finnish, Italian and Canadian children, Mäkinen et al. (2020) found that despite largely similar trends in development across the native languages some differences were observed, for example, in productivity with Italian children being more talkative. Furthermore, at an early age, Finnish children seem to show better performance than English children on some pragmatic tasks (Loukusa et al., 2007). However, the same pattern of performance related to age was found in our Norwegian sample as in the Finnish and Italian samples, which is also consistent with results of previous studies of children’s pragmatic development in different cultural contexts (Angeleri & Airenti, 2014; Białecka-Pikul et al., 2019; Bosco et al., 2018; Glenwright & Pexman, 2010; W. A. Helland & Møllerhaug, 2020; Loukusa & Leinonen, 2008; O’Neill, 2007).

When exploring the effect of age in relation to each question type; contextual interference with ToM demand, contextual inference without ToM demand, feelings recognition, relevant language use and false belief, the same pattern as for the Pragma total score emerged. Except for the false belief task, where no significant differences were found between the age groups, 5-year-olds performed significantly poorer than 6- and 7-year-olds on all question types. The finding of no differences between the age groups in the false belief task is consistent with studies indicating that such tasks are commonly mastered by the age of 4 or even earlier (Grosse Wiesmann et al., 2020; Milligan et al., 2007). Regarding performance in relation to question type, the false belief task also turned out to be the easiest one to understand, followed by feelings recognition and contextual inference without ToM demand, while relevant language use and contextual inference with ToM demand were the most challenging. To perform well on the two latter tasks, children must understand social norms, draw conclusions by combining information derived from different sources, as well as understand and consider the mental states of others. ToM abilities have been found to keep developing during childhood and into adolescence and thus it should be expected that questions requiring these abilities are among the most difficult ones (Bosco et al., 2014). These findings align well with those reported by Loukusa et al. (2017) and Gabbatore et al. (2021) for Finnish and Italian children respectively. Some further adjustments should probably be made to the three final items (all with ToM demand) of the Norwegian adaptation as the meaning of these stood out as challenging to grasp.

In addition to answering pragmatically complex questions, the children’s awareness of how they arrived at their correct answers was also investigated. Five-year-olds explained 21.5%, 6-year-olds 39.5% and 7-year-olds explained 43.3% of their answers correctly. There was a significant difference between 5- and 6-year-old children’s ability to provide appropriate explanations for their answers. Although 7-year-olds were able to explain more correct answers compared to 6-year-olds, this difference did not reach significance. These findings indicate that to verbally formulate how one arrives at an answer requires competence that takes a long time to master (Letts & Leinonen, 2001; Loukusa & Leinonen, 2008). The children in our Norwegian sample successfully explained their correct answers to a lesser extent than Finnish (Loukusa et al., 2017) and Italian (Gabbatore et al., 2021) children. However, the same pattern, showing increasingly better competence as a function of age, especially between the ages of 5 and 6, emerged.

In this study, the exploration of gender-based differences in pragmatic abilities showed that girls outperformed boys. This contrasts with the results of Gabbatore et al. (2021) who found that girls and boys performed equally well on the Pragma test in their Italian sample. However, our findings are in line with other cross-cultural studies reporting significant differences in pragmatic abilities in favour of girls (Ash et al., 2017; Białecka-Pikul et al., 2019; Ketelaars et al., 2010). When inspecting the separate age groups in our sample, it was found that while 5- and 7-year-old girls performed better than boys, no gender-based differences in pragmatic abilities were evident in 6-year-olds.

Some methodological limitations should be considered when evaluating the results of this study. As an inclusive approach was adopted, our sample may be more diverse than the Finnish (Loukusa et al., 2017) and Italian (Gabbatore et al., 2021) ones. Furthermore, our study included children in the age range 5–7 years while the age of children in the aforementioned studies ranged from 4 to 8 years. Thus, future research collecting data from Norwegian 4- and 8-year-olds should be carried out. Preferably, studies should also be carried out to investigate the ability of the Norwegian adaptation to distinguish between typically developing children and clinical samples. Information of parental education level or socioeconomic status (SES) was not collected in this study. However, in Norway, a relatively egalitarian income distribution and a universal social security system contribute to low poverty levels, and consequently significant differences in SES were not expected. Furthermore, pragmatic abilities have not been found to correlate with children’s SES (Gabbatore et al., 2021). The fact that the test manual was not formally translated from Finnish into Norwegian may have led to uncertainties of how to interpret and score some of the answers. This study applied a cross-sectional design, however, to assess within-subject growth in pragmatic competence over time, longitudinal studies are required.

The Norwegian adaptation of the Pragma test derived from the lack of assessment tools for individual assessment of pragmatic ability in children. Combining assessment tools focusing on similar but not fully overlapping aspects, a questionnaire to be completed by parents and a direct test protocol, should contribute to a more accurate and comprehensive assessment of pragmatic abilities in children. In sum, the results of this study support the psychometric qualities of the adaptation as a tool for assessing pragmatics in Norwegian children aged 5–7 years. Furthermore, our findings indicate that pragmatic ability, as measured by the Pragma test, shows similar evidence of age effects in a Norwegian setting as in Finnish and Italian contexts, paving the way for further cross-cultural and cross-linguistic studies. Although further research with larger samples is needed to validate our results and to look further into possible gender-differences, this study contributes with new knowledge of pragmatic development in Norwegian children. Subsequently, this should lead to better support for clinical populations in line with their actual developmental level.

Footnotes

Acknowledgements

First, we want to thank Soile Loukusa for her generous assistance and for giving us permission to adapt the Pragma test into Norwegian. We also thank all children and their parents, kindergartens and schools who participated in the study.

Author contributions

Wenche A. Helland: Conceptualization; Data curation; Formal analysis; Funding acquisition; Investigation; Methodology; Project administration; Supervision; Writing – original draft; Writing – review & editing.

Eline Aaland: Data curation; Formal analysis; Writing – review & editing.

Karoline L. Furebotn: Data curation; Formal analysis; Writing – review & editing.

Junna Nilsen: Data curation; Formal analysis; Writing – review & editing.

Helene Pettersen: Data curation; Formal analysis; Writing – review & editing.

Anne Lise Roe: Data curation; Formal analysis; Writing – review & editing.

Ann Kristin S. Wathne: Data curation; Formal analysis; Writing – review & editing.

Frøydis Morken: Conceptualization; Data curation; Formal analysis; Investigation; Project administration; Supervision; Writing – review & editing.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Norsk Logopedlag (NLL) and the University of Bergen.

Ethical approval

The study was given ethical approval by the Norwegian Centre for Research Data.

ORCID iD

Wenche A. Helland

References

Alduais

Al-Qaderi

Alfadda

(2022). Pragmatic language development: Analysis of mapping knowledge domains on how infants and children become pragmatically competent. Children, 9(9), Article 1407. https://www.mdpi.com/2227-9067/9/9/1407

Angeleri

Airenti

(2014). The development of joke and irony understanding: A study with 3- to 6-year-old children. Canadian Journal of Experimental Psychology / Revue Canadienne de Psychologie Expérimentale, 68(2), 133–146. https://doi.org/10.1037/cep0000011

Ash

A. C.

Redmond

S. M.

Timler

G. R.

Kean

(2017). The influence of scale structure and sex on parental reports of children’s social (pragmatic) communication symptoms. Clinical Linguistics & Phonetics, 31(4), 293–312. https://doi.org/10.1080/02699206.2016.1257655

Białecka-Pikul

Filip

Stępień-Nycz

Kuś

O’Neill

D. K.

(2019). Ratunku! or Just Tunku! Evidence for the reliability and concurrent validity of the Language Use Inventory–polish. Journal of Speech, Language, and Hearing Research, 62(7), 2317–2331.

Bishop

D. V. M.

(2011). The Children’s Communication Checklist Second Edition (CCC-2) [Norwegian version]. Pearson Assessment.

Bornstein

M. H.

Hahn

C.-S.

Haynes

O. M.

(2004). Specific and general language performance across early childhood: Stability and gender considerations. First Language, 24(3), 267–304. https://doi.org/10.1177/0142723704045681

Bosco

F. M.

Bucciarelli

Bara

B. G.

(2004). The fundamental context categories in understanding communicative intention. Journal of Pragmatics, 36(3), 467–488. https://doi.org/https://doi.org/10.1016/S0378-2166(03)00055-9

Bosco

F. M.

Gabbatore

(2017). Theory of mind in recognizing and recovering communicative failures. Applied Psycholinguistics, 38(1), 57–88. https://doi.org/10.1017/S0142716416000047

Bosco

F. M.

Gabbatore

Tirassa

(2014). A broad assessment of theory of mind in adolescence: The complexity of mindreading. Consciousness and Cognition, 24, 84–97. https://doi.org/https://doi.org/10.1016/j.concog.2014.01.003

10.

Bosco

F. M.

Tirassa

Gabbatore

(2018). Why pragmatics and theory of mind do not (completely) overlap. Frontiers in Psychology, 9, Article 1453. https://doi.org/10.3389/fpsyg.2018.01453

11.

Carruthers

Taylor

Sadiq

Tripp

(2022). The profile of pragmatic language impairments in children with ADHD: A systematic review. Development and Psychopathology, 34(5), 1938–1960. https://doi.org/10.1017/S0954579421000328

12.

Donaldson

M. L.

(1986). Children’s explanations: A psycholonguistic study. Cambridge University Press.

13.

Eriksson

Marschik

P. B.

Tulviste

Almgren

Pérez Pereira

Wehberg

Marjanovič-Umek

Gayraud

Kovacevic

Gallego

(2012). Differences between girls and boys in emerging language skills: Evidence from 10 language communities. British Journal of Developmental Psychology, 30(2), 326–343. https://doi.org/https://doi.org/10.1111/j.2044-835X.2011.02042.x

14.

Félix

Santos

M. E.

Benitez-Burraco

(2024). Specific language impairment, autism spectrum disorders and social (pragmatic) communication disorders: Is there overlap in language deficits? A review. Review Journal of Autism and Developmental Disorders, 11(1), 86–106.

15.

Fujiki

Brinton

(2009). Pragmatics and social communication in child language disorder. In Schwartz

(Ed.), Handbook of child language disorders (pp. 406–423). Psychology Press.

16.

Gabbatore

Bosco

F. M.

Mäkinen

Ebeling

Hurtig

Loukusa

(2019). Investigating pragmatic abilities in young Finnish adults using the Assessment Battery for Communication. Intercultural Pragmatics, 16(1), 27–56. https://doi.org/doi:10.1515/ip-2019-0002

17.

Gabbatore

Bosco

F. M.

Mäkinen

Leinonen

Loukusa

(2021). Social-pragmatic contextual comprehension in Italian preschool and school-aged children: A study using the Pragma test. Intercultural Pragmatics, 18(2), 131–162.

18.

Gabbatore

Dindar

Pirinen

Vähänikkilä

Mämmelä

Kotila

Bosco

F. M.

Leinonen

Loukusa

(2023). Silent Finns and talkative Italians? An investigation of communicative differences and similarities as perceived by parents in typically developing children. First Language, 43(3), 313–335.

19.

Glenwright

Pexman

P. M.

(2010). Development of children’s ability to distinguish sarcasm and verbal irony. Journal of Child Language, 37(2), 429–451. https://doi.org/10.1017/S0305000909009520

20.

Grosse Wiesmann

Friederici

A. D.

Singer

Steinbeis

. (2020). Two systems for thinking about others’ thoughts in the developing brain. Proceedings of the National Academy of Sciences of the United States of America, 117(12), 6928–6935. https://doi.org/10.1073/pnas.1916725117

21.

Helland

Jones

L. Ø.

Helland

W. A.

(2017). Detecting preschool language impairment and risk of developmental dyslexia. Journal of Research in Childhood Education, 31(2), 295–311. https://doi.org/10.1080/02568543.2016.1274928

22.

Helland

W. A.

Biringer

Helland

Heimann

(2009). The usability of a Norwegian adaptation of the Children’s Communication Checklist Second Edition (CCC-2) in differentiating between language impaired and non-language impaired 6- to 12-year-olds. Scandinavian Journal of Psychology, 50(3), 287–292. https://doi.org/https://doi.org/10.1111/j.1467-9450.2009.00718.x

23.

Helland

W. A.

Helland

(2017). Emotional and behavioural needs in children with specific language impairment and in children with autism spectrum disorder. The importance of pragmatic language impairment. Research in Developmental Disabilities, 70, 33–39. https://doi.org/10.1016/j.ridd.2017.08.009

24.

Helland

W. A.

Helland

Heimann

(2014). Language profiles and mental health problems in children with specific language impairment and children with ADHD. Journal of Attention Disorders, 18(3), 226–235. https://doi.org/10.1177/1087054712441705

25.

Helland

W. A.

Lundervold

A. J.

Heimann

Posserud

M. B.

(2014). Stable associations between behavioral problems and language impairments across childhood–The importance of pragmatic language problems. Research in Developmental Disabilities, 35(5), 943–951.

26.

Helland

W. A.

Møllerhaug

L. H.

(2020). Assessing pragmatic competence in 18- to 47-months-old Norwegian children. A pilot study with the Language Use Inventory (LUI). RASK, 52, 21–37.

27.

Hyter

(2017). Pragmatic assessment and intervention in children. In Cummings

(Ed.), Research in clinical pragmatics (pp. 493–526). Springer-Verlag.

28.

ITC Guidelines for Translating and Adapting Tests (Second Edition). (2018). International Journal of Testing, 18(2), 101–134. https://doi.org/10.1080/15305058.2017.1398166

29.

Ketelaars

M. P.

Cuperus

Jansonius

Verhoeven

(2010). Pragmatic language impairment and associated behavioural problems. International Journal of Language & Communication Disorders, 45(2), 204–214.

30.

Koo

T. K.

M. Y.

(2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/https://doi.org/10.1016/j.jcm.2016.02.012

31.

Letts

Leinonen

(2001). Comprehension of inferential meaning in language-impaired and language normal children. International Journal of Language & Communication Disorders, 36(3), 307–328. https://doi.org/https://doi.org/10.1080/13682820110045829

32.

Levinson

S. C.

(1983). Pragmatics. Cambridge University Press.

33.

Longobardi

Lonigro

Laghi

O’Neill

D. K.

(2017). Pragmatic language development in 18- to 47-month-old Italian children: A study with the Language Use Inventory. First Language, 37(3), 252–266. https://doi.org/10.1177/0142723716689273

34.

Loukusa

(2019). Pragma-testi. Sosiaalisen ja pragmaattisen kielen ymmärtämisen testi. Niilo Mäki Institutti.

35.

Loukusa

Leinonen

(2008). Development of comprehension of ironic utterances in 3- to 9-year-old Finnish-speaking children. Psychology of Language and Communication, 12(1), 55–69. https://doi.org/doi:10.2478/v10057-008-0003-0

36.

Loukusa

Leinonen

Ryder

(2007). Development of pragmatic language comprehension in Finnish-speaking children. First Language, 27(3), 279–296. https://doi.org/10.1177/0142723707076568

37.

Loukusa

Mäkinen

Gabbatore

Laukkanen-Nevala

Leinonen

(2017). Understanding contextual and social meaning in typically developing Finnish-speaking four-to-eight-year-old children. Psychology of Language and Communication, 21(1), 408–428. https://doi.org/10.1515/plc-2017-0020

38.

Loukusa

Mäkinen

Kuusikko-Gauffin

Ebeling

Leinonen

(2018). Assessing social-pragmatic inferencing skills in children with autism spectrum disorder. Journal of Communication Disorders, 73, 91–105. https://doi.org/10.1016/j.jcomdis.2018.01.006

39.

Mäkinen

Gabbatore

Loukusa

Kunnari

Schneider

(2020). A comparison of picture-based narratives by Finnish, Italian and English-speaking children. Early Education and Development, 31(3), 395–410. https://doi.org/10.1080/10409289.2019.1666446

40.

Matthews

Biney

Abbot-Smith

(2018). Individual differences in children’s pragmatic ability: A review of associations with formal language, social cognition, and executive functions. Language Learning and Development, 14(3), 186–223. https://doi.org/10.1080/15475441.2018.1455584

41.

Milligan

Astington

J. W.

Dack

L. A.

(2007). Language and theory of mind: Meta-analysis of the relation between language ability and false-belief understanding. Child Development, 78(2), 622–646. https://doi.org/https://doi.org/10.1111/j.1467-8624.2007.01018.x

42.

Murphy

Joffe

Messer

Crafter

Radley

Sunthararajah

Bell

Corbacho

Fairhurst

Rodgers

Torgerson

Welch

(2019). Evaluating ‘enhancing pragmatic language skills for young children with social communication impairments’ (E-PLAYS): Protocol for a feasibility randomised controlled trial study. Pilot and Feasibility Studies, 5(1), Article 75. https://doi.org/10.1186/s40814-019-0456-z

43.

Norbury

C. F.

(2014). Practitioner review: Social (pragmatic) communication disorder conceptualization, evidence and clinical implications. Journal of Child Psychology and Psychiatry, 55(3), 204–216. https://doi.org/https://doi.org/10.1111/jcpp.12154

44.

Nouraey

Ayatollahi

M. A.

Moghadas

(2021). Late language emergence: A literature review. Sultan Qaboos University Medical Journal, 21(2), e182–e190. https://doi.org/10.18295/squmj.2021.21.02.005

45.

O’Neill

D. K.

(2007). The language use inventory for young children: A parent-report measure of pragmatic language development for 18- to 47-month-old children. Journal of Speech, Language, and Hearing Research, 50(1), 214–228. https://doi.org/10.1044/1092-4388(2007/017)

46.

Pesco

O’Neill

(2016). Assessing early language use by French-speaking Canadian children: Introducing the LUI-French. Canadian Journal of Speech-Language Pathology and Audiology, 40(3), 198–217.

47.

Semel

Wiig

E. H.

Secord

W. A.

(2013). Clinical evaluation of language fundamentals-4. Pearson Assessment.

48.

Stolt

(2023). Internal consistency and concurrent validity of the parental report instrument on language in pre-school-aged children – The Finnish Communicative Development Inventory III. First Language, 43(5), 492–515. https://doi.org/10.1177/01427237231167301

49.

Tavakol

Dennic

(2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–56.

50.

Turkstra

L. S.

Clark

Burgess

Hengst

J. A.

Wertheimer

J. C.

Paul

(2017). Pragmatic communication abilities in children and adults: implications for rehabilitation professionals. Disability and Rehabilitation, 39(18), 1872–1885. https://doi.org/10.1080/09638288.2016.1212113

51.

Yoshinaga-Itano

Sedey

A. L.

Mason

C. A.

Wiggin

Chung

(2020). Early intervention, parent talk, and pragmatic language in children with hearing loss. Pediatrics, 146(Suppl. 3), S270–S277.

Investigating pragmatic abilities in 5- to 7-year-old Norwegian children: A study using the Pragma test

Abstract

Keywords

Introduction

Method

Participants

Procedure

Instruments

The Pragma test

Contextual interference with ToM demand

Contextual inference without ToM demand

Feelings recognition

Relevant language use

False belief

Norwegian translation and adaptation of the Pragma test

The CCC-2

Statistical analyses

Results

Internal consistency and inter-rater reliability

Concurrent validity

Performance across age groups on the Pragma test

Comparison of question types

Performance on the explanation tasks

Gender-based differences

Discussion

Footnotes

Acknowledgements

Author contributions

Funding

Ethical approval

ORCID iD

References