Sage Journals: Discover world-class research

Abstract

The dance art form involves audiovisual dual modality perception, and research is unclear regarding the original accompanying music, the audience's emotional response to the dance/music, and the impact of dance training on the audiovisual dual modality recognition and experience of emotion in dance. This study is a mixed experimental design that was adopted: 2 (presentation modality: audiovisual bimodal vs. visual modality only) × 2 (dance training experience: dance major vs. non-dance major) × 2 (emotional valence: happiness vs. sadness). The results show that audiovisual dual modality significantly impacts emotional experience but not emotional recognition. Additionally, individuals’ professional experience is critical for dance's emotional recognition and experience. Dancers can recognize and appreciate dance works through a single visual mode, which non-dancers are unable to do. The results provide insights into the relationships between dance training experience, audiovisual dual modality, and dance emotion.

Keywords

audiovisual dual modality dance movement emotion recognition emotion experience emotional valence

Dance is an art form involving audiovisual dual modality (i.e., visual and auditory) perception. It is also the most expressive form of emotional body language (Wigman, 1966), capable of expressing and communicating different emotions. Spectators visually perceive (visual modality) the images a dancer expresses, triggering emotions in the spectator (Langer, 1986; Ping, 2004).

Extensive research has been conducted on emotion in dance. However, most studies have focused on a single modality (Camurri et al., 2013). Specifically, only the visual perception of emotions expressed by bodily gestures and movements in dance has been investigated. For example, Zhang (2017) found that dance could promote positive emotions in the audience. Sawada et al. (2003a) found that bodily movements with different kinematic characteristics could express various emotions—slow movements indicate sadness, and rapid movements indicate anger (Sawada et al., 2003b). Using muted ballet videos, Christensen et al. (2016) investigated the emotional experiences that bodily movements elicit in viewers and found that the roundedness of movements was related to positive emotions and that dance triggered affective experiences in spectators by eliciting autobiographical memories. Van Meel et al. (1993) investigated the recognition of emotion in dance in children aged 5–12 years and found that five-year-old children performed more poorly in emotion recognition than other children and that children with and without dance training experience did not differ significantly in emotion recognition.

Compared with dance emotion, the research on music emotion is relatively rich. The concept of “musical mood” was proposed by Krumhansl (1997). The theory of emotivism defines musical emotion as the real emotion felt/experience experienced by listeners when they hear a piece of music, namely musical emotional experience. Cognitivism of musical emotions believes that music cannot make the audience experience strong emotions. Music can only express emotions as a representation of emotions. It is possible, for example, that a listener can identify the sad feelings that a piece of music is intended to convey but not feel sad when they hear the music. Therefore, cognitivists of musical emotion define musical emotion as listeners’ perception/recognition of the emotion intended to be expressed by music (Evans & Schubert, 2008; Juslin & Laukka, 2004; Lundqvist et al., 2009; Scherer et al., 2019; Zentner et al., 2008). It was also found that musical emotion recognition was stronger and more stable than musical emotion experience in self-reported results, but there was no difference in task difficulty between them (Dibben, 2004; Gabrielsson, 2002; Hunter & Schellenberg, 2010; Krahé et al., 2015; Schubert, 2007; Zentner et al., 2008). Although there are differences between music emotion recognition and music emotion experience, they are also highly correlated. On the one hand, emotional experience is accompanied by emotion recognition, which is mediated/moderated by emotion recognition (Hunter & Schellenberg, 2010). On the other hand, emotion recognition is not a simple cognitive process, and individual emotional experience can be used as effective information to assist emotion recognition (Dibben, 2004; Merrill et al., 2020; Schubert, 2013; Wöllner, 2012).

Dance, as an art form that integrates visual and auditory elements, underscores the importance of exploring embodied cognition theory. According to the embodied cognition paradigm, the environment, body, and cognition are all inextricably linked. Cognitive processing is influenced by the human body's morphological structure, sensory system, motor system, and nerve system (Anderson, 2003; Goldman & de Vignemont, 2009; Wilson, 2002; Ye, 2010). The visual, auditory, and emotional systems’ neurons are activated when a bear roars, according to James-Langer's embodied cognition theory. Then, when the same image is recalled, identical neuronal firing patterns continue even if the original scene is not present. This phenomenon arises because the sense of terror is made up of coordinated reactions in the visual, auditory, and emotional domains when an emotionally charged stimulus, such as a roaring bear, is perceived (Niedenthal, 2007). As a result, embodied emotion is a comprehensive condition that incorporates various senses. A key component of this process is multisensory integration, which is the convergence and interplay of data from several sensory channels. Studies have shown that humans receive information from the external world through multiple sensory modalities, and effective emotion processing through integrating emotional information from different modalities is an important social skill. Multimodal integration, also known as multisensory integration, refers to effectively combining combination cues from visual, auditory, and other modalities into coherent perceptual information (Ernst & Bülthoff, 2004; Kang & Luo, 2020; Lugo et al., 2008; Sun, 2018; Wen et al., 2009). When emotional visual and auditory information presented simultaneously is congruent, the auditory and visual modalities facilitate each other to enable more efficient emotion processing (i.e., the emotional congruent effect; Föcker et al., 2011; Hoenig et al., 2011). Among all sensory channels, visual and auditory stimuli contain the richest emotional information and have received more research attention. Dance is an audiovisual art, and for students majoring in dance, accompanying music is almost indispensable for routine training and performance of dance works. Therefore, music is important to the integrity of dance works and facilitates the appreciation, teaching and learning, and choreography of dance.

At present, few studies have examined the effects of audiovisual dual modality emotional mechanisms on the presentation and interpretation of emotion in dance. Previous studies usually considered dance and accompanying music as independent from each other. For example, it was found that preschool children could perceive emotions in dance movements and express movements of different emotions using the music of different emotions (Boone & Cunningham, 1998). Other studies have explored whether empirical evidence exists on emotional “matching” or emotional “congruence” between music and dance. A similar experiment was conducted by Howlin et al. (2020), who also discussed whether movement emotion matched music emotion. The findings from these studies show that the judgment of emotion perception using only the visual or auditory modality is consistent with that when using both visual and auditory modalities together (Kratus, 1993; Krumhansl & Schenck, 1997). However, when the emotion expressed by the visual and auditory modes was inconsistent, the participants’ pleasure was lowest. In addition, Reason et al. (2016) explored the influence of music on dance experience. Music, according to the results, can help audiences understand dance movements to some extent.

Notably, previous studies focused on the relationship between the audiovisual consistency between dance and music in terms of emotion but did not discuss whether the dance movements or the dance music triggered and affected the audience's emotions in the dance works. In addition, the music used in previous studies was not the original accompanying music (i.e., music composed specially for and matching the emotional characteristics of specific dance works). Using such music might result in a “split” between the music and dance and fail to effectively reveal the emotional expression intended by the dance. Thus, whether participants’ dance training experience affects the audiovisual dual modality recognition and experience of emotion in dance has not yet received research attention.

Based on previous research, this study investigated the accuracy rates and intensities of the recognition and experience of emotion in dance in dance- and non-dance majors using Chinese dance works with the original accompanying music. The study addressed three questions. 1. Can an appreciator recognize and experience the emotions intended by bodily dance movements through only a visual modality? 2. Do emotional valence and dance training experience affect the recognition and experience of emotion in dance?

Methods

Participants

To detect a medium-sized effect (f = 0.25) in a 2 × 2 × 2 mixed-design experiment, aiming for 95% power, at least 36 participants were needed(GPower; Faul et al., 2007). In the present study, college students were recruited as participants (N = 73; M_age= 22.77 years, SD =* 2.41). Among them, 40 (M_age= 22.38 years, SD = 2.16; men, n = 10: women, n = 30) were dance majors and received more than five years of dance study. There were 33 non-dance majors (M_age= 23.24 years, SD = 2.65; men, n = 5; women, n = 28) with no more than one year of learning experience in dance or music. All participants were right-handed and had normal or corrected-to-normal visual acuity and normal hearing. All participants volunteered for the experiment, and none had previously participated in a similar experiment. Students who participated in the experiment were paid 5 USD.

Experimental Design

A mixed experimental design was adopted: 2 (presentation modality: audiovisual bimodal [presence of accompanying music] vs. visual modality only [absence of accompanying music]) × 2 (dance training experience: dance major vs. non-dance major) × 2 (emotional valence: happiness vs. sadness). Presentation modality and emotional valance were within-subject variables. Dance training experience was a between-subject variable. Recognition and experience of emotion in dance (measured by accuracy rate and intensity) were dependent variables.

Experimental Materials

Three college-level dance teachers and the author selected dance videos for a preliminary list. Traditional Chinese dance works were used for evaluation. This type of dance has a distinctive style, strongly reflects traditional Chinese spirit, appeals to Chinese people's traditional aesthetic tastes, and is widely taught and performed in China. Therefore, the dance works selected for this experiment were of this type of dance. Excerpts with high emotional arousal were used as experimental materials. Thirty excerpts, each 30 s in length, were created from these dance videos using the professional video editing software Adobe Premiere Pro (PR). All dance videos were selected from classical theatrical pieces of previous sessions of the top professional dance competitions in China: the “Peach & Plum Cup” Dance Competition and the CCTV Dance Contest (Appendix 1). Among the excerpts, 15 express happiness, and 15 express sadness. All excerpts were converted into MP4 format using the following parameters: video compression: H.264; frame rate: 25 fps; target bit rate: 5.80 Mbps; maximum bit rate: 16.0 Mbp; video frame aspect ratio: 1920 × 1080 (1.0) stereophonic; 44100 Hz. Thirty-eight dance-major participants judged all dance video excerpts for their emotional type; the aforementioned participants didn't take part in the formal experiment. Excerpts in which emotional valence was correctly recognized by less than 80% of the dance majors were removed, leaving a list of 20 excerpts to be used in the subsequent experiment, with 10 for happiness and 10 for sadness (Appendix 2). Muted versions of the excerpts were created using PR software.

Experimental Procedure

The experiment was performed at a standardized behavior laboratory with good equipment and sound effects. E-Prime 2.0 was used for programming and execution. To avoid the effects of audiovisual bimodal information on visual modality, only visual-modality- (without music) excerpts were first presented in random order. Before the formal experiment began, participants were allowed to take part in three trials to familiarize themselves with the experimental procedure.

Participants were first provided the following instruction: “Welcome to the experiment. Next, you will be presented with several dance video clips. After watching each video clip, please judge the valence and rate the intensity of the emotional expressions intended by the video. Please select the valence and rate the intensity of emotion based on your own felt or experienced emotions after watching each video. The intensities of emotion recognition and experience are rated using a five-point scale (1 = very mild, 2 = mild, 3 = moderately intense, 4 = intense, 5 = very intense). Should you have any questions during the experiment, please contact the experimenter. If you have no questions, please press any key to continue.”

At the start of the experiment, a “+” was provided as a fixation point at the center of the screen for 500 ms to remind participants to focus. Participants were prompted to watch a 30 s dance video clip and judge the emotional valence of the video afterward viewing it. In order to distinguish between emotion recognition and emotion experience tasks, in the emotion recognition task, participants were prompted to enter the “1” key if happiness was perceived in a video and the “2” key if sadness was perceived. Participants were not informed whether they had selected the correct or incorrect key. After pressing the selected key, participants were prompted to rate the emotional intensity of the video using the five-point scale described above. Next, participants were prompted to judge the emotional valence they experienced while watching the video. Participants were prompted to press the “F” key for happiness and the “J” key for sadness. Finally, participants were prompted to rate the intensity of their emotional experience while watching each video using the same five-point scale. Participants were allowed to return to a neutral state of mind after watching a video before being prompted to press the space key to watch and evaluate the next video. Figure 1 illustrates the experimental procedure.

Figure 1.
Flowchart of the experimental procedure.

Statistical Analysis

Data from the experiment were sorted and pooled; all trials were used to quantify intensity scores. A three-way (2 [presentation modality] × 2 [emotional valence] × 2 [dance training experience]) repeated measures ANOVA for emotion recognition and experience (accuracy rate and intensity) was performed using SPSS 16.0. a. Computed using alpha = 0.05; **p <* .05; p < .01; p < .001.

Results

Emotion Recognition Accuracy Rate

Dance training experience had no effect, according to F(1,71) = 1.31, p = .257. The emotion recognition accuracy rate in the non-dance-major group was slightly lower than in the dance-major group, although there was no discernible difference (M : 0.96 < 0.98). There was no effect of variable presentation modality, F(1,71) = 2.92, p = .092. The emotion recognition accuracy in the audiovisual bimodal was slightly higher than in visual modality only; however, there was no discernible difference (M : 0.97 = 0.97). The main effect of emotional valence had no effect, F(1,71) = 0.01, p = .906. The recognition accuracy for sadness was slightly lower than for happiness; however, the difference had no effect. (M : 0.97 = 0.97) The interaction between dance training experience and presentation modality was significant, F(1,71) = 4.07, p = .047, η^²= .054 (Figure 2). A simple effect analysis showed that in the non-dance-major group, emotion recognition accuracy in audiovisual bimodal was significantly higher than that in visual modality only (M : 0.98 > 0.95), F(1,71) = 6.34, p = .014, η^²= .082. In the dance-major group, emotion recognition accuracy in audiovisual bimodal was lower than that in visual modality only; however, the difference had no effect (M : 0.98 = 0.98), F(1,71) = 0.05, p = .820. The interaction between emotional valence and dance training experience had no effect, F(1,71) = 0.01, p = .906. The interaction between presentation modality and emotional valence had no effect, F(1,71) = 2.00, p = .162. The interaction between presentation modality, emotional valence, and dance training experience had no effect, F(1,71) = 0.47, p = .495. Table 1 and Table 2 illustrate the data of emotion recognition accuracy.

Figure 2.
Interaction of presentation modality and specialty type on emotion recognition accuracy.

Table 1.
Descriptive Statistics of Emotion Recognition Accuracy of Presentation Modality, Emotion Valence, and Professional Type.

Emotional valence × Presentation modality Major M SD Sample size(n)

Audiovisual bimodal × Sad emotion Non-dance major 0.98 0.07 33

Dance major 0.97 0.14 40

Total 0.97 0.11 73

Audiovisual bimodal × Happy mood Dance major 0.98 0.08 33

Dance major 0.99 0.04 40

Total 0.99 0.06 73

Visual modality only × Sad emotion Non-dance major 0.96 0.15 33

Dance major 0.99 0.04 40

Total 0.97 0.11 73

Visual modality only × Happy mood Non-dance major 0.95 0.09 33

Dance major 0.98 0.08 40

Total 0.96 0.089 73

Table 2.
Three-Factor ANOVAs for Presentation Modality × Emotional Valence × Professional Type.

Source of variation MS df F p-value η^²

Presentation modality 0.01 1 2.92 .092 .040

Presentation modality × Dance major or not 0.02 1 4.07 .047* .054

Emotional valence 0.00 1 0.01 .906 .000

Emotional valence × Dance major or not 0.00 1 0.01 .906 .000

Presentation modality × Emotional valence 0.01 1 2.00 .162 .027

Presentation modality × Emotional valence × Dance major or not 0.00 1 0.471 .495 .007

Note. p < .05; p < .01; p < .001.

Emotion Recognition Intensity

The main effect of dance training experience was significant, F(1,71) = 7.90, p = .006, η^²= .100, with emotion recognition intensity in the non-dance-major group being significantly higher than that in the dance-major group (M: 4.38 > 4.06). The main effect of the modality of presentation had no effect, F(1,71) = 2.78, p = .100. The emotion recognition intensity in audiovisual bimodal was slightly higher than in visual modality only; however, the difference had no effect (M: 4.26 > 4.18). The main effect of emotional valence had no effect, F(1,71) = .920, p = .341. The emotion recognition intensity for sadness was slightly higher than that for happiness; however, the difference had no effect (M: 4.25 > 4.19). The interaction between presentation modality and dance training experience was significant, F(1,71) = 14.66, p < .001, η^²= .171 (Figure 3). A simple effect analysis showed that, in the non-dance-major group, emotion recognition intensity in audiovisual bimodal was significantly higher than that in visual modality only (M: 4.51 > 4.25), F(1,71) = 13.78, p < .001, η^²= .163. In the dance-major group, emotion recognition intensity in audiovisual bimodal was lower than that in visual modality only; however, the difference had no effect (M: 4.01 < 4.11), F(1,71) = 2.58, p = .112. In the audiovisual bimodal, emotion recognition intensity in the non-dance-major group was significantly higher than that in the dance-major group (M: 4.51 > 4.01), F(1,71) = 17.44, p < .001, η^²= .197. In visual modality only, emotion recognition intensity in the non-dance-major group was slightly higher than that in the dance-major group; however, the difference had no effect (M: 4.25 > 4.11), F(1,71) = 1.25, p = .267. The interaction between emotional valence and dance training experience had no effect, F(1,71) = 1.60, p = .210. The interaction between presentation modality and emotional valence was significant, F(1,71) = 10.21, p = .002, η^²= .126 (Figure 4). A simple effect analysis showed that, for sadness, emotion recognition intensity in the audiovisual bimodal was slightly lower than that in the visual modality only; however, the difference had no effect (M: 4.23 < 4.26), F(1,71) = .211, p = .647. For happiness, emotion recognition intensity in the audiovisual bimodal was significantly higher than that in the visual modality only (M: 4.28 > 4.10), F(1,71) = 9.12, p = .004, η^²= .114. In audiovisual bimodal, emotion recognition intensity for sadness was slightly lower than that for happiness; however, the difference had no effect (M: 4.23 < 4.28), F(1,71) = .609, p = .438. In visual modality only, emotion recognition intensity for sadness was significantly higher than that for happiness (M: 4.26 > 4.10), F(1,71) = 5.36, p = .024, η^²= .070. The interaction between presentation modality, emotional valence, and dance training experience had no effect, F(1,71) = 1.63, p = .207. Table 3 and Table 4 illustrate the data on emotion recognition intensity.

Figure 3.
Interaction of presentation modality and specialty type on emotion recognition intensity.

Figure 4.
Interaction of presentation modality and emotional valence on emotion recognition intensity.

Table 3.
Descriptive Statistics of Emotion Recognition Intensity of Presentation Modality, Emotional Valence, and Professional Type.

Emotional valence × Presentation modality Major type M SD Sample size(n)

Audiovisual bimodal × Sad emotion Non-dance major 4.50 0.41 33

Dance major 3.97 0.68 40

Total 4.21 0.63 73

Audiovisual bimodal × Happy emotion Non-dance major 4.52 0.43 33

Dance major 4.05 0.70 40

Total 4.26 0.63 73

Visual modality only × Sad emotion Non-dance major 4.39 0.61 33

Dance major 4.13 0.66 40

Total 4.25 0.65 73

Visual modality only × Happy emotion Non-dance major 4.11 0.69 33

Dance major 4.09 0.52 40

Total 4.10 0.60 73

Table 4.
Three-Factor ANOVAs for Presentation Modality × Emotional Valence × Professional Type.

Source of variation MS df F p-value η^²

Presentation modality 0.45 1 2.78 .100 .038

Presentation modality × Dance major or not 2.38 1 14.66 <.001 .171

Emotional valence 0.22 1 0.92 .341 .013

Emotional valence × Dance major or not 0.38 1 1.60 .210 .022

Presentation modality × Emotional valence 0.78 1 10.21 .002 .126

Presentation modality × Emotional valence × Dance major or not 0.12 1 1.63 .207 .022

Note. p < .05; p < .01; p < .001.

Emotional Experience Accuracy Rate

The main effect of dance training experience was significant, F(1,71) = 3.01, p = .087. The emotional experience accuracy rate in the non-dance-major group was slightly higher than in the dance-major group; however, the difference had no effect (M: 0.96 < 0.98). The main effect of presentation modality was significant, F(1,71) = 3.99, p = .050, η^²= .053, with the emotional experience accuracy rate in audiovisual bimodal significantly higher than in visual modality only (M: 0.98 > 0.96). The main effect of emotional valence had no effect, F(1,71) = 1.53, p = .220. The emotional experience accuracy rate for sadness was slightly higher than that for happiness; however, the difference had no effect (M: 0.98 > 0.97). The interaction between presentation modality and dance training experience had no effect, F(1,71) = 2.64, p = .109. The interaction between emotional valence and dance training experience had no effect, F(1,71) = 0.09, p = .772. The interaction between presentation modality and emotional valence had no effect, F(1,71) = 3.60, p > .05. The interaction between presentation modality, emotional valence, and dance training experience had no effect, F(1,71) = 0.12, p = .731. Table 5 and Table 6 illustrate the data on emotional experience accuracy.

Table 5.
Descriptive Statistics of Presentation Modality, Emotional Valence, and Professional Type on the Correct Rate of Dance Emotional Experience.

Emotional valence × Presentation modality Major type M SD Sample size(n)

Audiovisual bimodal × Sad emotion Non-dance major 0.967 0.07 33

Dance major 0.98 0.08 40

Total 0.98 0.07 73

Audiovisual bimodal × Happy emotion Non-dance major 0.97 0.09 33

Dance major 0.99 0.04 40

Total 0.98 0.07 73

Visual modality only × Sad emotion Non-dance major 0.96 0.13 33

Dance major 1.00 0.03 40

Total 0.98 0.09 73

Visual modality only × Happy emotion Non-dance major 0.93 0.12 33

Dance major 0.97 0.09 40

Total 0.95 0.10 73

Table 6.
Three-Factor ANOVAs for Presentation Modality × Emotional Valence × Professional Type.

Source of variation MS df F p-value η^²

Presentation modality 0.01 1 3.99 .050 .053

Presentation modality × Dance major or not 0.01 1 2.64 .109 .036

Emotional valence 0.01 1 1.53 .220 .021

Emotional valence × Dance major or not 0.00 1 0.09 .772 .001

Presentation modality × Emotional valence 0.02 1 3.60 .062 .048

Presentation modality × Emotional valence × Dance major or not 0.00 1 0.12 .731 .002

Note. p < .05; p < .01; p < .001.

Emotional Experience Intensity

There was no effect of variable dance training experience, F(1,71) = 0.10, p = .753. Emotional experience intensity in the non-dance-major group was slightly lower than that in the dance-major group; however, the difference had no effect (M: 3.95 < 3.98). The main effect of presentation modality was significant, F(1,71) = 30.53, p < .001, η^²= .301, with emotional experience intensity in audiovisual bimodal significantly higher than that in visual modality only (M: 4.13 > 3.81). There was no effect of variable emotional valence, F(1,71) = 2.02, p = .159. Emotional experience intensity for sadness was slightly higher than for happiness; however, the difference had no effect (M: 4.00 > 3.94). The interaction between presentation modality and dance training experience was significant, F(1,71) = 14.66, p < .001, η^²= .665 (Figure 5). A simple effect analysis showed that, in the non-dance-major group, emotional experience intensity in audiovisual bimodal was significantly higher than that in visual modality only (M: 4.46 > 3.44), F(1,71) = 138.09, p < .001, η^²= .660. In the dance-major group, emotional experience intensity in audiovisual bimodal was significantly lower than that in visual modality only (M: 3.80 < 4.17), F(1,71) = 22.27, p < .001, η^²= .239. In audiovisual bimodal, emotional experience intensity in the non-dance-major group was significantly higher than that in the dance-major group (M: 4.46 > 3.80, F(1,71) = 30.52, p < .001, η^²= .301. In visual modality only, emotional experience intensity in the non-dance-major group was significantly lower than that in the dance-major group (M: 3.44 < 4.17), F(1,71) = 47.31, p < .001, η^²= .400. The interaction between emotional valence and dance training experience had no effect, F(1,71) = 0.00, p = .990. The interaction between presentation modality and emotional valence had no effect, F(1,71) = 0.37, p = .543. The interaction between presentation modality, emotional valence, and dance training experience had no effect, F(1,71) = 0.90, p = .345. Table 7 and Table 8 illustrate the data on emotional experience intensity.

Figure 5.
Interaction of presentation modality and specialty type on emotional experience intensity.

Table 7.
Descriptive Statistics for the Intensity of Dance Emotional Experience by Presentation Modality, Emotional Valence, and Professional Type.

Emotional valence × Presentation modality Major type M SD Sample size(n)

Audiovisual bimodal × Sad emotion Non-dance major 4.47 0.41 33

Dance major 3.84 0.67 40

Total 4.12 0.65 73

Audiovisual bimodal × Happy emotion Non-dance major 4.46 0.40 33

Dance major 3.76 0.65 40

Total 4.08 0.65 73

Visual modality only × Sad emotion Non-dance major 3.50 0.56 33

Dance major 4.20 0.51 40

Total 3.88 0.63 73

Visual modality only × Happy emotion Non-dance major 3.38 0.58 33

Dance major 4.15 0.45 40

Total 3.80 0.64 73

Table 8.
Three-Factor ANOVA for Presentation Modality × Emotional Valence × Professional Type.

Source of variation MS df F p-value η^²

Presentation modality 7.61 1 30.53 < .001 .301

Presentation modality × Dance major or not 35.12 1 140.93 < .001* .665

Emotional valence 0.29 1 2.02 .159 .028

Emotional valence × Dance major or not 2.34 1 0.00 .990 .000

Presentation modality × Emotional valence 0.04 1 0.37 .543 .005

Presentation modality × Emotional valence × Dance major or not 0.09 1 0.90 .345 .013

Note. p < .05; p < .01; p < .001.

Discussion

The effects of dance training experience, emotional valence, and presentation modality on the recognition and experience (both measured by intensity and accuracy rate) of emotion in dance were experimentally investigated. The results showed that the main effect of presentation modality on emotion recognition had no effect, whereas that on emotional experience was significant. The main effect of emotional valence on emotion recognition and experience had no effect. The main effect of dance training experience on emotion recognition intensity and emotional experience accuracy rate was significant. The interaction between dance training experience and presentation modality had a significant effect on emotion recognition and experience intensities. The interaction between presentation modality and emotional valence had a significant effect only on emotion recognition intensity.

Visual Modality Only Has the Advantage of Emotion Recognition

The results showed that presentation modality had a nonsignificant main effect on emotion recognition, indicating that music, or audiovisual dual modality emotion processing, had an insignificant advantage in the recognition of emotion in dance. This finding was inconsistent with the author's expectations. Based on findings in the psychology of emotion (Fu, 2016), humans obtain most information through visual stimuli. In emotion recognition, an individual recognizes emotion in dance through facial, postural, and other external manifestations. Several studies have shown that the visual channel dominates, emphasizing the dominance of facial expressions and bodily gestures (Donohue et al., 2013; Gao et al., 2018; Hirst et al., 2018; Jessen & Kotz, 2011; Kret et al., 2013; Strelnikov et al., 2015; Tatz et al., 2020). For all the dance video clips, participants could observe the dancers’ facial expressions, gestures, and movements as well as use only the visual modality to correctly recognize the emotional expression intended by the dance works without musical rendering (i.e., with the auditory mode off). Several dancers (Lv, 2003; Wigman, 1966) have also argued that music enters dance as a temporal factor and mainly manifests in the tempo of dance movements, indicating that dance music is not a key factor in dance emotion recognition. The key factor that determines emotion recognition in dance is still the body movements in dance. Dance is a visual art. We find that in different dance segments, dancers have a common feature; that is, in the performance, dancers will show different emotional expressions through the “speed” and “force” of their movements. For example, when expressing sadness, dancers usually use slow and strong movements, such as “beating their chest and feet,” to express their sadness. When expressing happiness, dancers usually use quick, light movements to express their happiness. This also proves that in dance works, dance movement is the “soul” of dance. Therefore, in terms of emotion recognition, the audiovisual dual modal presentation has no significant difference in emotion recognition. Therefore, how to use body language to accurately convey the emotions to be expressed in the dance work needs the choreographer to constantly explore and think.

Audiovisual Dual Modality Emotion Processing Has Advantages in Emotional Experience Only

In contrast, presentation modality was found in this study to have a significant main effect on emotional experience, indicating that spectators relied more on audiovisual dual modality cues to experience emotion through viewing dance. This is basically consistent with findings from some studies (Hanser et al., 2015; Welch et al., 1986), which have suggested that bimodal (auditory and visual) presentation of affective information leads to better emotional experiences than unimodal (visual) presentation. This phenomenon happens because the individual's emotional reaction is intensified and emotional processing efficiency is improved when emotional input from several sensory channels is coordinated to express the same emotion. This phenomenon is known as the emotional congruence effect (Chen et al., 2016). This explains why participants reported better emotional experiences through audiovisual dual modality perception. In contrast, the audience did not fully experience the emotional sensations that were part of the performance when individuals were exposed to dance in a unimodal visual setting. Due to the visual channel's limitations, spectators could only deduce the feelings by observing the dancers’ facial expressions and body language. Since dance movements are typically refined, processed, or even exaggerated body movements, the audience's perception of the emotional information is reduced when there is no auditory mode. This is due to the complexity and diversity of dance art, even though participants can generally understand the emotions to be conveyed by dance movements. Therefore, when the emotional information presented through a channel is unclear, spectators integrate other modal information to obtain more cues and have a more accurate emotional experience (Welch et al., 1986). Thus, emotional experience in the absence of music was poorer than in the presence of music indicating that audiovisual dual modality affective perception has advantages for the experience of emotion in dance.

These results also indicate that adding music (auditory) to the dance performance that matches the mood of the movement can lead spectators to better experience dance works. In addition, in the dance music composition, whether the dance music composition fits the dance work will affect the overall emotional experience effect of the dance work. We found that in some dance works if the composer could not adequately understand the connotation of the dance or the mood of the dance expressed by the choreographer, even though the dancers’ body movements expressed very strong emotions, the feedback of the subjects on the emotional experience was not ideal. There are only a handful of composers who specialize in dance composition. Most composers have no dance experience, making it necessary to train specialized dance composers or a series of dance professional knowledge-related training for dance music composers. This can help them fully understand different types of dance movements and the emotional connotations expressed by the dance movements, to make dance music suitable for the dance works according to the dance movements themselves, and to achieve the unity of visual and auditory effects.

Audiovisual Dual Modality Emotion Processing Only Has Advantages for Non-Professional Spectators

Notably, the advantages of audiovisual dual modality perception in emotional experience were observed only in non-dance-major participants. This was possibly because of the differences between individual participants. Individual differences in the capacity to perceive emotion-inducing events also determine the final differences in emotional experiences. Individual differences that affect emotion recognition and emotional experiences mainly include differences in age, gender, education, and experience (Fu, 2016). Age and gender were not included in the scope of this research and are not presently considered. In addition, differences in education and experience led to the differences in the experience of emotion in dance.

Studies of emotion in music have shown that musical training experience affects individuals’ musical emotion processing. In particular, musical professionals are capable of more accurately processing musical information (Battcock & Schutz, 2022; Chen & Wang, 2019; Kantor-Martynuska & Horabik, 2015; Sun et al., 2011; Zhu, 2017) and more strongly perceiving emotions in music (Akkermans et al., 2019; Alluri et al., 2015; Chaddock-Heyman et al., 2021; Di Mauro et al., 2018; Ma et al., 2013; Spitzer & Coutinho, 2014; Wang & Liu, 2013). Zhou et al. (2020) found that the musical training experience affected individuals’ emotional experience because those who were not musical professionals lacked relevant musical expertise. Thus, a single modality was inadequate, and assistance from other channels was required. Pollak and Sinha (2002) found that physically abused children exhibited more sensitive recognition of angry expressions, due to their previous experiences with physical punishment.

Similarly, after long-term learning and training in dance, dance students have mastered the well-established dance language system and are more sensitive to the connotations expressed by dance movements (i.e., non-propositional bodily movements), especially stylized movements resulting from long-term evolution that have become the equivalent of “words” commonly used by dancers (Lv, 2003). The connotations of this body language cannot be understood by non-dance-major participants. Thus, with their proficiency and understanding of this body language, dance-major participants could respond accurately to emotion and accurately understand and experience the intended meaning using only information presented visually. This finding has not been reported in previous studies on emotion in dance.

In contrast, their lack of dance training experience prevented non-dance-major participants from fully understanding the internal effects of dance movements. Thus, they needed to resort to dual modality information for assistance. Music provides the most direct means of emotion perception besides dance movements and can effectively render and color dance. The affective and behavioral experience sharing model suggests that, when appreciating a live performance, a spectator imitates the performers’ facial and bodily expressions to acquire an empathetic emotional experience, and the degree of emotion activation is highly related to the degree of synchronization (Ammirante & Thompson, 2012; Livingstone & Thompson, 2009; Molnar-Szakacs & Overy, 2006; Thompson et al., 2008; Wöllner, 2012).

Therefore, audiovisual bimodal presentation of dance performances allows non-dance-major spectators to acquire richer affective information and thus have a more intense emotional experience than would occur if only the visual modality were presented. Therefore, non-dance-major participants better experienced the dance works with accompanying music. However, dance practitioners account for only a small part of the population. The vast majority of dance spectators have received little to no dance training. For these spectators, dance works need the rendering and coloring of music to augment the affective expressions. Hong (2003) and Wang (2006) have argued that dance movements and music constitute an integrated, unified relationship, with dance expressing the music's intentions through bodily movements and music augmenting the dance movements’ tempo and rhythm.

In the past emotional dance therapy, in most cases, patients were made to move their bodies by playing different music so as to achieve emotional regulation (Dieterich-Hartwell, 2019; Punkanen et al., 2014). However, previous studies have overlooked patients who are physically challenged and in dire need of emotional adjustment. This study found that dance videos with music can also play a role in emotional regulation, which could be applied to the mental health treatment or mental health counseling of college students in the future.

Audiovisual Dual Modality Emotion Processing Only Has Advantages in Recognition of Happiness

Notably, the intensity of recognition of emotion in dance was affected by the interaction between emotional valence and presentation modality. More specifically, the intensity of recognition of happiness in dance in the presence of music was significantly higher than in the absence of music. In contrast, the intensity of recognition of sadness in dance in the absence of music was higher than that in the presence of music, although the difference was non-significant. Moreover, in the absence of music, the intensity of recognition of sadness in dance was significantly higher than that of happiness. However, previous studies showed that sad or happy background music led to participants providing higher ratings of sadness or happiness (Logeswaran & Bhattacharya, 2009) or that sad music led to sadder ratings of sad movements, but happy music did not lead to happier ratings of happy movements (Christensen et al., 2014). The findings from this study were inconsistent with the findings of these two previous studies. This inconsistency might be because of two reasons.

The first reason is possibly related to the selection of dance works. The sad dance works were mainly narrative (with a storyline), whereas the happy dance works were mainly expressive (without a storyline). The narrative description of plots in sad dance works presented characters with distinctive characteristics. Therefore, the dance movements were not only pure bodily movements but also had been transformed into narrative body language that more readily elicited empathy from the spectator (Lv, 2003). Thus, presentation modality had an insignificant impact on participants’ emotion recognition. In contrast, the happy dance works mainly presented purely expressive arrangements of bodily movements, as they were intended to express the beauty of bodily movements. Expressive dance movements were not meant to imitate life but rather to vent emotions; thus, they were intended to express deep emotion and did not function as narratives. As a result, expressive dance is less effective in eliciting empathy in spectators than is narrative dance. Therefore, happy dance works need to be accompanied by music to facilitate the elicitation of empathy. This explains why, in the absence of music, the intensity of emotion recognition was significantly higher for sadness than it was for happiness.

Another possible reason could have been the force of the dance movements. The sad dance works selected for this experiment consisted of more forceful bodily movements than the happy ones. For example, the excerpt from Kong Yiji used in the experiment, which comes from the second half of the work, describes the experience and inner feeling of Kong Yiji, the protagonist, after his legs are broken. To present the protagonist's sadness to the audience, the dancer increased the force of his bodily movements and presented matching facial expressions. The Music and Dance volume of the Encyclopaedia of China (1985) defines dance as an art to express internal affection through bodily movements (speed, force, and other factors). Adding music to dance weakens the spectator's ability to capture the emotion intended by the dance movements alone, thereby decreasing the intensity of emotion recognition. Therefore, spectators could prefer it if sad dance works did not have accompanying music. This indicates that, for the recognition of bodily emotional expression, processing bias is not simply negative or positive; rather, it depends on how eye-catching the medium conveying the information is behind the emotional valence (Liang et al., 2019).

Research Limitations and Outlook

In this study, the effects of emotional valence and dance training experience on the recognition and experience of emotion in dance under audiovisual dual modality versus unimodal presentation were investigated. The results could be used to guide dance teaching and learning, choreography, and appreciation. In addition, these results support the use of dance as an operable means of adjuvant emotional therapy for improving psychological health.

However, this study has limitations. First, considering that the judgment of the intended emotion in dance might be affected by the type of dance, video clips of a single style of dance (traditional Chinese dance) were used in the experiment. In the future, more dance styles can be used to validate this study's findings. Second, the participant sample used in this study did not adequately represent the larger population. The participants recruited in this study were college students aged 16–28 years old. The dance-major participants were generally younger than the non-dance-major participants, which might have biased the results. In addition, the sample size was small; thus, whether the findings are applicable to other population groups (e.g., different ages and cultures) or if gender differences exist in the recognition and experience of emotion in dance require further examination. Third, researchers have suggested that the perception of dance languages is not on a single level; some dance languages cannot be expressed by bodily movements but usually can be expressed by facial movements (Lv, 2003). Therefore, further research should investigate the effects of the different parts of the body and facial expressions used by the dancer on emotional perception in dance using only the visual modality. Fourth, we discovered that the color of the dancers’ clothing in the dance videos might have an impact on how emotionally intense some participants felt their emotions. Therefore, we will ask the dancers to wear clothing that is the same color and style in the subsequent experiment. In the end, potential confounding effects of listening to traditional Chinese music were not thoroughly investigated. It is still unknown if cultural familiarity with the music or dancing training influenced the replies.

Conclusions

In this study, the effects of three variables—emotional valence, presentation modality, and dance training experience—on the recognition and experience of emotion in dance were investigated. The results show that audiovisual dual modality does not have advantages in recognition of emotion in dance but does have advantages in the experience of emotion in dance. For an individual's emotional experience, audiovisual dual modality affective perception only has advantages when experiencing positive emotion in dance. Dance training experience leads to better recognition and experience of emotion in dance. Non-dance-major college students rely more on audiovisual dual modality affective perception, whereas dance-major college students rely more on their specialized training experience. These results indicate that adding auditory cues to dance performances contributes to a better experience when viewing dance works.

Supplemental Material

sj-docx-1-art-10.1177_02762374251320531 - Supplemental material for Effects of Emotional Valence, Training Experience, and Audiovisual Dual Modality on Emotion in Dance

Supplemental material, sj-docx-1-art-10.1177_02762374251320531 for Effects of Emotional Valence, Training Experience, and Audiovisual Dual Modality on Emotion in Dance by Yi Wang and Maoping Zheng in Empirical Studies of the Arts

Emotional valence × Presentation modality	Major	M	SD	Sample size(n)
Audiovisual bimodal × Sad emotion	Non-dance major	0.98	0.07	33
Dance major	0.97	0.14	40
Total	0.97	0.11	73
Audiovisual bimodal × Happy mood	Dance major	0.98	0.08	33
Dance major	0.99	0.04	40
Total	0.99	0.06	73
Visual modality only × Sad emotion	Non-dance major	0.96	0.15	33
Dance major	0.99	0.04	40
Total	0.97	0.11	73
Visual modality only × Happy mood	Non-dance major	0.95	0.09	33
Dance major	0.98	0.08	40
Total	0.96	0.089	73

Source of variation	MS	df	F	p-value	η^²
Presentation modality	0.01	1	2.92	.092	.040
Presentation modality × Dance major or not	0.02	1	4.07	.047*	.054
Emotional valence	0.00	1	0.01	.906	.000
Emotional valence × Dance major or not	0.00	1	0.01	.906	.000
Presentation modality × Emotional valence	0.01	1	2.00	.162	.027
Presentation modality × Emotional valence × Dance major or not	0.00	1	0.471	.495	.007

Emotional valence × Presentation modality	Major type	M	SD	Sample size(n)
Audiovisual bimodal × Sad emotion	Non-dance major	4.50	0.41	33
Dance major	3.97	0.68	40
Total	4.21	0.63	73
Audiovisual bimodal × Happy emotion	Non-dance major	4.52	0.43	33
Dance major	4.05	0.70	40
Total	4.26	0.63	73
Visual modality only × Sad emotion	Non-dance major	4.39	0.61	33
Dance major	4.13	0.66	40
Total	4.25	0.65	73
Visual modality only × Happy emotion	Non-dance major	4.11	0.69	33
Dance major	4.09	0.52	40
Total	4.10	0.60	73

Source of variation	MS	df	F	p-value	η^²
Presentation modality	0.45	1	2.78	.100	.038
Presentation modality × Dance major or not	2.38	1	14.66	<.001***	.171
Emotional valence	0.22	1	0.92	.341	.013
Emotional valence × Dance major or not	0.38	1	1.60	.210	.022
Presentation modality × Emotional valence	0.78	1	10.21	.002**	.126
Presentation modality × Emotional valence × Dance major or not	0.12	1	1.63	.207	.022

Emotional valence × Presentation modality	Major type	M	SD	Sample size(n)
Audiovisual bimodal × Sad emotion	Non-dance major	0.967	0.07	33
Dance major	0.98	0.08	40
Total	0.98	0.07	73
Audiovisual bimodal × Happy emotion	Non-dance major	0.97	0.09	33
Dance major	0.99	0.04	40
Total	0.98	0.07	73
Visual modality only × Sad emotion	Non-dance major	0.96	0.13	33
Dance major	1.00	0.03	40
Total	0.98	0.09	73
Visual modality only × Happy emotion	Non-dance major	0.93	0.12	33
Dance major	0.97	0.09	40
Total	0.95	0.10	73

Source of variation	MS	df	F	p-value	η^²
Presentation modality	0.01	1	3.99	.050*	.053
Presentation modality × Dance major or not	0.01	1	2.64	.109	.036
Emotional valence	0.01	1	1.53	.220	.021
Emotional valence × Dance major or not	0.00	1	0.09	.772	.001
Presentation modality × Emotional valence	0.02	1	3.60	.062	.048
Presentation modality × Emotional valence × Dance major or not	0.00	1	0.12	.731	.002

Emotional valence × Presentation modality	Major type	M	SD	Sample size(n)
Audiovisual bimodal × Sad emotion	Non-dance major	4.47	0.41	33
Dance major	3.84	0.67	40
Total	4.12	0.65	73
Audiovisual bimodal × Happy emotion	Non-dance major	4.46	0.40	33
Dance major	3.76	0.65	40
Total	4.08	0.65	73
Visual modality only × Sad emotion	Non-dance major	3.50	0.56	33
Dance major	4.20	0.51	40
Total	3.88	0.63	73
Visual modality only × Happy emotion	Non-dance major	3.38	0.58	33
Dance major	4.15	0.45	40
Total	3.80	0.64	73

Source of variation	MS	df	F	p-value	η^²
Presentation modality	7.61	1	30.53	< .001***	.301
Presentation modality × Dance major or not	35.12	1	140.93	< .001***	.665
Emotional valence	0.29	1	2.02	.159	.028
Emotional valence × Dance major or not	2.34	1	0.00	.990	.000
Presentation modality × Emotional valence	0.04	1	0.37	.543	.005
Presentation modality × Emotional valence × Dance major or not	0.09	1	0.90	.345	.013

Footnotes

Authors Note

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. Furthermore, the authors disclose no potential conflicts of interest concerning the research, authorship, or publication of this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the The Science and Technology Research Program of Chongqing Municipal Education Commission, the Humanities and Social Sciences program of Chongqing Municipal Education Commission, (grant number Grant No. KJZD-K202002301, Grant No. SKGH295).

Ethics Statement

The studies involving human participants were reviewed and approved by the Research Project Ethical Review Application Form, Faculty of Psychology, Southwest University (approval number: H22081).

ORCID iDs

Yi Wang

Maoping Zheng

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Yi Wang is an external teacher at the School of Music of Southwest University. She graduated from the School of Music of Southwest University with a master's degree in art theory. Wang's research focuses on the relationship between dance and emotion.

Maoping Zheng is the professor and doctoral supervisor of the School of Music of Southwest University. He graduated from the School of Psychology of Southwest University with a doctorate degree in applied psychology. Zheng's research focuses on the relationship between music and mood.

References

Akkermans

Schapiro

Müllensiefen

Jakubowski

Shanahan

Baker

, … Frieler

(2019). Decoding emotions in expressive music performances: A multi-lab replication and extension study. Cognition and Emotion, 33(6), 1099–1118. https://doi.org/10.1080/02699931.2018.1541312

Alluri

Brattico

Toiviainen

Burunat

Bogert

Numminen

Kliuchko

(2015). Musical expertise modulates functional connectivity of limbic regions during continuous music listening. Psychomusicology: Music, Mind, and Brain, 25(4), 443–454. https://doi.org/10.1037/pmu0000124

Ammirante

Thompson

W. F.

(2012). Continuation tapping to triggered melodies: Motor resonance effects of melodic motion. Experimental Brain Research, 216(1), 51–60. https://doi.org/10.1007/s00221-011-2907-5

Anderson

(2003). Embodied cognition: A field guide. Artificial Intelligence, 149, 91–130. https://doi.org/10.1016/S0004-3702(03)00054-7

Battcock

Schutz

(2022). Emotion and expertise: How listeners with formal music training use cues to perceive emotion. Psychological Research, 86(1), 66–86. https://doi.org/10.1007/s00426-020-01467-1

Boone

T. R.

Cunningham

J. G.

(1998). The specification of emotion in music and dance: Evidence for structural cues. Infant Behavior and Development, 21(Suppl), 239. https://doi.org/10.1016/S0163-6383(98)91454-0

Camurri

Lagerlöf

Volpe

(2013). Recognizing emotion from dance movement: Comparison of spectator recognition and automated techniques. International Journal of Human-Computer Studies, 59(1-2), 213–225. https://doi.org/10.1016/S1071-5819(03)00050-8

Chaddock-Heyman

Loui

Weng

T. B.

Weisshappel

McAuley

Kramer

A. F.

(2021). Musical training and brain volume in older adults. Brain Sciences, 11(1), Article 50. https://doi.org/10.3390/brainsci11010050

Chen

X. H.

Pan

Z. H.

Wang

Yang

X. H.

Liu

You

X. Q.

Yuan

J. J.

(2016). The integration of facial and vocal cues during emotional change perception: EEG markers. Social cognitive and affective neuroscience, 11(7), 1152–1161. https://doi.org/10.1093/scan/nsv083

10.

Chen

Wang

(2019). Yin Yue Xun Lian Dui Da Nao Qian Zhu Yi Jia Gong De Ying Xiang [the effect of music training on pre-attentive processing of the brain]. Xinli Kexue Jinzhan, 27(6), 1036–1043. https://doi.org/10.3724/SP.J.1042.2019.01036

11.

Christensen

J. F.

Gaigg

S. B.

Gomila

Oke

Calvo-Merino

(2014). Enhancing emotional experiences to dance through music: The role of valence and arousal in the cross-modal bias. Front Hum Neurosci, 8(8), 757. https://doi.org/10.3389/fnhum.2014.00757

12.

Christensen

J. F.

Pollick

F. E.

Lambrechts

Gomila

(2016). Affective responses to dance. Acta Psychologica, 168, 91–105. https://doi.org/10.1016/j.actpsy.2016.03.008

13.

Compilation Committee for Volume. (1985). Zhong Guo Da Bai Ke Quan shu (Yin Yue Wu Dao Juan) [Encyclopedia of China (Vol. Music and Dance)]. Encyclopedia of China Publishing House.

14.

Dibben

(2004). The role of peripheral feedback in emotional experience with music. Music Perception, 22(1), 79–115. https://doi.org/10.1525/mp.2004.22.1.79

15.

Dieterich-Hartwell. (2019). Music, movement, and emotions: An inquiry with suggestions for the practice of dance/movement therapy. Body, Movement and Dance in Psychotherapy, 14(4), 249–263. https://doi.org/10.1080/17432979.2019.1676310

16.

Di Mauro

Toffalini

Grassi

Petrini

(2018). Effect of long-term music training on emotion perception from drumming improvisation. Frontiers in Psychology, 9, Article 2168. https://doi.org/10.3389/fpsyg.2018.02168

17.

Donohue

S. E.

Appelbaum

L. G.

Park

C. J.

Roberts

K. C.

Woldorff

M. G.

(2013). Cross-modal stimulus conflict: The behavioral effects of stimulus input timing in a visual-auditory Stroop task. PLoS One, 8(4), Article e62802. https://doi.org/10.1371/journal.pone.0062802

18.

Ernst

M. O.

Bülthoff

H. H.

(2004). Merging the senses into a robust percept. Trends in Cognitive Sciences, 8(4), 162–169. https://doi.org/10.1016/j.tics.2004.02.002

19.

Evans

Schubert

(2008). Relationships between expressed and felt emotions in music. Musicae Scientiae, 12(1), 75–99. https://doi.org/10.1177/102986490801200105

20.

Faul

Erdfelder

Lang

Buchner

(2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146

21.

Föcker

Gondan

Röder

(2011). Preattentive processing of audio-visual emotional signals. Acta Psychologica, 137(1), 36–47. https://doi.org/10.1016/j.actpsy.2011.02.004

22.

(2016). Qing Xu Xin Li Xue [Psychology of emotion]. East China Normal University Press.

23.

Gabrielsson

(2002). Emotion perceived and emotion felt: Same or different? Musicae Scientiae, 5(1), 123–147. https://doi.org/10.1177/10298649020050S105

24.

Gao

Wedell

D. H.

Kim

Weber

C. E.

Shinkareva

S. V.

(2018). Modelling audiovisual integration of affect from videos and music. Cognition and Emotion, 32(3), 516–529. https://doi.org/10.1080/02699931.2017.1320979

25.

Goldman

de Vignemont

(2009). Is social cognition embodied? Trends in Cognitive Sciences, 13, 154–159. https://doi.org/10.1016/j.tics.2009.01.007

26.

Hanser

W. E.

Mark

R. E.

Zijlstra

W. P.

Vingerhoets

A. J.

(2015). The effects of background music on the evaluation of crying faces. Psychology of Music, 43(1), 75–85. https://doi.org/10.1177/0305735613498132

27.

Hirst

R. J.

Stacey

J. E.

Cragg

Stacey

P. C.

Allen

H. A.

(2018). The threshold for the McGurk effect in audio-visual noise decreases with development. Scientific Reports, 8(1), Article 12372. https://doi.org/10.1038/s41598-018-30798-8

28.

Hoenig

Müller

Herrnberger

Sim

E. J.

Spitzer

Ehret

Kiefer

(2011). Neuroplasticity of semantic representations for musical instruments in professional musicians. NeuroImage, 56(3), 1714–1725. https://doi.org/10.1016/j.neuroimage.2011.02.065

29.

Hong

(2003). Wu Dao Ben Ti Yi Shi De Tu Xian: Xin Shi Qi Wu Dao Yu Yin Yue Guan Xi Lun Zheng Gai Guan [Increasing recognition of the independent value of dance: An overview of the debate on the relationship between dance and music in the new era]. Journal of Beijing Dance Academy, (Z1), 83–90. https://bjwd.cbpt.cnki.net/portal/journal/portal/client/paper/f0a6aa0083f9e1cf21a6b79702df05c7

30.

Howlin

Vicary

Orgs

(2020). Audiovisual aesthetics of sound and movement in contemporary dance. Empirical Studies of the Arts, 38(2), 191–211. https://doi.org/10.1177/0276237418818633

31.

Hunter

P. G.

Schellenberg

E. G.

(2010). Music and emotion. In Riess Jones

Fay

Popper

(Eds.), Music perception (Vol. 36, pp. 129–164). Springer. https://doi.org/10.1007/978-1-4419-6114-3_5

32.

Jessen

Kotz

S. A.

(2011). The temporal dynamics of processing emotions from vocal, facial, and bodily expressions. NeuroImage, 58(2), 665–674. https://doi.org/10.1016/j.neuroimage.2011.06.035

33.

Juslin

P. N.

Laukka

(2004). Expression, perception, and induction of musical emotions: A review and a questionnaire study of everyday listening. Journal of New Music Research, 33(3), 217–238. https://doi.org/10.1080/0929821042000317813

34.

Kang

Luo

(2020). Shi Ting Kua Tong Dao De Zheng He Yu Chong Tu Kong Zhi [The integration and conflict control in audiovisual processing]. Psychological Science, 43(5), 1072–1078. https://doi.org/10.16719/j.cnki.1671-6981.20200507

35.

Kantor-Martynuska

Horabik

(2015). Granularity of emotional responses to music: The effect of musical expertise. Psychology of Aesthetics, Creativity, and the Arts, 9(3), 235–247. https://doi.org/10.1037/a0039107

36.

Krahé

Hahn

Whitney

(2015). Is seeing (musical) believing? The eye versus the ear in emotional responses to music. Psychology of Music, 43(1), 140–148. https://doi.org/10.1177/0305735613498920

37.

Kratus

(1993). A developmental study of children’s interpretation of emotion in music. Psychology of Music, 21(1), 3–19. https://doi.org/10.1177/030573569302100101

38.

Kret

M. E.

Roelofs

Stekelenburg

J. J.

de Gelder

(2013). Emotional signals from faces, bodies and scenes influence observers’ face expressions, fixations and pupil-size. Frontiers in Human Neuroscience, 7(810), Article 810. https://doi.org/10.3389/fnhum.2013.00810

39.

Krumhansl

C. L.

(1997). Psychophysiology of musical emotions. Canadian Journal of Experimental Psychology, 51(4), 336–353. https://doi.org/10.1037/1196-1961.51.4.336

40.

Krumhansl

C. L.

Schenck

D. L.

(1997). Can dance reflect the structural and expressive qualities of music? A perceptual experiment on Balanchine’s choreography of Mozart’s Divertimento No. 15. Musicae Scientiae, 1(1), 63–85. https://doi.org/10.1177/102986499700100105

41.

Langer

(1986). Feeling and form (D. Liu, Z. Fu, & F. Zhou, Trans.). China Social Sciences Press. (Original work published 1986).

42.

Liang

Yang

Zhang

Yang

Zheng

Cao

(2019). Shen Ti Biao Qing Shi Bie De ERP Yan Jiu [the recognition of bodily expression: An event-related potential study]. Studies of Psychology and Behavior, 17(3), 318–325. https://psybeh.tjnu.edu.cn/CN/Y2019/V17/I3/318

43.

Livingstone

S. R.

Thompson

W. F.

(2009). The emergence of music from the theory of mind. Musicae Scientiae, 13(2_suppl), 83–115. https://doi.org/10.1177/1029864909013002061

44.

Logeswaran

Bhattacharya

(2009). Crossmodal transfer of emotion by music. Neuroscience Letters, 455(2), 129–133. https://doi.org/10.1016/j.neulet.2009.03.044

45.

Lugo

J. E.

Doti

Wittich

Faubert

(2008). Multisensory integration: Central processing modifies peripheral systems. Psychological Science, 19(10), 989–997. https://doi.org/10.1111/j.1467-9280.2008.02190.x

46.

Lundqvist

L. O.

Carlsson

Hilmersson

Juslin

P. N.

(2009). Emotional responses to music: Experience, expression, and physiology. Psychology of Music, 37(1), 61–90. https://doi.org/10.1177/0305735607086048

47.

(2003). Wu Dao Xue Dao Lun [Introduction to dance theory]. Shanghai Literature and Art Publishing House.

48.

Bai

Tao

(2013). Yin Yue Yu Qing Xu You Fa De Ji Zhi Mo Xing [Music and model of emotion-inducing mechanism]. Xinli Kexue Jinzhan, 21(4), 643–652. https://doi.org/10.3724/SP.J.1042.2013.00643

49.

Merrill

Omigie

Wald-Fuhrmann

(2020). Locus of emotion influences psychophysiological reactions to music. PLoS One, 15(8), Article e0237641. https://doi.org/10.1371/journal.pone.0237641

50.

Molnar-Szakacs

Overy

(2006). Music and mirror neurons: From motion to ‘e’ motion. Social Cognitive and Affective Neuroscience, 1(3), 235–241. https://doi.org/10.1093/scan/nsl029

51.

Niedenthal

P. M.

(2007). Embodying emotion. Science, 316, 1002–1005. https://doi.org/10.1126/science.1136930

52.

Ping

(2004). Wu Dao Xin Li Xue [Dance psychology]. Higher Education Press.

53.

Pollak

S. D.

Sinha

(2002). Effects of early experience on children’s recognition of facial displays of emotion. Developmental Psychology, 38(5), 784–791. https://doi.org/10.1037/0012-1649.38.5.784

54.

Punkanen

Saarikallio

Luck

(2014). Emotions in motion: Short-term group form dance/movement therapy in the treatment of depression: A pilot study. The Arts in Psychotherapy, 41(5), 493–497. https://doi.org/10.1016/j.aip.2014.07.001

55.

Reason

Jola

Kay

Reynolds

Kauppi

J.-P.

Grobras

M.-H.

, … Pollick

F. E.

(2016). Spectators’ aesthetic experience of sound and movement in dance performance: A transdisciplinary investigation. Psychology of Aesthetics, Creativity, and the Arts, 10(1), 42–55. https://doi.org/10.1037/a0040032

56.

Sawada

Suda

Ishii

(2003a). Relationship between Leg Movement Quality and Emotional Expression in Dance [Conference presentation]. Annual Meeting of the International Association for Dance Medicine & Science, London, England.

57.

Sawada

Suda

Ishii

(2003b). Expression of emotions in dance: Relation between arm movement characteristics and emotion. Perceptual and Motor Skills, 97(3 Pt 1), 697–708. https://doi.org/10.2466/pms.2003.97.3.697

58.

Scherer

K. R.

Trznadel

Fantini

Coutinho

(2019). Assessing emotional experiences of opera spectators in situ. Psychology of Aesthetics, Creativity, and the Arts, 13(3), 244–258. https://doi.org/10.1037/aca0000163

59.

Schubert

(2007). Locus of emotion: The effect of task order and age on emotion perceived and emotion felt in response to music. Journal of Music Therapy, 44(4), 344–368. https://doi.org/10.1093/jmt/44.4.344

60.

Schubert

(2013). Emotion felt by the listener and expressed by the music: Literature review and theoretical perspectives. Frontiers in Psychology, 4, Article 837. https://doi.org/10.3389/fpsyg.2013.00837

61.

Spitzer

Coutinho

(2014). The effects of expert musical training on the perception of emotions in Bach’s Sonata for unaccompanied Violin No. 1 In G Minor (BWV 1001). Psychomusicology: Music, Mind, and Brain, 24(1), 35–57. https://doi.org/10.1037/pmu0000036

62.

Strelnikov

Foxton

Marx

Barone

(2015). Brain prediction of auditory emphasis by facial expressions during audiovisual continuous speech. Brain Topography, 28(3), 494–505. https://doi.org/10.1007/s10548-013-0338-2

63.

Sun

(2018). Neural Basis of the Effects of Musical Training on Phrase Structure Processing [Doctoral dissertation]. Shanghai Normal University.

64.

Sun

Zhang

Xun

Liu

Zhang

(2011). Duo Gan Jue Zheng He Ce Liang Fan Shi Zhong Cun Zai De Ying Xiang Yin Su Tan Tao [an investigation on the effect factors in the paradigm of multisensory integration]. Acta Psychologica Sinica, 43(11), 1239–1246. https://journal.psych.ac.cn/xlxb/CN/Y2011/V43/I11/1239

65.

Tatz

J. R.

Peynircioğlu

Z. F.

Brent

(2020). Face-voice space: Integrating visual and auditory cues in judgments of person distinctiveness. Attention, Perception & Psychophysics, 82(7), 3710–3727. https://doi.org/10.3758/s13414-020-02084-1

66.

Thompson

W. F.

Russo

F. A.

Quinto

(2008). Audio-visual integration of emotional cues in song. Cognition and Emotion, 22(8), 1457–1470. https://doi.org/10.1080/02699930701813974

67.

Van Meel

Verburgh

De Meijer

(1993). Children’s interpretations of dance expressions. Empirical Studies of the Arts, 11(2), 117–133. https://doi.org/10.2190/V69N-VB0T-A9Q3-TJ04

68.

Wang

(2006). Zong He Shi Ye Zhong De Yin Yue Yu Wu Dao Guan Zhao [Multi-perspective examination of music and dance]. Journal of Beijing Dance Academy, (1), 71–76. https://bjwd.cbpt.cnki.net/portal/journal/portal/client/paper/dcaf93990a1fa3d4d01313c09173aa81

69.

Wang

Liu

(2013). Yin Yue Qing Xu Xin Li Ji Zhi De Yan Jiu Xian Zhuang Ji Zhan Wang [Current status and outlook of the research of emotion-psychological mechanisms of music]. Chinese Music, (2), 108–111. https://cmj.ccmusic.edu.cn/zh/article/42092730/

70.

Welch

R. B.

DuttonHurt

L. D.

Warren

D. H.

(1986). Contributions of audition and vision to temporal rate perception. Perception & Psychophysics, 39(4), 294–300. https://doi.org/10.3758/BF03204939

71.

Wen

Liu

Sun

Zhang

Yin

Hao

(2009). Duo Gan Guan Xian Suo Zheng He De Li Lun Mo Xing [Theoretical models of multisensory cues integration]. Advances in Psychological Science, 17(4), 659–666. https://journal.psych.ac.cn/xlkxjz/CN/Y2009/V17/I4/659

72.

Wigman

(1966). The language of dance (W. Sorell, Trans.). Wesleyan University Press.

73.

Wilson

(2002). Six views of embodied cognition. Psychonomic Bulletin & Review, 4, 625–636. https://doi.org/10.3758/BF03196322

74.

Wöllner

(2012). Is empathy related to the perception of emotional expression in music? A multimodal time-series analysis. Psychology of Aesthetics, Creativity, and the Arts, 6(3), 214–223. https://doi.org/10.1037/a0027392

75.

H. S.

(2010). Embodied cognition: A new approach to cognitive psychology. Advances in Psychological Science, 18(5), 705–710. https://journal.psych.ac.cn/xlkxjz/CN/Y2010/V18/I5/705

76.

Zentner

Grandjean

Scherer

K. R.

(2008). Emotions evoked by the sound of music: Characterization, classification, and measurement. Emotion, 8(4), 494–521. https://doi.org/10.1037/1528-3542.8.4.494

77.

Zhang

(2017). Wu Dao Dui Qing Xu De Ying Xiang [Effects of dance on emotions] [Master’s thesis]. Beijing Dance Academy.

78.

Zhou

Zheng

(2020). Ji Yu Shi Ting Shuang Tong Dao De Yin Yue Qing Xu Jia Gong Ji Zhi Ji Qing Xu Lei Xing He Yin Yue Jing Yan De Ying Xiang [The mechanism of musical emotion based on audio-visual multisensory and influence of emotion type and musical expertise]. Journal of Psychological Science, 43(1), 39–45. https://jps.ecnu.edu.cn/CN/Y2020/V43/I1/39

79.

Zhu

(2017). Duo Gan Guan Xian Suo Zheng He De Li Lun Mo Xing [Empirical study of the effects of melody modality on emotion] [Master’s thesis]. Southwest University.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

Effects of Emotional Valence,Training Experience,and Audiovisual Dual Modality on Emotion in Dance

Abstract

Keywords

Methods

Participants

Experimental Design

Experimental Materials

Experimental Procedure

Statistical Analysis

Results

Emotion Recognition Accuracy Rate

Emotion Recognition Intensity

Emotional Experience Accuracy Rate

Emotional Experience Intensity

Discussion

Visual Modality Only Has the Advantage of Emotion Recognition

Audiovisual Dual Modality Emotion Processing Has Advantages in Emotional Experience Only

Audiovisual Dual Modality Emotion Processing Only Has Advantages for Non-Professional Spectators

Audiovisual Dual Modality Emotion Processing Only Has Advantages in Recognition of Happiness

Research Limitations and Outlook

Conclusions

Supplemental Material

sj-docx-1-art-10.1177_02762374251320531 - Supplemental material for Effects of Emotional Valence, Training Experience, and Audiovisual Dual Modality on Emotion in Dance

Footnotes

Authors Note

Declaration of Conflicting Interests

Funding

Ethics Statement

ORCID iDs

Supplemental Material

Author Biographies

References

Supplementary Material