Abstract
Purpose
The paper shows the interpretive impact of different constructions of the point of view available to the reader/viewer in book and animated movie versions of a children's picture book, a novel for pre-adolescents/early teenagers, and a graphic novel for adolescents and adults.
Design/Approach/Methods
Excerpts from book and animated movie versions of the same story are compared using multimodal analysis of interpersonal meaning to show how the reader/viewer is positioned in relation to the characters in each version, complemented by analyses of ideational meaning to show the effect of point of view on interpretive possibilities.
Findings
Focusing mainly on multimodal construction of point of view, the analyses show how interpretive possibilities of ostensibly the same story are significantly reconfigured in animated adaptations compared with book versions even when the verbal narrative remains substantially unchanged.
Originality/Value
The study shows that it is crucial to students’ critical appreciation of, and their creative contribution to, their evolving digital literary culture that in this new era of educational technology, attention in literacy and literary education focuses on developing understandings of digital multimodal narrative art, and that animated movie adaptations are not presented pedagogically as isomorphic with, or simply adjunct to, corresponding book versions.
Introduction
The integration of literature and new technologies continues to evolve, maintaining a longstanding trajectory of literary adaptation and innovation with the ever-changing development of films and new communication technologies. Novels, picture books, and graphic novels are increasingly being adapted as animated movies. This world of narrative versions is the norm for young people growing up in the 21st century, and this important cultural phenomenon warrants explicit attention in English language arts classrooms. The convergence of new developments in digital animation, along with the burgeoning of renowned contemporary literature now readily accessible as animated movies, and the recognition by societies and education systems of the distinctive narrative art of digital animated literature represents an important aspect of a new era for educational technology as a significant creative form of multimodal social semiotic in the 21st century. It is, therefore, crucial to students’ critical appreciation of and creative contribution to their evolving digital literary culture that they develop an understanding of digital multimodal narrative art and that animated movie adaptations are not presented pedagogically as equivalent to or simply as adjuncts to the paper media versions of literary narratives.
Through empirical multimodal analyses, this study seeks to illustrate how digital animated versions of literary texts provide distinctively different interpretive possibilities from their paper-media counterparts, emphasizing the contribution of educational technology within literacy and literary education in an increasingly digital world. In the digital animated re-versioning of literary narratives, it is often the inclusion and/or changing of the visual art against the maintenance of the verbal art across versions that reconstructs the interpretive possibilities of the narrative at the visual/verbal interface. This study focuses on the multimodal construction of points of view in the same literary narratives found in paper and digital animated formats to demonstrate how the interpretive possibilities of what is ostensibly the same story are significantly reconfigured in the different versions, even when the verbal narrative remains substantially unchanged. This study analyzes the corresponding segments of the book and animated movie versions of a popular picture book for elementary school-age children and a novel for pre-adolescents to early teenagers. The analyses draw on systemic functional linguistics (Halliday & Matthiessen, 2004; Martin & Rose, 2007; Martin & White, 2005), semiotic accounts of the meaning-making resources of images (Kress & van Leeuwen, 2021), and the multimodal semiotics of picture books (Painter et al., 2013).
Interpersonal meanings in narrative images
The analyses of interpersonal meanings in images draw on the “grammar of visual design” (Kress & van Leeuwen, 2021) and related work focusing on picture book analyses (Martin, 2008; Painter, 2007, 2008; Painter et al., 2011, 2013; Painter & Martin, 2011). In their description of interactive relations between represented participants in images and between viewers and image representations, Kress and van Leeuwen (2021) propose a number of systems of visual meaning-making resources. This study draws on their INVOLVEMENT, CONTACT, and SOCIAL DISTANCE systems.
The INVOLVEMENT system describes the positioning of the viewer to feel part of the social context of the represented participant(s) to a certain degree. The degree of involvement is determined by the horizontal angle (Kress & van Leeuwen, 2021, pp. 135–38). If the characters face the viewer “front on,” that is, the horizontal frontal plane of the viewer and that of the character are parallel, the sense of viewer involvement is maximized. However, when characters are depicted at an oblique angle, that is, the frontal plane of the viewer and that of the depicted characters are not parallel, the viewer is positioned as more detached from depicted characters and thereby influenced to see them as “other.”
In their CONTACT system, Kress and van Leeuwen (2021) refer to images where a depicted character gazes out at the viewer as “demand” images, and where there is no such gaze, they refer to the image as an “offer.” For Painter et al. (2013), the gaze of the depicted participant does not actually “demand” anything, but it does make interpersonal contact with the viewer; hence they refer to such images as “contact” images. Where the depicted participants do not make eye contact with the viewer, the interactive role of the viewer is to observe what is in the image; hence, such images are referred to as “observe” images. As in previous studies where these issues have been noted (Unsworth, 2014a, 2014b), the terms “contact” and “observe” are used in this paper.
The system of SOCIAL DISTANCE (Kress & van Leeuwen, 2021, pp. 123–29) has been discussed as being realized by the “size of frame.” This means that the characters may appear as heads and shoulders only, largely occupying the image frame; hence, they appear close to the viewer. Alternatively, perhaps only their face or part of their face is visible, which makes them appear to be at an intimate social distance from the viewer (Unsworth, 2014b). However, if the entire body of the character is visible, it appears as if the character is farther away from the viewer. If the whole-body depiction is in the background of the image, the character may appear very remote from the viewer. These social distance options are commonly referred to as a “close-up” or “long shot,” with a “mid shot” indicating a commonly accepted interactive social distance (Unsworth, 2014b).
In the construction of images, choices are made simultaneously from the systems of INVOLVEMENT, CONTACT, and SOCIAL DISTANCE (in addition to other systems proposed by Kress and van Leeuwen [2021] but not discussed here). Various combinations can result in very different interactive relationships with the viewers. For example, a close-up contact image with the frontal planes of the depicted character and the viewer parallel can construct a sense of intimacy and engagement, whereas if the parameter of social distance is changed to a “long shot,” the impact of the “contact” dimension is greatly reduced, and the interactive meaning becomes more like that of an “observe” image.
Point of view and “focalization” (Genette, 1980) continue to attract detailed scholarly inquiry (Huhn et al., 2009), but here I will simply draw on the basic distinction between (i) who is telling the story and (ii) from whose point of view, or through whose eyes, we experience the story. Usually, the issue of who is telling the story is determined by the verbal text only, but the question of “who sees,” or from whose point of view are we experiencing the story, can apply to both verbiage and the images. The points of view constructed through the verbiage and images can be consistent or divergent. In this study, emphasis is placed on the construction of a point of view within images. Images can position a viewer to assume different viewing personas. One option is to position the viewer as an outside observer. However, viewers can also be positioned as if they are one of the characters in the story or as if they have a visual perspective that is not identical but nevertheless similar to that of a character so that the viewer sees “along with” the character (Painter, 2007; Painter et al., 2013; Unsworth, 2006).
There are several methods by which viewers can be positioned as if they are characters in an image (Painter, 2007; Painter et al., 2013), such as the depiction of just a part of the body of a character from a viewpoint that could only be from the eyes of the focalizing character (such as the hands or feet out in front of the unseen body). Since viewers can see only that part of the body (such as the hands or feet) that would be visible to the focalizing character, then the reader is positioned as if s/he were the focalizing character—from that character's point of view (see also Kress & van Leeuwen, 2021, pp. 133–141).
Viewers can also share a character's point of view rather than being positioned as the character. The reader sees a part of the character while also observing what can be seen from that character's perspective. This is achieved by having the reader view what is depicted “along with” or “over the shoulder” of the focalizing character. The “over the shoulder” view can be achieved by positioning the reader's point of view as being slightly to the rear and to one side of the focalizing character (Unsworth, 2006, pp. 95–97). This is frequently achieved in the movie of The Lost Thing (Ruhemann & Tan, 2010) through a close-up foreground image of the boy's right shoulder and the right side and rear of his head, constructing our point of view as “over the shoulder.” However, this can also be achieved through a rear view of the character, provided it is not a very distant view, as will be suggested in the discussion of The Lost Thing movie.
The Lost Thing
The Lost Thing (Tan, 2000) is a humorous story about a boy who discovers a bizarre-looking creature while collecting bottle tops at a beach. Having guessed that it is lost, he tries to determine who owns it or where it belongs, but the problem is met with indifference by everyone else. Everyone is unhelpful; strangers and parents are unwilling to entertain this uninvited interruption in their daily lives. Even his friend is unable to help despite some interest. The boy feels sorry for this hapless creature and attempts to discover where it belongs (Unsworth, 2014c). The animated movie adaptation of the picture book won the best animated short movie Oscar at the 2011 Academy Awards in the United States (Ruhemann & Tan, 2010). We will compare the book and movie versions of two short excerpts: The first is from when the boy temporarily hides the Lost Thing in the family shed and feeds it something that it likes to eat; the second is from near the end of the story when the boy and the Lost Thing part company as the Lost Thing enters a kind of sanctuary for similar bizarre creatures.
Feeding the Lost Thing
In the book, there is only one page showing the boy feeding the Lost Thing (Figure 1).
Feeding the Lost Thing—Distant view (Tan, 2000).
The narration in the corresponding book and movie segments is displayed in Table 1.
Narration of the feeding event in the book and movie versions of The Lost Thing.
The gray print in the movie column of Table 1 indicates the narration from the book that has been omitted from the movie. In the movie's narration, there is no equivalent of the second two lines from the extract of the book: And gave it something to eat,
Once I found out what it liked.
The upper-case print in Table 1 indicates the narration that occurred in the movie only, and the upper-case italics show the sections in the movie narration that have some grammatical variations from the book. Both the image and text in Figure 1 from the book show the boy as the active agent and the Lost Thing as the passive recipient. Commitment to the significance of the feeding event in the book's verbal text is very limited. In the line “… I found out what it liked,” we do not know what is entailed in “I found out,” and there is no complementary commitment in the image to clarify this. However, the movie is visually committed to showing viewers how the boy came to know what the Lost Thing liked to eat.
From our first view, when the boy is inside the shed, we see the Lost Thing's tentacles exploring the shelves along the shed wall until it pulls down a box of what appears to be brightly colored baubles from the shelf right in front of the boy. As it falls, the boy catches the box in his arms (Figure 2).
The Lost Thing pulling down the box (Ruhemann & Tan, 2010).
The Lost Thing then gestures with his tentacles for the boy to follow, moving back from the boy while simultaneously opening the lid on its top extremity, allowing more tentacles to emanate from its inside to reveal a bright yellow light. It then uses its large pincers to manipulate a ladder against its side (Figure 3), which the boy climbs until he can look inside the top of the Lost Thing at a brightly lit swirling of tentacles in a compartment of dials and levers. Into this dynamic combination of organic and mechanical matter, the boy throws the baubles from the box selected by the Lost Thing. In this purely visual account of the narrative in the movie, the Lost Thing is clearly agentive, taking the initiative in looking for food and arranging the ladder, facilitating the boy's ability to feed it. The visual rendition of these events in the movie is not, in fact, inconsistent with the narrative in the book version. Rather, the movie provides more ideational details than the synoptic snapshot provided by the book's image (Figure 1). This greater commitment in the movie shows that the Lost Thing was agentive in this process, in direct contrast to what is implied in the book.
The Lost Thing moving the ladder (Ruhemann & Tan, 2010).
The image from the book (Figure 1) is a long-distance, “observe” image since there is no gaze from either of the characters toward the reader. The depiction of the boy is so distant and hence so small that it is impossible to discern any facial expressions. The vertical angle is at eye level, and the horizontal angle is oblique so that our observation of the scene shows the boy almost in the profile view. When the horizontal angle is such that the frontal plane of the represented participant is parallel to the frontal plane of the viewer, the viewer's maximum involvement in the world of the represented participant is indicated (Kress & van Leeuwen, 2021). An oblique horizontal angle, however, constructs the relationship between the viewer and the represented participants as “other.” Notwithstanding the first-person narration of the verbal text, this image positions the viewer as a detached outside, even remote, observer, constructing a reader/viewer stance referred to as appreciative (Painter et al., 2013), where readers and viewers can observe and learn in a detached way without becoming involved with characters due to the build-up of a pseudo-interpersonal relationship.
In the movie segment, there is much greater variation in the positioning of the viewer in relation to the images. As the Lost Thing is pulling the box from the shelf, we see a full rear view of the boy, which is parallel with our frontal plane. This involvement, coupled with the rear view, positions the audience's point of view as “along with the character” of the boy (Figure 2). Subsequently, as the Lost Thing opens its top lid, the boy's head and gaze move up synchronously, mediating our view as “along with” that of the boy. The view then shifts to a very high angle, looking down at a frontal view of the boy holding the box (Figure 3). To the left and right of the screen, in partial view, are the enormous pincers of the Lost Thing, with the right one beginning to move the ladder. This view of the boy and the Lost Thing's pincers could only be from the top center of the Lost Thing. As viewers, we are positioned to have the Lost Thing's point of view since our viewpoint has been mediated to become that of the character (Painter et al., 2013). After the boy has climbed the ladder, our next view is from across the top of the Lost Thing's brightly lit opening, with the face of the boy looking straight at us—a contact image constructing a pseudo-interpersonal contact between the viewer and the character. The next shift is to an eye-level profile view of the boy atop the ladder feeding the Lost Thing (Figure 4), which is quite similar to the corresponding image in the book.
Feeding the Lost Thing in the movie version (Ruhemann & Tan, 2010).
While the appreciative reader/viewer stance in the book maintains a distanced relationship with the characters, the viewer's stance in the movie is very different. Painter and her co-authors contrast the appreciative stance with an empathetic stance “where common humanity is recognized and the reader stands in the character's shoes” (Painter et al., 2013, p. 33). In the movie, the closer social distance, inclusion of contact images, and particularly the mediated point of view “along with” and “as” the depicted characters shift the reader/viewer engagement from appreciative to empathetic.
Saying goodbye to the Lost Thing
As the story moves to its conclusion in the book, the parting of the boy and the Lost Thing is shown in a long distance, “observe” image of the boy and the Lost Thing in profile facing each other—the boy with his hands stretched forward and the Lost Thing with its tentacles extended in a farewell gesture (Figure 5). The text on this page describes the parting as emotionally neutral and inconsequential: It seemed as good a time as any to say goodbye to each other. So we did.
Saying goodbye in the book version of the story (Tan, 2000).
Then I went home to classify my bottle-top collection.
The narration accompanying the “departure” image is omitted in the movie, and the entire departure segment is conveyed through the movie's animated images. It begins with close-up contact images of the boy looking straight out at the viewer as the door of the strange creature's sanctuary opens. After this, two long-distance “observe” images show the boy and the Lost Thing beside each other facing the viewer. This suggests collaborative companionship between the two characters. Subsequently, the boy turns and looks up at the Lost Thing. His mouth opens to a half-smile in profile view. The view then shifts to a more socially distant view of the boy's upper body and head in profile but with the just discernible smile sustained and still looking at the Lost Thing.
While the book displays images of the boy and the Lost Thing gesturing goodbye, in the movie, there is a greater visual commitment to the depiction of the actions that occur immediately before and after this common gesturing scene. This is where we see the full rear view of the boy parallel to the frontal plane of the viewer, with the boy facing the door of the sanctuary as the Lost Thing departs through it (Figure 6). The camera lingers on the rear view of the boy for a few seconds, and as the sanctuary door closes, the boy's head is tilted to one side so that he can maintain his view through the remaining opening (Figure 7).
Waving goodbye in the movie (Ruhemann & Tan, 2010)—audience view “along with” the character.
A last look—providing a visual suggestion of emotional attachment between the characters (Ruhemann & Tan, 2010).
For most of the departure segment in the movie, the point of view is “unmediated” (Painter et al., 2013), where the viewer is positioned as external to the story context and the point of view is not aligned with any of the represented participants and does not become that of any of the represented participants. However, the point of view shifts significantly when the camera moves to show the full rear view of the boy parallel to the frontal plane of the viewer. In this case, although it is a long-distance view, the rear-view image does indeed position the audience's view “along with” that of the boy. Collectively, these movie images strongly imply a significant affective bond of companionship between the boy and the Lost Thing. While their parting is ostensibly similar in the book and movie versions of the story, the differences in the visual and verbal commitments to affect; the use of close-ups, rear-view images, and mediated points of view in the movie; and the absence of such perspectives in the book privilege very different interpretive possibilities for this segment of the story in the book and the movie.
The visual point of view in the book is an overwhelmingly unmediated observation at a remote social distance. Although the verbal dimension of The Lost Thing is a first-person narrative, the images in the book construct a reader's stance as predominantly distant and detached. The tension between the alignment of the reader and the character of the boy as the first-person narrator and the remote social distance of the images reflects the relationship between the boy and the Lost Thing, where the boy believes he should be concerned about society's abandonment of the Lost Thing but does not establish any interpersonal closeness with it.
In the movie, there are many more mid-distance and close-up views of the characters. Sometimes, the main character, the boy, looks directly out at the audience, and viewers are very frequently positioned to have a point of view “along with” one of the characters or even as one of the characters. While the story events are almost exactly equivalent in both versions and the narration varies only moderately, the visual depiction in the movie significantly affects the interpretive possibilities. The movie substantially maintains the minimalist style of the book, but the closer social distance presented, the higher frequency of contact images, and the mediated point of view “along with” and “as” the depicted characters, shift the reader/viewer engagement from appreciative to empathic. The movie visually accentuates the boy and the Lost Thing as companionable and the boy as a focus for interpretive issues concerning difference, conformity, acceptance, and interpersonal outreach.
Coraline
Neil Gaiman's popular novel, Coraline (Gaiman & McKean, 2002), also published as a graphic novel (Gaiman & Russell, 2008), has been adapted as an animated movie (Selick, 2009). Coraline is a story about a pre/early teenager who lives with her preoccupied parents in part of a huge old house divided into apartments, two of which are home to two retired actresses and an unusual elderly gentleman. Coraline contents herself for weeks by exploring the vast garden and grounds. However, when the rainy weather comes, she becomes bored and begins to count everything blue, including the windows and doors, and the 14th door, although blocked with a wall of bricks, opens to admit Coraline into an alternate distorted-mirror world, containing everything Coraline has ever dreamed of: people who pronounce her name correctly (not “Caroline”), delicious meals (not like her father's overblown “recipes”), and an unusually pink and green bedroom (not like her own dull one). Her mirrored parents, her “other mother” and “other father,” look just like her own parents but with big, shiny, black button eyes, paper-white skin, and a keen desire to keep her on their side of the door. Coraline must fight with all her wit and courage to save herself and her real family and return to her ordinary life.
Here, we will consider a brief segment of this story over two pages in the book (Gaiman & McKean, 2002, pp. 109–110) and a 1-min and 15-s segment in the movie where Coraline proposes to play a game with her other mother. She hopes this will be the key to rescuing herself and her parents, who, along with other previous victims, have been captured by the other mother. In comparing the book and movie versions of this story segment, it can be seen that there is a contrast in the fidelity of the movie to the book version of this segment of the story. The first difference is their ideational meanings, that is, the setting and what is discussed between the other mother and Coraline. The second difference is the interpersonal representations in the two versions; that is, how they interact dialogically to convey emotional and evaluative meanings.
The communication of Coraline's fear in the book excerpt occurs in only one sentence in the narration: “she gripped her knees under the table, to stop them from shaking.” In the movie, from the moment Coraline enters the breakfast room, her fear is intimated by her facial expressions (Figure 8). This intimation is confirmed by the voiceover of Coraline's internal resolve: “Be strong, Coraline.”
Before the game proposal—A facial expression of trepidation (Selick, 2009).
A further facial expression of insecurity occurs a few seconds later as Coraline sits at the breakfast table (Figure 9).
Facial expression of trepidation at the breakfast table (Selick, 2009).
This expression of insecurity is maintained when Coraline's pupils shift from looking to her side at the other mother to looking down at the table in front of her. Then, we see a box on the table containing buttons, needles, and threads. What is important here is that as the camera shifts from Coraline's eyes looking down at the table to showing only the button box (Figure 10), the viewers are positioned as the character, as if we were Coraline looking at the button box.
The threat of the buttons—Viewer positioned as Coraline (Selick, 2009).
The camera then zooms in slightly on the button box and shifts to show an extreme close-up of Coraline's brow and eyes, with a bead of sweat rolling down her forehead (Figure 11). Therefore, not only are these early signals of Coraline's fear gradually intensifying, but as viewers, we are positioned as if we are Coraline experiencing these fears.
Extreme close-up view of Coraline's facial expression of fear (Selick, 2009).
These expressions of fear in the movie occur just before Coraline broaches the subject of the game. When the other mother asks what happens if Coraline loses the game, the camera again shows us the button box from Coraline's point of view, positioning us as the character once more. When Coraline replies that she will stay with the other mother and let her sew buttons into her eyes, the camera again shifts to an extreme close-up of Coraline's brow and eyes, and we see her eyebrows furrow downwards as she utters the dreadful consequence of losing the game (Figure 12).
Extreme close-up as Coraline realizes the consequences of losing the game (Selick, 2009).
Thus, the audience is positioned to adopt Coraline's point of view at two moments of great tension and vulnerability for Coraline. While the book version tells the reader on one occasion about Coraline's fear, movie viewers see the physicality of that fear and are positioned as Coraline as if to experience that fear, which is intensified and sustained through the dialogic exchanges.
The other mother is portrayed in the book version of this segment as coldly unfeeling, demonic, and perhaps also dishonest. This is conveyed partly through the references to her black eyes, which “flashed” at the mention of “games” and “stared” at Coraline, “unblinking” as she agreed to have buttons sewn into her eyes should she lose the game. The other mother's honesty is called into question when Coraline asks if she would be happier if she won Coraline “fair and square”—and the other mother answers simply, “Possibly.” The demonic nature of the other mother is further strongly implied through her look of “unconcernedness” as “her fingers twitched and drummed, and she licked her lips with her scarlet tongue.” In the movie, an extreme close-up of the other mother's brow and eyes shows her black eyes flashing at the mention of games, and she drums her fingers on the bench as she asks what Coraline will be finding in the proposed “finding things” game. However, there is no scarlet tongue or licking of the lips in the movie version. The other mother in the movie is certainly grotesque-looking, with a peculiarly thin misshapen body and an extraordinarily elongated neck, so her appearance is disconcertingly unsettling, but it does not suggest the demonic characteristics implied in the book's description.
A comparison of the other mother's questions to Coraline about the proposed game in the book and movie versions (see Table 2) reveals that in the movie, there is a less demeaning attitude toward Coraline and a softening of the power exerted by the other mother.
Comparison of the other mother's questions in the book and the movie.
In the book, the first question diminishes any acknowledgment that something is to be found, trivializes the game as a “hide-and-go-seek game,” and overtly demonstrates the power differential between the two using Coraline's full name. In the second question, the use of “offering” graduated by “exactly” emphasizes the “hardball” bargaining and greater power of the other mother. The third question in the book uses “not lose” rather than “win,” reducing the other mother's acknowledgment of Coraline's possible success in the game. The questions in the movie are much more “matter-of-fact” without the power differential implications of the book version questions. However, the interpersonal remoteness between the other mother and Coraline is indicated as most of their interchange occurs without them looking at each other. When the other mother does look at Coraline, the camera is at a low angle, so the other mother is looking down on Coraline from what appears to be a towering height, conveying her more powerful position (Figure 13). In summary, the other mother is presented as cold and more powerful in the movie but not as demonic and domineering as in the book.
Low-angle view of the other mother, indicating her relative power (Selick, 2009).
We have noted that the characters, the activities they are involved in, and the content of their discussions—the ideational meaning—remain the same across the book and movie versions. However, the interpersonal meaning—the portrayal of the characters’ effect and the judgment of their personal qualities—while also broadly consistent, shows some variation across the versions. A further important aspect of interpersonal meaning to consider is the expression of evaluative meanings or appreciation of inanimate items in the interactions. Just prior to Excerpt 1 from the book, the other mother tells Coraline that she is making Coraline breakfast: “Cheese omelette. Your favourite.” That is why the excerpt begins with “Coraline's mouth watered.” The book version is replete with a highly graduated positive appreciation of the other mother's breakfast preparations for Coraline. The sizzling bacon “smelled wonderful,” the cheese omelet was “a perfect omelette shape,” the orange juice was “freshly squeezed,” and there was “a mug of frothy hot chocolate.” In the movie, there is no orange juice, hot chocolate is visible on the table, we hear the bacon sizzling and can picture it, and the omelet is prominent on the table. While some appreciation of the food might be inferred from the sounds and images, there is no movie equivalent of the highly graduated explicit, positive verbal appreciation of the breakfast that we see in the book version.
In these different renditions of the same story segment that maintain the same ideational meaning, how do the subtle nuances of interpersonal meaning influence the variation in the interpretive possibilities of the different story versions? The movie version of this segment seems to downplay the attraction of the material indulgences offered by the other mother, which led Coraline, and consequently her parents, into their current captivity. While both versions portray the superficiality and delusory nature of the other mother's interpersonal relationship with Coraline and at least suggest her moral callousness, only the book indicates the demonic basis of this. Thus, in this segment, the movie presents the nature of Coraline's struggle as pitting her determination and wit against the power and duplicity of the other mother, whereas the book sustains something of Coraline's original succumbing to temptation and the consequent need to extricate herself from the forces of evil. Of course, the analysis of one small story segment is insufficient for extrapolation to the entire story, but it emphasizes the importance of regarding the book and movie versions of stories, even the same stories, as having their own distinctive interpretive possibilities.
Conclusion
In the narrative excerpts, the portrayal of ideational meanings is consistent across the digital and paper media formats. However, in each case, the emphases on the thematic concerns are distinctively different in the animated versions. This occurs principally through the changes in the interpersonal meanings in the images. The social distance and involvement between characters and the audience in the movie versions are more intimate due to the greater number of close-ups and frontal views of the characters as well as the more frequent contact established through characters looking directly at the audience. In the digital animated versions, the reader/viewer viewpoint is “along with” or “as” the character, whereas, in the book versions, the reader/viewer viewpoint is more frequently that of an outside observer. The visual and verbal expressions of evaluative meanings in the digital animated versions emphasize the characters’ feelings of happiness or sadness, insecurity or confidence, and their judgments of their normality and capacities. However, in the book versions, implicit or explicit ethical and moral judgments are more prominent. The animation of The Lost Thing is more optimistic, showing the agentive role of the thing, respectful collaboration, and shared empathy between the boy and the thing. The digital animation of Coraline aligns the audience with her bravery, tenacity, and ingenuity in overcoming the duplicity and power of the other mother rather than the moral struggle of denying material desires to return to the circumstances and values of her real family. The analyses in this study draw attention to the increased richness of literary experience potentially achieved through exploring the interpretive possibilities of the book and digital animated versions of these narratives as complementary rather than convergent. This is indicative of a new era for the recognition of educational technology as creating distinctive and innovative literature rather than simply existing as an alternative channel for the dissemination of established literature.
For children now growing up in an online multimedia world, their experience of literary narrative largely takes the “multiplicity of media and versions for granted” (Mackey, 1994, p. 19), and hence discussing children's literature as books alone “ignores the multimedia expertise of our children” (Mackey, 1994, p. 17). However, while classroom work with literary texts in new media forms is crucial (Mackey, 1999, 2001a, 2001b; Unsworth, 2006; Unsworth et al., 2005; Unsworth & Thomas, 2014), it cannot be simply assumed that the experience of multiple-versioned stories equips students with an understanding of how the interpretive possibilities of stories are being shaped by the affordances of the different media. Despite many young people being highly adept at using digital media, they are not necessarily adept at understanding how multimedia affordances influence the interpretive possibilities of the texts they are negotiating (Jenkins, 2006; Kellner & Share, 2007; Luce-Kapler, 2007). While contemporary curricula such as the new Australian national curriculum can mandate such outcomes (through the requirement for year nine students to, for example, “examine how authors adapt and subvert text structures and language features by experimenting with spoken, written, visual and multimodal elements, and their combination” (ACARA, 2022, Section AC9E9LA03), the pedagogy needs to draw on semiotic accounts of how meaning is constructed jointly by language and images in different contexts (Martin, 2008; Moya-Guijarro, 2016; Painter et al., 2013; Painter & Martin, 2011; Unsworth, 2007) and use those accounts to address moving image re-versioning of multimodal literary texts in multiple digital formats including animation, iPad apps, and various forms of augmented and virtual reality (Mills et al., 2023; Ngo, 2018; Serafini et al., 2015; Unsworth, 2008, 2014a, 2014c; Zhao & Unsworth, 2017). The analyses in this study can directly inform classroom learning experiences. Teachers can model the comparative analysis of one common book and movie excerpt, such as The Lost Thing feeding scene, and then collaboratively and comparatively analyze a second excerpt with students, such as the departure. Students could then undertake an independent comparative analysis of a third excerpt of their choice. By infusing this kind of work into teacher education and working with teachers and students in classrooms, we can develop and refine effective educational semiotics for digital multimodal literacy in the 21st century. Animated movies of literary picture books provide a very enjoyable and productive context in which to progress such work.
Footnotes
Author's note
The screen shots and images from the animated movie of The Lost Thing in this article have been authorized by Passion Pictures Australia Pty. Ltd. and HACHETTE AUSTRALIA PTY LTD. The images from the animated movie of Coraline in this article have been authorized by CMO & SVP Operations - LAIKA LLC.
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
