Resemiotization of Illustrations in Children’s Picture Books Between English and Arabic

Abstract

Translation is no longer limited to transfer the meaning contained in a language sign; it includes instead the transposition of meaning from one non-verbal sign system into another. Children’s picture books are multimodal texts that include two different semiotic modes; the text itself and the illustrations. The present study aims at investigating the extent to which illustrations (non-verbal signs) in children’s picture books are translatable/resemiotized from English to Arabic and whether this translation/resemiotization process is considered as an intersemiotic translation. To achieve the objectives of this study, Charles Peirce’s theory of signs was adopted. Six examples of illustrations and their Arabic translations were collected from the Diary of a Wimpy Kid series. The selected illustrations, their meanings, and their relationships were analyzed using Peirce triadic sign model. The results revealed that illustrations in children’s picture books are translatable to a great extent, their translations are intersemiotic, and that they go through resemiotization process. Because illustrations are non-verbal signs which can be analyzed and broken down into representamens, objects, and interpretants (according to Peirce), thus can be translated into other non-verbal signs in any other language.

Introduction

Communication is a basic process that allows individuals to exchange thoughts, concepts, and sometimes, emotions. This act of sharing and generating meanings can be carried out using words, sounds, moves, signs, and symbols. Each community has its own language and culture. Translation helps understand ideas in a language different from the one they are originally conveyed in. Translation plays an important role in facilitating connection and intercultural communication since it enables the process of understanding thoughts, beliefs, habits, and values of different nations by conveying ideas across languages. However, translation is not only limited to verbal signs or languages, but also includes translating pictures, symbols, signs, and other non-verbal systems through what is known as intersemiotic translation.

Roman Jakobson first mentioned the concept of intersemiotic translation in his paper titled “On Linguistic Aspects of Translation.” According to him, there are three types of translation: intralingual translation, interlingual translation, and intersemiotic translation. Intralingual translation refers to transferring meaning of verbal signs to other verbal signs of the same language. Interlingual translation involves the interpretation of verbal signs from one language to another. However, intersemiotic translation (transmutation) “the interpretation of verbal signs by means of signs of nonverbal sign system” (Jakobson, 1959, p. 233 as cited by Jia, 2017, p. 32), such as translating a literary text to a painting, or a film, or a piece of music.

Intersemiotic Translation

O’Halloran et al. (2016) point out that intersemiotic translation is the foundation of cultural communication offering experiences throughout processes of resemiotization, since “it is possible to conceptualize shits of meaning across semiotic resources which are fundamentally different in nature to reveal which meanings are retained and changed as a result of resemiotization.”(O’Halloran et al., 2016, p. 225). The inter semiotic translation is a semiotic and iconic-dependent process, illustrating the action of signs. Aguiar and Queiroz (2009) argue that the issue in intersemiotic translation is that it is hard to compare different semiotic resources, thus they propose to adopt a conceptual framework founded on Peirce’s model of sign process resolve this methodological difficulty.

Oittinen (2001) conducts a study dealing with the visual in of picture books being iconic texts, describing texts formed by images, effects and, words, meaning the verbal and visual semiotic systems. Oittinen (2001) discusses the translation of verbal and visuals in of picture books in the light of Peirce’s theory of sign since the picture books includes the verbal, the visual, and the effects. Translators deals with illustrations by attempting to match text and illustration with each other, meaning “they - either consciously or unconsciously - have internalized the images from their reading of the words and illustrations. . .To be successful, translators need to know how to interpret the whole embedded in all these different ‘languages’” (Oittinen, 2001, p. 116). Oittinen (2001) sees that translator of picture books need to be aware of the ability to read visual elements such as illustrations, images, effect, and have a thorough knowledge and awareness of language and culture, in addition to the role of situation.

Before going through the translating and transferring signs from a language to another, several key concepts regarding the elements of context of situation needs to be tackled. Context of situation contains three aspects: field, tenor, and mode. Field indicates to what is being spoken or written and what is the aim of the text. Then, tenor which refers to the relationship between the speaker and the receiver, who is talking to whom. While Mode refers the manner and form of communication “what is being achieved by the text in terms of such categories as persuasive, expository, didactic and the like. Collectively the three aspects of situational context are called register” (Hu, 2010, p. 324). Halliday (2002 [1977], p. 58) sees that:

The principle is that each of these elements in the semiotic structure of the situation activates the corresponding component in the semantic system, creating in the process a semantic configuration, a group of favored and foregrounded options from the total meaning potential that is typically associated with the situation type in question. This semantic configuration is what we understand by the register. Giving that this study is dealing with picture book for children, then the mode of the text relies on both written text and signs (intersemiotic language), while the tenor represents the relationship between the author of picture books and the receivers who are the children. And the field represents children literature. Transferring the sense of a sign from one language to another involves the process of determining the semiotic properties of the relationships between the source and the target signs. Semiotics is the general science of communication through signs and sign systems, which was introduced by de Saussure (2016). A sign comprises the signifier and the signified. The signifier is the acoustic image or the word, while the signified is the concept that the signifier stands for. In another words, the sign has two aspects, the signified “concept” and the signifier “sound-image” which the relation between both is considered arbitrary. The translation at the verbal level consists of translating the signifier into the same signified. The feature of arbitrariness is what makes translation possible in the relation between the signifier and the signified.

Signs can be found in many different forms, such as illustrations or pictures that are widely used in children’s picture books because they are considered a very attractive element for children. According to Oittinen and Davies (2008, p. 6): “all the different signs can be found in a picture book. A picture is an icon; a picture of a girl resembles a real girl. A word in a picture book is a symbol based on agreement.” Picture books combine visual and verbal signs where both words and pictures play a role in conveying meaning. The colorful pictures included in children’s books are called illustrations.

A children’s picture book is a multimodal text by nature; it contains words and illustrations that both play a key role in communicating the message or idea. This type of books is designed applying a multimodal approach that makes the meaning through different semiotic modes. According to Tuominen et al.:

Multimodality is typically defined from one of two perspectives: it can be described either as the coexistence of multiple modes within a particular context or as the process of decoding the coexisting modes from a viewer’s or a reader’s standpoint. The latter emphasizes that coexisting modes do not actually interact unless they are being interpreted by someone; multimodality is understood as being the interaction of modes in the cognitive system of the viewer or the reader (2018, p. 2)

Picture books allow children to get accustomed to new words and to build up their vocabulary through both verbal and visual references. They open the door for children to understand the world. Picture books often trigger children’s imaginations and help them think of new ideas and bring new possibilities into their lives (Hladíková, 2014). Translating picture books is not a regular translation, but an intersemiotic one according to Oittinen and Davies (2008) as it involves transferring the verbal and visual elements in picture books, which is understood as a semiotic process.

The present study investigates the intersemiotic translation of visual signs (illustrations) in children’s picture books; the non-verbal signs by means of other non-verbal signs, by applying Peirce’s theory of signs as a model. The current study attempts to find answers to the following questions:

1. To what extent are illustrations in children’s picture books translatable from English into Arabic?

2. How the meaning of illustrations is resemiotized from English into Arabic through intersemiotic translation applying Peirce’s model?

Materials and Methods

Data Collection

The sources of the data for this descriptive and qualitative study are six examples taken from original English versions of two books selected from the Diary of a Wimpy Kid series. Diary of a Wimpy Kid is a popular series of books originally written in English and illustrated by American author Jeff Kinney (2008). The books are journals of the protagonist, the wimpy kid Greg Heffley. He writes about his daily life and events with his family and friends. The books that were chosen from the series for this study were Rodrick Rules (2008) and Hard Luck (Kinney, 2013). In addition, their Arabic versions, which were translated from the English version by Zaina Idris and published by Arab Scientific Publishers, Inc., were also selected. The books selected were Qwānyn Al-āẖ Al-ākbr (Kinney, 2012) and Al-ḥẓ Al-ʿāṯr (2012).

The series was selected because of its global popularity as children are able to relate to Greg’s life, which is described in detail, and the challenges he faces at home, school, and with his brothers and friends. There is a great number of illustrations in each book that support the scene and tone of the events. These illustrations have the power to convey how the main characters feel about the actions that take place in the story and help show emotions on the character’s faces. The illustrations also include a number of signs and symbols that reflect the protagonist’s culture and lifestyle. The researchers selected six examples of illustrations from the original books and the target translation along with their context for the sake of analysis (see Appendices A and B).

Data Analysis

This study aims at investigating and analyzing the signs or illustrations that had been translated into Arabic. To answer the research questions, all illustrations were analyzed based on Peirce triadic sign model. Each example in a figure divided into two sections, the first included the application of the Peirce triadic model on the source illustration and the other included the application of the Peirce triadic model on the target illustration. Under each figure, there is a discussion of the components of the source and target signs, and a description of the discrepancies between them as a result of the intersemiotic translation process.

Some issues may encounter the translator when translating and resemiotizating verbal and visual elements in the children picture books, mainly, regarding the cultural gap between Arabic and language users. In this study, many examples demonstrate crucial issues like taboos and untransability of some of verbal and visual elements that are considered natural for the source language users, contrary to the target language users who considers some encounter as taboos and untranslatable items.

Taboo words, or signs are often seen as offensive, rude, or inappropriate. Almost all languages include taboo words or signs which are derived from culture or linguistic specific sources (Al-Yasin & Rabab’ah, 2019). Catford (1965, p. 94) distinguishes between linguistic and cultural untranslatability: “in linguistic untranslatability the functionally relevant features include some which are in fact formal features of the language of the SL text. If the TL has no formally corresponding feature, the text, or the item, is (relatively) untranslatable” (cited in Al-Yasin & Rabab’ah, 2019, p. 23). The translator may opt to perform several strategies when dealing with translating taboo and untranslatable items throughout: (1) omission and censorship, (2) substitution, (3) rendering taboo for taboo (finding an adequate equivalence in the target language, and (4) using euphemism.

Theoretical Framework

The present study adopted Peirce theory of signs as its theoretical framework. According to him, an image, a written word, or a thing are considered a sign, irrespective of whether it is simple or complex, once it becomes a part of the semiosis process. There are three interrelated parts that constitute the sign, namely, the representamen, which is the form the sign takes, the interpretant, which is the sense made of the sign, and the object, which is the representation of the sign. The representamen is the form of the sign. What the sign represents is called the object, while the interpretant is the sense, idea, or implication of the sign. The sense made by this sign (interpretant) provides further determination of the meaning mediated between the relation of the representamen and object as seen in Figure 1 (Chandler, 2017, p. 30).

Figure 1.

Peirce’s model as a semiotic triangle.

Peirce (2001, as cited in Atkin, 2013) also introduces three stages of the semiosis: Firstness, in which the translator understands and gets the first impression of the picture book to be translated, the Secondness when the translators start inferences and conclusions from the story, then Thirdness, when s/he brings up the new interpretations and make the suitable translation.

The illustration in a children’s picture book is the sign that has been investigated by applying Peirce’s sign triadic model as seen in Figure 1. In order to understand what the illustration (the sign) depicts, the relationships among its components were analyzed. In other words, the representamen, object, and interpretant were explained on the triad and each component function was identified. After recognizing the message behind the illustration and understanding the relationships between its parts, the illustration can be resemiotized through intersemiotic translation into Arabic context.

Consequently, the act of translating the original English illustration into Arabic is intersemiotic and occurs through resemiotization as it goes between these two non-verbal signs and it is possible while understanding the relationship between the sign components. Here lies the key role of the translator in rendering the illustration and conveying its meaning accurately into Arabic using semiotic information with respect to the target culture in a way that enhances the communication in English and Arabic. Consider this illustration in the English version as seen in Figure 2.

Figure 2.

Source sign.

To understand the sense of the sign (illustration), we need to identify the function of its components as shown in Figure 3.

Figure 3.

Peirce triad application of the English illustration.

As Figure 3 shows, the representamen forms the illustration, the object represents the signage of pork bellies, and the interpretant is the understanding that this is a place where people order and eat pork. The understanding of what the interpretant represents lead the translator to an additional triadic Arabic sign (see Figure 4).

Figure 4.

Peirce triad application of the Arabic illustration.

This example includes interlingual translations where the translator replaced the signage from “Pork bellies” to “traditional food,” because of restrictions on the consumption of pork and pork products for Muslims, being this a taboo, the translator chose to translate to “traditional foods” referring to the foods the public are accustomed to. Thus, the representamen is the illustration, the object represents the signage of traditional food and the interpretant is the sense made that this place offers traditional food options.

By considering the sign relations, it is possible to develop a new non-verbal sign (illustration) in Arabic that is acceptable in the Arabic target culture. After discussing the process of intersemiotic translation and how it is possible, the researchers added Figure 5 to give a final look of how the illustration appeared in both the source version in English and the target version in Arabic:

Figure 5.

Source and target illustrations

Discussion

The Intersemiotic Translation of Illustrations as Resemiotization Applying Peirce’s Model

The set of meanings in this illustration is resemiotized through different semiotic modes. In addition to the researcher’s explanation on the application of Peirce’s sign triadic model, a figure comprising two parts, namely the source illustration and the translation is provided where a diagram of the Peirce triad as applied to each illustration (as they appear in the books from which they were extracted) is presented as shown in the examples (Figure 6).

Figure 6.

Peirce triad application of the source and target illustrations for example 1.

In this example, for the source illustration, the object is the concept that “Greg’s Parents are watching a kissing scene on TV.” The sign represents this. The representamen is the illustration itself and the interpretant or response to the sign is the sense made by the interpreter while understanding that there is a couple kissing on a TV show. This means that what is being shown on TV is a romantic scene filled with passion and affection that may be thought of as the reader’s response to the illustration.

On the other hand, in the triadic model of the target illustration, the object is the idea of parents watching TV together. The representamen is the illustration itself. The interpretant forms a scene shown on TV.

As noticed, the phrase “kissing scene” is omitted due to the cultural taboo of demonstrating kissing references especially to children. As the target Arab Muslim community does not accept the act of showing affection and kissing between couples on TV. To tackle this cultural problem, translation by omission strategy was used to deal with the cultural disparities between the source and target audiences (Figure 7).

Figure 7.

Peirce triad application of the source and target illustrations for example 2.

Taking into consideration the context around the illustration, the protagonist Greg has a science paper due, in which he has to choose an animal to talk about, as instructed by his science teacher Mrs. Breckman. He chose the moose and began writing about it without even going to the library to find information. Greg concluded his paper saying that “Everybody knows that the moose evolved from birds, just like people did. Somewhere along the line, people got arms, and the moose got stuck with those useless horns” (Kinney, 2008, p. 137).

The content of the source illustration presents the idea of a human being having evolved from a bird into a person much like the moose (representamen). The object is this concept of human beings having evolved from birds while the interpretant relates to human evolution, which implies the process by which human beings evolved on earth. It is believed here that humans were birds first, and then evolved with different characteristics. Similarly, it is believed that the moose was originally a bird. The point is that human beings are equated to the moose with respect to evolution.

However, in the translation, the object is the concept of the evolution of the moose, as shown in stages in the illustration (representamen). The understanding that the moose evolved from birds is the interpretant. It is noteworthy that the translator omitted the key idea of the source illustration, that is, human beings evolved from birds just like the moose did (source interpretant).

The translator changed the entire idea of the illustration by omitting the concept that humans evolved from birds because that idea is not acceptable among Arab audiences, especially for children. This is from a religious perspective because of the belief that Allah created humans. To overcome this cultural-religious problem, the strategy of modulation was applied to change the entire perspective in the source illustration.

Figure 8 shows the source illustration that functions as the representamen. It represents the concept of having all family members doing different things in the same place and at the same time (the object). For example, Rodrick is sitting on the toilet clipping his nails. The interpretant is the understanding that Greg is thinking about what it means to live together while sharing everything in the same house.

Figure 8.

Peirce triad application of source and target illustrations for example 3.

However, in the translation of the illustration, the Arabic representamen and the object are the same as the source sign, which are the interpretants for the same concept that Greg is thinking about what it means to live together. However, the translator omitted the image of Rodrick sitting on the toilet and clipping his nails for religious and cultural reasons. This problem led the translator to rely on the strategy of omission because of the culture-specific sensitivity around the human body among the target readers (Figure 9).

Figure 9.

Peirce triad application to the source and target illustrations for example 4.

The context and narration of events in example (4) explain that Greg was asked to take pictures of the Class Favorites winners as part of award ceremony. As shown in the source illustration, the last two people walked into the room to have their pictures taken. The object in this example is the impression that Greg feels terrible when he sees his friends holding hands as a couple. The representamen is the illustration itself and the interpretant is the meaning behind holding hands, which is being together for each other having a strong connection. It thus involves a cultural cue that is particular to the Western world.

The main difference between the source and target illustration is that Greg’s friends are not shown holding hands to the Arab audience. In other words, the translator chose to separate the couple’s hands in the Arabic illustration because of religious preservation. Therefore, the object is the impression that Greg feels terrible when he sees his friends as a couple. The interpretant is the effect produced by means of the sign, which is different in each version. It is that Greg feels jealous when he sees them together whereas the source interpretant forms the idea behind a couple holding hands (Figure 10).

Figure 10.

Peirce triad application to source and target illustrations for example 5.

In example (5), the representamen in the source illustration functions as a book cover of the object, which is a cover for a book about boyfriend trouble. It shows a boy giving his girlfriend flowers. The interpretant forms the symbol of romantic love.

On the other hand, the translation, that is, the illustration in the Arabic version forms the representamen of the cover of a book about friendship trouble (the object). The understanding that a boy and a girl are looking at each other while another girl watches them with surprise is the interpretant. A key point to mention here is that the symbolism of romantic love is implied by receiving flowers in the source sign, and this is omitted in the target sign. The translator aimed at substituting the idea of romantic relationships and release it with a much natural and culture-friendly idea, that is to say friendships, thus rendering the title “boyfriend trouble” as “friendship trouble,” and omitted the image of a boy giving a girl flowers. This comes as a way to overcome cultural taboos of romantic relationships outside of marriage in the target culture, where having a boyfriend/girlfriend is not allowed.

The context explanation is necessary to discuss this example. Greg went over to his friend Fregley’s house and knocked on the front door to call him to walk to school together. He heard some noises inside. Fregley told him that he had accidentally put on his shirt upside down. He appears as shown in Figure 11, which functions as the representamen in the source illustration. The object is the view of Greg helping his friend, whose pants have fallen down whereas the interpretant presents how Fregley looks with no clothes on except his underwear and t-shirt.

Figure 11.

Peirce triad application to source and target illustrations for example 6.

In the translation, the sign has the same semiosis for both the source and target illustrations. However, both the source and target interpretants are translations for the same idea, that is Fregley’s shirt is stuck and that he consequently appears with no pants in the picture, the translator opts for employing naturalization strategy to render it by covering Fregley’s legs so that he appears dressed. The reason behind this is a cultural peculiarity for Arabic readers that does not accept one’s appearance in an unacceptable manner.

The Extent to Which Illustrations in Children’s Picture Books Are Translatable From English into Arabic

Based on the findings in response to the first question and as discussed in the previous section, illustrations in children’s picture books are translatable as each is a sign that comprises three elements (representamen, object, and interpretant) according to the Peirce sign model. The object and the interpretant interact with the representamen to convey the meaning of the sign.

The representamen can be changed into another illustration in any other language and can thus be translated after analyzing its relationship with the object and interpretant. These arbitrary relationships between the sign parts make the intersemiotic translation possible.

This can be applied to illustrations (pictures) in children’s picture books first, by considering the content of the source sign. Then by analyzing the relationship between its components (in order to understand the original idea that the sign seeks to address), then by translating it in a way that conveys the message properly, and finally by producing a new sign (target sign) in Arabic.

How the Meaning of Illustrations Is Resemiotized From English Into Arabic Through Intersemiotic Translation Applying Peirce’s Model?

It is apparent from the examples above that the interpretant is the most essential part of the sign in the context of a translation, wherein it forms a sense of the sign that should be transferred into Arabic. Lambert and Robyns (1997) (as cited in Bezuidenhout, 1998) emphasized that the interpretant itself is a sign that can be interpreted. They described it as the translation of the sign, while some scholars consider interpretants as translations in light of Peirce’s insights and have even emphasized that interpretants need to be treated as translations (Atkin, 2013, p. 3).

In summary, the possibility of translating signs (illustrations) in children’s picture books from English into Arabic with the same effect while taking into consideration the role of semiotics makes an intersemiotic translation of illustrations possible. According to O’Halloran et al. (2016, p. 225), “it is possible to conceptualize shifts of meaning across semiotic resources which are fundamentally different in nature to reveal which meanings are retained and changed as a result of resemiotization.”

Results

In light of Peirce’s views, the intersemiotic translation between the illustrations is described as a process of semiosis. From the perspective of the multimodal approach to intersemiotic translation, there is a considerable conceptual overlap between both concepts: intersemiotic translation and resemiotization. In this study, intersemiotic translation refers to transferring the meaning between semiotic modes and this is associated with both concepts, in which intersemiotic translation goes through resemiotization processes. Therefore, intersemiotic translation can go beyond Jakobson’s definition in which the process of transferring meaning between two non-verbal semiotic modes can be included. Illustrations are resemiotized through multimodal texts from one semiotic system to another. Iedema (2003, p. 29) stated that intersemiotic translation as “resemiotization” is mainly involved in “how semiotics is translated from one into the other as social processes unfold.”

Conclusion

The present study concludes that the translation of illustrations in children’s picture books is an intersemiotic one that goes through resemiotization processes since it mediates between two non-verbal sign systems. The sense of a sign is not captured easily by looking at it. Thus, identifying the representation and interpretation of the components of the sign is a means of realizing the meaning embedded in it. As Noroozi and Tork (2017) emphasized the importance of semiotics and that, the meaning of a sign arises in its interpretation, but not contained within it only.

The study also concludes that procedures like omission as relied on in translations between languages can be used in intersemiotic translation. Saeedipour and Sharif (2016, p. 471) noted that “Translating a text from one language to another, some parts of the text is simply omitted. The same process happens in intersemiotic translation.”

More studies and research should be conducted regarding exclusive procedures and strategies for translating illustrations in children’s picture books between English and Arabic to properly transfer the intended meaning of an illustration.

Under the direction of the first researcher, this study was based on the second researcher’s MA thesis, which is extended in exploring the potential cultural hindrances translators encounter when translating illustrations and evaluating the degree of equivalence between the source and target illustrations. Moreover, there are a limited number of examples of the illustrations included in this paper, the most relevant ones from the MA thesis.