Abstract
This article investigates the relationship between young people’s game-making practices and meaning-making in videogames. By exploring two different games produced in a game-making club in London through a multimodal sociosemiotic approach, the author discusses how semiotic resources and modes were recruited by participants to realize different discourses. By employing concepts such as modality truth claims and grammar, he examines how these games help us reflect on the links between intertextuality, hegemonic gaming forms and sign-making through digital games. He also outlines how a broader approach to what has been recently defined as the ‘procedural’ mode by Hawreliak in Multimodal Semiotics and Rhetoric in Videogames (2018) can be relevant for promoting different and more democratic forms of meaning-making through videogames.
Keywords
Introduction
Throughout the last decades, we witnessed ‘the institutionalization of video game practices, experiences, and meanings in contemporary society, which places video games and video gaming as an important part of our social imaginary’ (Muriel and Crawford, 2018: 19). This process of institutionalization helps us explore the relationship between videogames and meaning-making through discourses, ‘socially constructed knowledges of (some aspect of) reality . . . developed in specific social contexts, and in ways which are appropriate to the interests of social actors in these contexts’ (Kress and Van Leeuwen, 2001a: 5).
Discourses mediate communicative acts within a certain context: in videogames, for example, Consalvo and Paul (2019) explore how ideas around the ‘most legitimate ways’ of engaging with games are socially constructed in the public arena. Discourses are not only limited to influencing the public imaginary about a specific context, but can also mediate how meaning-making processes are carried out in a specific context: different authors (Bogost, 2006; Flanagan and Nissenbaum, 2014; Freedman, 2020) identify how certain values can find their ways into semiotic resources in gaming, for example. By becoming enmeshed in the material conditions for meaning-making, these values influence the regular uses – ‘grammars’ (Kress and Van Leeuwen, 2002) – within a certain domain. But how limiting are these ‘grammars’? Can we subvert sociohistorical uses of semiotic resources in an intelligible manner?
In this article, I focus on young people’s game-making practices, investigating how inexperienced game-makers worked within this heavily culturally-coded field to produce their own games. By examining the different strategies adopted by participants to produce games, I intend to clarify in which ways discourses, conventions and ‘regularities’ in semiotic work influence game-making practices.
To investigate this issue, I focus on cases from a game-making club organized in London, UK, between October 2017 and January 2018, analysing these through a multimodal sociosemiotic approach (Kress, 2010; Kress and Van Leeuwen, 2001a). In the next section, I will outline the conceptual framework that grounds the analytical approach for this article, with special attention to previous research on multimodal sociosemiotics and videogames (Hawreliak, 2018; Pérez-Latorre et al., 2017).
An Overview of Social Semiotics and Multimodality
Social semiotics is a social theory of meaning-making, interested in the contextual uses of different semiotic resources (Gualberto and Kress, 2019). This social aspect of meaning-making is evidenced, for example, through the epistemological position that considers that all signs are motivated. According to Kress (2010: 10), motivated signs are the expression of the interest of socially formed individuals who, with these signs, realize – give outward expression to – their meanings, using culturally available semiotic resources, which have been shaped by the practices of members of social groups and their cultures.
These semiotic resources are communicated through modes, sets of ‘resource[s] for making signs in a social–cultural group’ (Kress and Van Leeuwen, 2002: 346). Modes are combined in a unique semiotic event to produce a semiotic ensemble (Kress, 2010), and multimodality is the perspective that is interested in the meaning-making processes implicated in these communicative acts (Gualberto and Kress, 2019; Hawreliak, 2018).
Multimodal sociosemiotics explores how sign-makers recruit and arrange different semiotic resources to fulfil particular social functions (Kress, 2010). In this process, signs are never used, but always made, since they simultaneously invoke certain meaning(s) and resignify their possible uses. The intelligibility of these (new) signs is dependent on the conventional regularity found in the use of these resources, or ‘grammar’ (Kress and Van Leeuwen, 2002).
In this article, an important element for understanding young people’s meaning-making processes through game-making is socio-semiotic modalities. Modalities are truth claims established by texts: they can be understood as socially constructed ‘frames’ that help interpretation (Kress and Van Leeuwen, 2001b). Kress and Van Leeuwen list four main modality coding orientations, which refer to the strategy – the domain invoked – to construct these claims: naturalistic (produced in relation to the natural world), sensory (appeals to emotions and sensory aspects), abstract (constructed through essentialization of meanings through signs, such as scientific models) and technological (based on a functional relationship); usually these coding orientations are combined to construct truth claims.
In this article, I examine how games produced in non-mainstream contexts are often subjected to the institutional position that reifies dominant gaming practices through the available modal affordances and semiotic resources, sometimes forcing sign-makers to repurpose their messages to operate within the medium of videogames.
Social Semiotics, Multimodality and Videogames
No communication event is created or perceived in a vacuum. When we consider which modes are employed and which are not, or how . . . a particular videogame chooses to represent a group of people, system, or power structure, we must take note of the myriad contextual factors that both drive these choices and potentially influence a player’s perception. (Hawreliak, 2018: 5)
Multimodal approaches were applied to several research objects, from graphic design (Kress and Van Leeuwen, 2001b) to films (Burn, 2013) to social media (Moschini, 2018); however, (multimodal) social semiotics perspectives were seldom invoked to explore videogames (Hawreliak, 2018). Pérez-Latorre et al. (2017) relate this gap to an ontological understanding about games. Following the historical constitution of Game Studies as an academic field through the ludology/narratology divide (see Pearce, 2005), videogames can be considered part procedural, part representational (Sicart, 2013), with players interacting with and interpreting these two layers to construct meaning. While this model acknowledges a game’s procedural core (e.g. mechanics, rules, the computer code itself) and its representational aspects (e.g. images, audio), as well as the player as interpreter, it can easily confine semiotics to the representational layer (Pérez-Latorre et al., 2017). Semiotic approaches to digital communication often prioritize the ‘most salient’ modes – those that are easier to access, such as graphics and audio (Hawreliak, 2018) – while neglecting what is happening ‘behind the scenes’, for instance, at the level of computer code and platforms (Moschini, 2018). How to overcome this salience-oriented approach, then?
Recent studies (Hawreliak, 2018; Pérez-Latorre et al., 2017; Toh, 2018) proposed sociosemiotic approaches for videogame analysis. Pérez-Latorre et al. (2017), for example, argue that game design can be understood as a mode. Even if contestable, 1 Pérez-Latorre et al.’s model plays an important role in demonstrating how the most evident semiotic resources for game design (e.g. game mechanics and rules) have socially shaped signifying potentials, as happens with any mode (Kress, 2010). Moreover, their model also bridges the gap between ‘procedural’ and ‘representational’ layers, providing a more holistic model for game analysis through conceptual constructs such as the ludo-narrative dimension of videogames – ‘the representational value of game design, and its connection to audiovisual narrative’ (Pérez-Latorre et al., 2017: 591). This ludo-narrative construct is one of the central analytical concepts adopted by Toh (2018) in his research on videogame players’ experiences, and is used to operationalize the meaning-making processes made by players when engaging with the different modes found in videogames.
While these models bridge the gap between ludic and representational layers of gaming, they might not acknowledge all the different modes involved in meaning-making processes through videogames. Pérez-Latorre et al. (2017) name theirs a ‘social semiotic theory of videogame design’; however, while they recognize the relevance of procedural rhetoric (Bogost, 2007) for meaning-making in videogames, their model ends up obfuscating the role played by digital technologies and, more specifically, by programming, leaving aside the digital nature of videogames (Bogost, 2006, 2007). This model should acknowledge ideational (representational), textual (compositional) and the interpersonal (Kress, 2010) functions of the digital, including its ideological dimensions (Freedman, 2020), such as the enmeshment of institutionalized gaming practices in technologies, shaping semiotic resources and their possible uses (Flanagan and Nissenbaum, 2014), as happens, for instance, in the case of game engines (Bogost, 2006; Freedman, 2020).
Hawreliak (2018) combines procedurality (Bogost, 2007) and multimodality (Kress, 2010) to propose a ‘procedural mode’, ‘the expression through interactive, rule-based systems enacted by a player’ (Hawreliak, 2018: 82). The procedural mode, therefore, would be the socially shaped set of resources to communicate meaning through computer-based procedures: in a game such as Papers, Please, the weight of a bureaucratic, totalitarian state is not only conveyed through ‘representational’ modes (e.g. music, colour, images, written text), but also through the game mechanics, including here the universe of possible actions inside the game (e.g. acting as a border officer). More importantly, each of these modes presents specific affordances and conventional regularities in its use – a ‘grammar’ (Kress and Van Leeuwen, 2002) – culminating in signifying potentials for each motivated sign produced by a rhetor.
While this model affords a thorough investigation of rhetoric in games, articulating the uses of modal consonance and dissonance for ideological persuasion (Hawreliak, 2018), the inclusion of the procedural mode to videogames does not completely acknowledge the role played by the digital for meaning-making in games. This obfuscation becomes clear when Hawreliak (2018: 85) reminds us that rules and systems must ultimately be translated in a programming language to be implemented in a videogame. At this point a clear challenge arises. Apart from computer programmers and computers themselves, this mode of expression is esoteric and ultimately unintelligible until it is tied to other forms of representation, such as auditory and visual modes.
Here, Hawreliak assumes the position of the sign interpreter (player): when playing a videogame, we seldom have access to all semiotic resources (including the code) that are creating the experience. But we cannot forget that the game developer – the sign-maker – also must engage with the procedural mode when producing a game, transducing semiotic resources into specific modes and making use of the available affordances to communicate meanings. These semiotic resources are subjected to different discourses and can shape the meaning-making possibilities of a sign maker, ultimately limiting the kinds of meanings this rhetor can produce.
Moreover, the stance adopted by Hawreliak focuses on the persuasiveness of a text, leaving aside poetic aspects and the aesthetic functions these semiotic processes can have (Burn, 2021). This is why I echo the call made by other researchers about the importance of exploring ‘technical’ processes for meaning-making, no matter how esoteric they might seem (Moschini, 2018).
This article investigates the influential role played by these ‘technical’ elements in meaning-making. This meaning-shaping process, found in every communicational event, can become more evident in situations where sign-makers are not all completely familiar with the modes (and ‘grammars’) available for meaning-making. This situation often culminates in tentative communicational processes that are influenced not only by discourses, but also by the resources being employed in these communicational acts.
By investigating two cases, I discuss how affordances and constraints might shape meaning-making strategies. This discussion supports reflections about the possibilities and limits for the resignification of semiotic resources’ meaning potentials within digital game production.
Methods
In this article, I rely on data from recently concluded research investigating expressive practices in game production, with attention to how platforms and game design conventions actualize discourses and influence young people’s game-making. The cases explored here come from a game-making club organized in a community-led centre 2 in London, UK, between October 2017 and January 2018. Participants had 12 hours of activities to conceptualize and develop games using MissionMaker, software that allows designers to produce 3D videogames quickly through simplified programming logic and pre-made 3D assets (e.g. environments, props, characters) (de Paula, 2021). In both contexts, I was responsible for organizing the content, preparing and leading the sessions, and generating data that was later analysed. 3
Sessions usually started with an initial, shorter (around 30% of the available time) instructional section, which covered a specific topic related either to game production (e.g. rules, game mechanics, character development) or to the use of the software (e.g. how to import external media files, or how to program non-playable characters’ movement). After these initial instructions, participants worked on their games with my support.
I employed different methods to generate data. Throughout all sessions, I kept an audio 4 recorder registering all interventions and discussions with participants, leading to conversations that were later considered unstructured interviews. Moreover, at the end of every session, I collated all participants’ material production, such as storyboards and project files of their games, creating a game archive. This archive, in a similar fashion to that explored by other research on young people’s game-making (see Pelletier, 2008), allowed a progressive analysis of game ideas, tracking down how their games evolved in a week-by-week basis, as well as informing semi-structured interview schedules based on participants’ game development. These semi-structured interviews were carried out outside activity time either before or after sessions.
This data, comprising unstructured and semi-structured interviews and game-related material was analysed through a multimodal sociosemiotics framework (Kress, 2010; Kress and Van Leeuwen, 2001a). By reconstructing the evolution of their games, including how ideas shifted from time to time as they progressed in the use of MissionMaker, I was able to investigate how different elements – from perceived technical affordances to understandings about videogames – were invoked by participants in their games.
In this article, I focus on how semiotic resources and modal affordances were articulated in their games, exploring the relationship between the available semiotic resources and their (regular and contraventional) uses, outlining the idea of ‘game grammars’. To investigate these productive processes, I examine two games: Extrovertido, a game produced by two young women, Marta (aged 14) and Carla (aged 15), 5 and Experiment Z, developed by two young men, Yerry (aged 13) and Juan (aged 17).
Extrovertido is an adventure game loosely inspired by Orange Marmalade, a South Korean teen drama by KBS TV adapted from a homonymous comics series. In a world where vampires and humans coexist, the player steps into the shoes of Baek Ma-Ri, a shy vampire who sees her romantic interest, the human Jung Jae-Min being kidnapped by her rival Ah Ra. The player’s aim is to search and find their romantic counterpart within the maze-like environment created by Marta and Carla, and fight Ah Ra to rescue Jae-Min; what happens to the young couple after the reunion, however, was subjected to different iterations, with Marta and Carla trying out different endings until reaching a final version that included a significant plot twist.
Experiment Z, on the other hand, is constructed around Sherlock, an accomplished scientist who seeks the truth after a terrible experiment from Mar Industries, his old company, has gone wrong. Elsare, Sherlock’s boss, sabotaged the main character’s samples in a genetic experimental part of Project SRX. This bad gene ended up injected into a specific DNA chain, creating a super-resistant virus that affected all human population except for those who had A-type blood, killing billions (Sherlock’s family among those). Sherlock, therefore, must traverse the game world collecting clues, solving small puzzles using collectible objects and fighting mutants to produce a cure and save humanity from an apocalyptic ending.
In the following sections, I explore how these different productions produce meaning through the actualization of different discourses, foregrounding how ‘grammars’ and available semiotic resources can shape the design processes and meanings conveyed by game-makers.
Modality, Genres and Meaning-making
Extrovertido and Experiment Z have some thematic differences: one is a game about young love and jealousy; the other, about revenge and unscrupulous scientific companies trying to conquer the world. The former is a game produced by two young women who, according to themselves, knew little about and seldom played videogames; the latter, by two seasoned young male players, knowledgeable about gaming platforms and genres.
Despite these differences, designers approached their ideation process through similar strategies since both games rely on similar core game mechanics (e.g. environment exploration, fighting) and on intertextual relationships. These intertextual relationships support the use of different modality coding orientations, producing different truth claims and, consequently, realizing different discourses. But how are these modality coding orientations manifested in their games, and how important are they?
In both games we have a clear use of naturalistic coding orientation. These are, as discussed earlier, the use of the rules of the natural world to judge whether the meanings communicated in these games are credible or not. Besides naturalistic coding orientation, Extrovertido and Experiment Z also articulate abstract coding orientations: an example here is participants’ use of the property hasGravity, which allows game-makers to individually manipulate whether game objects ‘fall to the floor’ or ‘levitate’ in the game world, essentializing natural phenomena such as gravity. 6
These two modality coding orientations, however, are a clear result of the material conditions for game-making found by participants. They are a direct consequence of MissionMaker’s affordances, pertaining, therefore, to the two first layers of a semiotic mode as defined by Bateman (2011), the material substrate and the semiotic resources. To produce a game through MissionMaker, participants depended on the ready-made 3D models that invoked a naturalistic coding orientation, and on the simulated gravity that can be part of any game produced through this software, both conveyed through specific computer code that generates images and audio to be later displayed on-screen – reiterating the relationship between the procedural mode (Hawreliak, 2018) and sign-making (the game-making practice) in videogames.
Nevertheless, as we consider game design as a(n orchestrating) mode (see Burn, 2013; Pérez-Latorre et al., 2017), we cannot forget that there is a third layer represented here, discourse semantics (Bateman, 2011). It is in this layer that sign-makers organize semiotic resources in order to make them intelligible to a sign interpreter, and it is where we can find most of the intertextual relationships articulated to construct different modality coding orientations.
In Extrovertido, Marta and Carla drew on Orange Marmalade, a teen Korean soap opera where vampires and humans coexist, invoking therefore a specific narrative genre, fantasy. Their game should not be solely judged on their credibility against the natural world, but as a fictional work, where fantastic elements exist. This fictional dimension was expressed not only through narrative aspects of their game, but also procedurally represented through different programmed game sequences in their game, with Jae Min’s kidnapping being represented through a ‘magic teleportation’ in the opening sequence of their game, for example.
Experiment Z also relied on fictional modality claims, even if through a different strategy. Rather than seeking a direct link with a specific media text, Yerry and Juan opted for a combination of elements that refer to a longstanding tradition of sci-fi texts, such as the brilliant scientist, the unscrupulous scientific corporation, an experiment gone wrong. As was the case with Extrovertido, these modality claims were not only expressed through narrative elements, but also through some game mechanics implemented in their game, as I discuss in following sections. More importantly, these elements provide a solid reading frame through which audiences can understand their game as one relying on multiple genres – a sci-fi videogame (Krzywinska and MacCallum-Stewart, 2009).
But how important are these combinations of different truth modality claims in their texts? And how do they relate to the different discourses actualized through their game-making practices? In Extrovertido, we have a direct relationship between the game and Orange Marmalade, providing an orientational frame to understand Marta and Carla’s production as a fantasy-based fan text. Experiment Z, on the other hand, is a media text that relies on generic elements from scientific fiction, situating their game in relation to a broader landscape of media texts.
By relying on these already consolidated narrative genres (fantasy, sci-fi) in their games, participants produced new signs based on the previous uses of specific signs (Kress and Van Leeuwen, 2001a), making clear how their game was part of the meaning-making domain they were subjected to at the moment. These intertextual relationships ‘ground’ the meanings being produced by offering reading frames for the discourses (about young love, scientific experiments, gaming, etc.) realized in these games. Nevertheless, the sole use of intertextual relationships might culminate in a cacophony of references rather than an intelligible text.
As discussed earlier, in both games we notice the presence of different modality coding orientations, either through a deliberate design decision or due to the cultural–material affordances found by them. But were participants able to produce intelligible communicational acts? In the following section, I will focus on how Marta and Carla worked with existing meaning-making possibilities to produce Extrovertido.
‘We will think about it’: Grammar and Transductional Issues
Extrovertido helps us understand these relationships between intertextuality, genres, grammars and meaning-making through games. Through character design, Marta and Carla were able to construct intertextual ties with other genres and media, widening their meaning-making possibilities beyond gaming.
Character design was one of the strategies sought by Marta and Carla to construct these logical bonds between their game and other texts. Their version of Baek Ma Ri presents similarities to the one from the series Orange Marmalade, such as the fair skin and deep black hair, as seen in Figure 1. There is, however, a sign that requires further discussion in their redesign of the protagonist: the odango (‘double-bun’) hairstyle.
Baek Ma Ri, as interpreted by Seolhyun in KBS TV’s Orange Marmalade (left, source: KBS 2021 https://kbsworld.kbs.co.kr/program/view.php?pg_seq=933) and as designed by Marta and Carla in Extrovertido (right).
Through the odango hairstyle, Marta and Carla were seeking a different type of relationship between theirs and the TV show’s Baek Ma Ri. Extrovertido, as Orange Marmalade, is a text constructed around Korean characters, and Baek Ma Ri’s odango hairstyle works as a sign for establishing this sense of Korean-ness/Asian-ness. Odango is a hairstyle stereotypically linked to Asian characters’ representation in anime, manga and other media forms, such as Street Fighter’s Chun Li or Sailor Moon. In this case, the odango hairstyle works as a way to reinforce their Ma Ri’s truth claim as Asian (even if through a stereotype). They used the available set of semiotic resources – in this case, the existing hairstyles within the platform – to communicate specific meanings according to the aptness of the accessible modes (Kress, 2010), constructing ties between their game and the TV show.
Throughout most of the development process of their game, Marta and Carla were able to work within the available meaning-making possibilities offered by MissionMaker. Extrovertido was mostly constructed according to the ‘grammatical’ rules of (adventure) videogames. As an example, Marta and Carla implemented a game sequence where the player would meet Ah Ra, the kidnapper of Jae Min, and would be attacked by her, thereby prompting a fight. This fighting mechanics – regularly explored in adventure videogames (Hayse, 2014), and therefore a regular ‘structure’ in this ‘game grammar’ – is reasonably easy to be implemented in MissionMaker, and Marta and Carla appropriated this procedure after some time (as can be seen in Figure 2 below, representing the rule that controls Ah Ra’s hostile behaviour). However, this grammatical stability would be later challenged by the proposed ending for Extrovertido.
Rule programmed by Marta and Carla, making Ah Ra follow and attack the player when the player gets too close (entering the Trigger Volume).
In Marta and Carla’s original idea, the climax of the game would not be the fight against Ah Ra or Jung Jae Min’s rescue, but a following sequence where Baek Ma Ri, noticing that her lover was in a catatonic state, would have to take care of him until his mental faculties were restored. While the proposal of this sequence is interesting for discussing aspects such as gendered roles and identities (Jenson and De Castell, 2018), something beyond the scope of this article, it also highlights the multimodal grammatical nature of game design since ‘what makes a grammar grammar-like is that it has characteristics that can be contravened’ (Kress and Van Leeuwen, 2002: 346). Seeking, fighting and rescuing are ‘grammatically acknowledged’ resources for game design, ‘taking care’ is not – at least not as pervasive as the aforementioned ones. 7 Having a game sequence where the player should take care of an NPC (non-player character), therefore, would be a noticeable ‘grammatical contravention’, at least in relation to the ‘game grammar’ favoured by MissionMaker. These limited modal affordances of MissionMaker are a direct result of mainstream gaming discourses ingrained in the software (Flanagan and Nissenbaum, 2014). Marta and Carla’s struggle with these limited modal affordances to convey the experience they planned can be noticed in the excerpt below:
How can we take care of him?
Oh, this is about . . . ah, yes, the end of your game . . . what did you want her to do?
We could tuck him into a bed and . . .
Hmm . . . we cannot do that . . . remember what we can do here in the game . . . it is in actions [points to the menu on the screen]
Yes, but that . . .
What if she brings some stuff to him, like medicines . . .
I don’t know . . . we will think about it.
This situation culminated in a transductional (Kress, 2010) issue. Marta and Carla were not able to move from their design phase (the ideational, intentional moment) to production (the actual materialization of the signs to convey the desired meaning), transducing the ‘caring’ sequence from written/speech to a procedural form orchestrated by game design. This transduction issue, as indicated by Excerpt 1 above, exposes the limitation of the software: by invoking elements that are not often mechanically explored in videogames (such as caring for someone else), Marta and Carla highlight the narrowness of a game grammar influenced by hegemonic gaming discourses. By eschewing my suggestions – also influenced by conventional game grammars, relying on how games such as Stardew Valley mechanically represented caring attitudes (de Paula, 2021) – Marta and Carla were highlighting the limitations of the available semiotic resources, indicating therefore an intention to seek their own way of representing caring.
This design passage from Extrovertido indicates the limitations of MissionMaker, especially in facilitating meaning-making processes by game-makers who were unfamiliar with (a conventional adventure-based) game grammar. But how would sign-makers that are more knowledgeable about (the) game grammar (invoked by MissionMaker) fare in this situation? In the following section, I investigate how Yerry and Juan explored these grammars before reflecting about Marta and Carla’s final design decision.
Invisible Barriers and Gaming Grammars
Yerry and Juan relied on sci-fi elements to organize Experiment Z: their game revolved around a brilliant scientist, an unscrupulous corporation and a failed experiment that almost wiped out humanity. These, however, were not the only influential aspects stemming from their game: differently from Marta and Carla, Yerry and Juan claimed to be experienced videogame players.
Juan and Yerry, therefore, were more familiar with a conventional game grammar than Marta and Carla. This allowed them to work more confidently with the available semiotic resources and modes, borrowing elements from different domains such as sci-fi or 3D adventure games – the kind of game grammar favoured by MissionMaker. While there was a noticeable overlap in some of the game design grammatical constructions in Extrovertido and Experiment Z, such as tracking characters and fighting, Experiment Z did not deal with any grammatical contravention such as the caring ending sequence proposed by Marta and Carla.
Moreover, comparing the initial design of both games, it becomes clear that Experiment Z relies less on a specific media text and more on generic gaming conventions, working within the dimensions of what a consolidated game grammar would look like. An example of this use of game mechanics as conventional semantic constructs is the way the environment is manipulated in their game, with areas being accessible or locked according to the player’s progress through ‘invisible barriers’ as happens in conventional action-adventure games.
These ‘invisible barriers’ were used in the game’s opening sequence, where a dialogue (audio recorded by game-makers) between the main character, Sherlock, and the villain, Elsare, happens, outlining the game’s plot. During this dialogue, the player realizes that the main objective of the game is to kill Elsare: to prevent the player from doing so in that moment – thereby spoiling the game – Yerry and Juan constructed an ‘invisible barrier’ using Trigger Volumes and rules (detailed through Figures 3 to 5).
When the player enters the first Trigger Volume (in the middle of the road), the initial dialogue plays.
A second Trigger Volume, placed in front of the door, between Sherlock and Elsare, is set by default as repulsive, preventing objects from crossing it. When the dialogue starts playing, a timer will start ticking (lasting 26 seconds, the exact duration of the dialogue), and after this timer this second Trigger will cease to become repulsive – being crossable again.
A third rule completes the sequence, closing the door when the previously repulsive Trigger Volume is entered by the player, preventing her from reaching Elsare.
Juan and Yerry’s idea for implementing invisible barriers stemmed from a specific moment, when I was introducing Trigger Volumes to the group. During that session, the following dialogue happened:
This ‘repulsive’ here, what does it mean?
Ok, remember what the Trigger does?
It detects when you enter . . .
But another thing that it can do is to stop you from entering, like those invisible barriers in the end of a scenario . . .
Like a force field?
Yes, [when set] as “repulsive”, a Trigger works like an invisible force field.
In my exposition, I had not yet addressed the ‘repulsive’ property of Trigger Volumes. 9 The invisible barrier programmed here works to establish ties between the narrative (communicated through audio and visual modes) and the ludic elements of the game, preventing the player from achieving the final objective in the opening sequence of Experiment Z, but also ties in with the broader corpus of action-adventure games. Through these invisible barriers, Yerry and Juan were also able to reinforce the chosen game mechanics that relied on pickups to solve small puzzles. By exerting a stronger control on which areas of the game environment were accessible (and which were not), game-makers reinforced the purpose of the environment exploration game mechanic since, without exploring all spaces and existing objects, players would not be able to progress in Experiment Z.
This design choice also allowed them to articulate different types of modality claims: modalities can be either representational or presentational (Van Leeuwen, 1999). Extrovertido, where Marta and Carla’s digital Baek Ma Ri was judged as a representation of the TV show character, with her Asian-ness, is a case of representational modality.
Experiment Z’s modality claims, however, are mostly presentational since they are better understood as ‘true to the spirit of the genre, and the values which underpins it in its context’ (Van Leeuwen, 1999: 180). The use of different elements that are easily recognizable as part of gaming – such as the ‘invisible barriers’ example mentioned above – support these presentational modality claims.
Extrovertido also articulates presentational modality truth claims, but these are built sometimes on gaming – e.g. seeking and fighting – and sometimes on their narrative/fantasy modality claim – the romance between protagonists. This mixture, although creative, culminates in a situation where the presentational modality claims (gaming and teen romance) can work against each other, since some discourses (e.g. romance) are not easily transduced from one domain (e.g. youth fantasy) to another (e.g. digital game), as noticed in the final example presented in the previous section (Excerpt 1).
This is a different process from that followed by Yerry and Juan, who constructed a presentational modality that articulates scientific fiction (through the narrative of Experiment Z) and traditional gaming (through gaming conventions, such as the invisible barrier). This difference between Extrovertido and Experiment Z raises questions about the possibilities of meaning-making in videogames: are we fated to subject ourselves to existent ‘game grammars’ to have our messages becoming intelligible? Can we shift conventions to be more inclusive through game design? Can we actualize gaming discourses to encompass different positionings and voices?
Aliens and Grammatical Contraventions
By investigating how young people produce videogames in a non-mainstream context, I intended to showcase how discourses and material constraints become significant elements in meaning-making, and how different strategies to engage with these conditions could be sought to produce videogames that are intelligible and true to designers’ intentions.
As discussed throughout this article, the level of alignment to gaming discourses – more specifically, to mainstream gaming, with 3D graphics and complex controllers (Consalvo and Paul, 2019) – was influential in game-makers’ production strategies. These discourses were influential not only in participants’ ideas, but also in aspects that were beyond participants’ control: since these discourses were ingrained in MissionMaker, they shaped the semiotic resources available to game-makers. This helps us to understand why a ‘more conventional’ structure such as the use of invisible barriers in Experiment Z was easily implemented, whereas a ‘contravention’ such as the caring game sequence in Extrovertido was not.
This difference culminates in what seems an obvious argument: a better understanding of the ‘grammar’ in use – in this case, mainstream game design conventions – affords a vaster repertoire for game designers to rely on. Experiment Z is an example of this process, since Juan and Yerry’s knowledge about (mainstream) game grammar allowed them to work more confidently in their production, as they were more familiar with the semiotic resources available through MissionMaker. It was their previous knowledge and interest that led them to ask about and experiment with a specific function of the Trigger Volume object – one that had not yet been described at the moment – and, by identifying this possibility, they were able to construct a specific semantic structure in their game design, creating an opportunity to expand the core ‘environment exploration’ game mechanic and exerting a higher level of control on how players experienced their game.
This is not to say, however, that a game produced by someone who ‘knows’ the grammar will necessarily be a better game. Knowing the conventions might lead to a text that is ‘truer to its genre’, but it can also become a bounding aspect, reducing the creative possibilities outside that genre envisioned by sign-makers. To exemplify this situation, I retake Extrovertido’s actual final ending sequence, as it hints at how ‘contraventions’ can be an important path to broaden this grammar.
In their initial plan, Marta and Carla opted for a ‘caring’ game sequence, something they were not able to produce through MissionMaker. After struggling about how to end their game, Carla proposed a completely new version, detailed in a conversation presented as Excerpt 3 below:
We are not putting objects . . . we changed the game!
Oh, that’s ok, not a problem! So, what happens now when you rescue him?
When you rescue him, two men in suits and sunglasses show up and tell you to not come close to him, because he is very important . . . [both laugh]
Wow, so who is he?!
You haven’t seen everything yet . . .
In that moment, this Alien [points to the Alien icon in the object mode] shows up, and he says that in fact Jung Jae Min is a spy for the Aliens, and they both disappear. The End.
. . .!!!
This new ending (see Figure 6), while true to the fantastic modality coding orientation envisioned earlier, subverts expectations by breaking the fairy tale-like ending that could be expected considering the romantic starting point from where Extrovertido begins.
Baek Ma Ri encounters Jung Jae Min, in his ‘real’ (alien) form.
More interestingly, as becomes clear in Excerpt 3 above, the creative solution was, to some extent, prompted by the semiotic resources available in the platform that had constrained their design before. In a similar way to what happened with Juan and Yerry’s invisible barriers, the platform showed Carla and Marta a possible solution to the design conundrum they found themselves in.
This creative intervention indicates the dynamism involved in the work with grammars: the most productive path here might be to focus on how to identify and support these ‘transgressional’ uses of semiotic resources, towards a more democratic gaming culture (Harvey, 2014), one that accommodates diverse discourses and ways of being.
Final Remarks
The examples discussed here also remark on the importance of reflecting about how we support and explore the use of procedural elements for meaning-making through non-mainstream game development. Even if, at different degrees, participants were able to engage with procedural elements (e.g. rules, code), most of their sign-making relied on narrative/representational elements, which indicates a prevalence of these dimensions in game-making. Marta and Carla’s subversive ending is a clear example of this limitation: while they were capable of producing a (traditional game) grammar contravention, this is mostly dependent on a narrative/representational device, relying on the most salient modes. This overreliance on the representational sphere indicates a necessity of exploring the implications of a procedural mode (Hawreliak, 2018) not only for sign-interpreters, but also for sign-makers, especially in how we support designs that contravene and subvert regular game grammar structures.
It also indicates the importance of expanding Hawreliak’s model to work not only with rhetoric, but also with poetics (Burn, 2021): in the same way as Hawreliak (2018) remarks on the relevance of ‘modal dissonances’ for ideological persuasion, Extrovertido presents a relevant case on how ‘modal dissonances’ can also fulfil aesthetic functions, articulating discourses (e.g. about gender and fantasy) and potentially reshaping the ‘regular uses’ of semiotic resources – e.g. the alien as a former romantic interest.
Moreover, in the same way as public discourses and debates are constitutive of what are perceived as ‘proper’ games (Consalvo and Paul, 2019), game-making experiences such as these can play a similar role. In this sense, it becomes important to reflect on how well-intentioned initiatives to include more people in game development can backfire, reifying the same exclusionary structures they aim to disrupt if they are not careful enough to support experimentations and transgressions (de Paula, 2021). In this particular case, Marta and Carla were able to work through the challenges to accommodate their ideas and available semiotic resources into their game, but this transductional issue could have culminated in a more negative outcome, with game-makers becoming frustrated about their options and abandoning their game.
Further research on the relationship between grassroots game-making initiatives and their formative role, including how they relate to what is publicly understood as ‘real’ games (Consalvo and Paul, 2019), seem important lines of inquiry to be carried further. Understanding these communicational and aesthetics functions in all levels of game-making (from small creative exercises such as the ones discussed here to professional ones), therefore, might be the next challenge for comprehending meaning-making in videogames, not only on how different grammars can rise and become recognized as significant, but, more importantly, on how we could promote a more democratic and open attitude towards different game grammars and, ultimately, towards these cultural artefacts.
