Sage Journals: Discover world-class research

Abstract

This article investigates the phenomenon of open and participative development (e.g. beta testing, Kickstarter projects)—i.e. extended prototyping—in digital entertainment as a potential source of insights for instructional interventions. Despite the increasing popularity of this practice and the potential implications for educators and instructional designers, little efforts have been done in enlightening the topic. This study aims to address this lack by staging a bridge between Instructional Design and Game Design with an empirical inquiry. N:130 subjects (beta testers and Steam Early Access and Kickstarter users) were recruited with a quantitative questionnaire about their contribution to open development instances. Behavioral patterns, effective policies and managements, and subjective profiles and opinions were gathered and tied to instructional design models and concepts. Results point to successful techniques in designing and applying such a process, while mistakes and unproductive tactics are highlighted as well. To summarize, instruction can take advantage of an increased participation of targeted audiences/learners to its development phases. Benefits span transparency, engagement, and commitment. However, poor communication and incoherence between testing and final product may weaken the overall outcome.

Keywords

Beta testing extended prototyping game design instructional design open development

Introduction

Instructional design (in the following, ID) can be described as the process through which an instructor plans an educational intervention to lead his/her students toward specific learning goals. Game design (in the following, GD) might be depicted as the technique for building an engaging ludic experience. These two approaches thrived in the last decades informing teaching strategies and involving activities, respectively. It can be argued that both have achieved a relevant maturity and a well-established relation with media and technology. Moreover, despite their paths seem to diverge, they show more similarities than just the term design. Indeed, several concepts and models are shared in between the two disciplines, and insights can be moved from one to another (e.g. rapid prototyping, user-centered design). This interplay may mean that digital entertainment trends can provide stimuli to instructional designers and vice versa. Merrill and Willson (2007: 435) observe that “more could be done to apply gaming and simulation principles to instructional design,” and we are just starting to deepen how gamification trends—i.e. applying ludic mechanics to nonescapist settings—and game characteristics can fit into ID (Landers, 2014). Moreover, instructional designers usually make several mistakes in harnessing game dynamics, from avoiding playtesting to missing the link between ludic rules and learning goals (Boller and Kapp, 2017).

The objective of this article is to contribute to such a potential synergy by adopting an alternative perspective, i.e. looking at the game industry as a possible depository of stimuli. Specifically, the focus will be on deepening a widespread practice in game development, i.e. participative phases in which players are actively involved (e.g. on online platforms like Steam Early Access or Kickstarter) and reflecting on its potential implications for ID. To reach this goal, n:130 subjects were recruited with a quantitative survey about perceptions, best practices, and issues in experiencing such a bottom-up phenomenon, which can be interpreted as an “extended prototyping” (or “open development”) aimed to improve a final product or intervention. Theoretical references follow a multidisciplinary approach spanning GD, game studies, ID, and media studies.

The article is structured as follows: in the first section, bridges between ID and GD are uncovered and bottom-up development is addressed as a potential crossroad. The second depicts the research design and the third illustrates the results. Finally, the fourth regards discussion and conclusion by highlighting implications, limitations, and possible steps forward. To summarize, an extended prototyping can play a significant role in education harnessing its debugging and open orientation. However, learners should be highly motivated and constantly followed by content providers in a bidirectional exchange for maximizing the outcome.

Seeking a crossroad between instruction and gaming

ID and GD

ID is a “systemic process, usually conducted by a team of professionals (…) [that] involves analysis of a performance problem, and the design development, implementation, and evaluation of an instructional solution to the problem or need” (Gustafson and Brunch, 2007: 14). ID (and technology) addresses the evaluation of performance problems and learners, and then design, develop, and test an array of instructional and noninstructional interventions aimed to solve performance issues. The leading premise is that instructors are active designers rather than just content deliverers; therefore, they are asked to plan, shape, and craft curricula iteratively and trying to satisfy the students’ needs (Brown and Green, 2011; Reiser, 2007: 7; Wiggins and McTighe, 2005: 13). To summarize, instructional designers aim to create, handle, and finalize a learning process through which specific goals are set and (hopefully) reached. Usually, a list of objectives is staged and then assessment modalities are designed and proper educational strategies follow (e.g. see the Understanding by Design (UbD) model by Wiggins and McTighe (2005)).

Several techniques have been developed to guide instructional interventions, from the famous ADDIE (Molenda, 2003) to the systems approach (Dick and Carey, 2014) and Kemp (Morrison et al., 2010) models. In addition, ways to visualize thinking and differentiate between multiple types of content and practices were advanced (e.g. Ritchhart et al., 2011; Wiggins and McTighe, 2005). ID embraces different models and approaches with a broad range of extension and specificity, with an increasing emphasis on flexibility and adaptability (Gibbons et al., 2014). The increasing influence of the constructivism theory has led to open approaches such as the so-called research design, which harnesses educational proto-theories (emergent, developmental theories) in its own development (Jonassen et al., 2007: 48), and grounded design, i.e. an open-minded approach, which combine multiple perspectives, frameworks, and methodologies (Hannafin and Hill, 2007: 56). This approach is especially glaring if we observe how the field embraces suggestions from other disciplines. For instance, rapid prototyping—i.e. a production approach based on increasingly developed prototypes from the initial idea to the final product (Brown and Green, 2011)—is a widely adopted technique from software development. It implies a progressive process that overturns the classical sequence of design/production for saving time, preventing high-cost revision, and better matching clients’ expectations and needs.

The GD can be described as a creative process aimed to involve potential players with a ludic system. Swain (2008: 119) considers “Game design (…) the art of crafting player experiences,” and this view can be found in other definitions (e.g. Bateman and Boon, 2006: 4; Fullerton, 2008: 115; Sylvester, 2013: 4). Several game designers have advocated for the centrality of iteration—i.e. developing and trying increasingly advanced versions of a game until the designer’s criteria are matched (Fullerton, 2008: 14)—within this creative process. This ongoing formative evaluation has been highlighted as an essential strategy to refine a product (Schell, 2008; Swain, 2008), and prototypes play a key role in such an approach (Brathwaite and Schreiber, 2008) considering rapid prototyping an essential step in testing cycles. Flanagan (2009) claims the same importance in reaching a perfect match between core design elements and game mechanics (i.e. the game rules and processes) (see also Adams and Dormans, 2012; Sylvester, 2013)

Similarly to ID, GD is often open to other domains, from architecture (Adams and Dormans, 2012; Schell, 2008) to education. Especially the latter domain is increasingly associated with video games and ludic mechanics, which can provide significant opportunities for fostering learning, meta-cognition, and critical thinking (Gee, 2007; Merchant et al., 2014). Mimicking the most recent approaches in ID, GD for learning is embracing insights from constructionism by highlighting the shared dimension of play (Egenfeldt-Nielsen, 2007). This medium has been proved to improve collaboration, prosocial attitudes, and metacognition indeed (e.g. Badatala et al., 2016; Dale and Shawn Green, 2017).

Such a potential goes beyond the so-called edutainment titles—games mainly designed for instructional purposes (Egenfeldt-Nielsen, 2006)—and addresses leading genres and practices in game industry and consumption (Gandolfi, 2017; Lombardi, 2012). Mainstream trends inform the current expectations about the medium indeed, and studying them is crucial for understanding ludic engagement and involvement, which are key dimensions to consider if we intend to look at video games with educational plans (Egenfeldt-Nielsen, 2007). The increasing importance of the shared dimension of digital play (Consalvo, 2007) demands a better attention to game communities and environments, in which game (learning) outcomes are framed, shaped, and reformulated (Egenfeldt-Nielsen, 2007; Gandolfi, 2017). This lens implies that players are active and social learners. Therefore, building a bridge between ID and GD means to reflect on the leading game dynamics and processes in digital entertainment itself even when our objectives are instructional at their core. Such an approach expands the scope with which educators approach video games, going beyond their formal features (e.g. rules, dynamics, mechanics) and targeting topics like (game) production, consumption, and community.

Combining perspectives

Before deepening such a bond, it is worthy to mention two other similarities between GD and ID. First, the idea that knowledge can be split in two categories—core and specific. According to Bateman and Boon (2006: 110), the former informs the so-called Tight Design, which “use the minimum quantity of elements required to support the desired gameplay”; neither distractions nor ephemeral details are included in such a category (Sylvester, 2013: 167). The latter is addressed by the Elastic Design, which relies on “having more resources than you need with the express purpose of whittling the set of components down to the best minimal set that the production constraints allow” (Bateman and Boon, 2006: 114). The combination between these orientations is crucial to stage a well-balanced experience (Bateman and Boon, 2006: 116; see also Fullerton, 2008: 198). However, the risks span excessive costs, frustrating experiences, and bad balanced gameplays (Sylvester, 2013: 332; see also Adams and Dormans, 2012). In ID, we can see this difference in the relation between big ideas and specific information. The former ones refer to the core of a subject area and they need to be uncovered, while the latter regards secondary details (Wiggins and McTighe, 2005: 65–70). For instance, in learning about the American Civil War slavery is a big idea because it concerns the main causes of such a conflict. Otherwise, knowing the second battle won by the North army tell us less about the big picture. This distinction can be noticed also in the difference between supportive information and specific learning task in the 4c-ID model (Van Merrienboer, 2007: 78) and in the distinction between whole learning problems and single examples (Merrill, 2007: 65)

Second, the user-centered approach is getting popular in both the fields. Indeed, GD stresses the importance of the player: “the sooner you can bring the player into the equation, the better, and the first way to do this is to set ‘player experience goals’” (Fullerton, 2008: 10). Brathwaite and Schreiber (2008: 2) observe that ideal GD means informing and shaping experiences that players perceive as relevant. In ID, the idea that learning is multiple and variegated rather than sequential and hierarchical is flourishing (Ritchhart et al., 2011: 6–7); Gustafson and Brunch (2007: 14) consider ID “an empirical process that is learner centered and goals oriented.” The research and grounded designs described above highlight the openness of this approach. In other words, the need of listening to final users is becoming crucial. Sugar and Luterbach (2016) interviewed several instructional designers, who reported positive experiences when social connections are staged with students and collaboration with subject matter experts, instructors, and clients is included in the design process (see also Reiser, 2017). Digital entertainment is embracing such a trend by developing several models of participative design production, from open beta testing to crowdsourcing. Such processes can be referred as an “extended prototyping.” Several approaches and procedures can be deployed, with a relevant use of social media to communicate between producers and testers. More specifically, we can categorize this phenomenon in two main orientations:

Crowdsourcing: interested users (future players, in this case) economically support the development since the preliminary phases (e.g. on online portals like Kickstarter and Indiegogo) or joining an in-progress version of the game (on Steam Early Access). Their role is significant both in terms of funding and feedback.

Testing: developers ask the game community to test a preliminary version of the software to refine it. Testing can be alpha (preliminary) or beta (almost finished), closed or public (the former relies on selected testers and the latter is open to everyone).

Despite the increasing popularity of these initiatives, little efforts have been done in understanding how testers perceive their own efforts. Banks (2013) made a point in suggesting that participating players are a complex audience, whose traits go beyond the mere distinction between exploited labor and active involvement, and Consalvo and Begy (2015) proved that the relation between developers and supporting community is variegated and multidimensional. Going at the core of the audience studies, it has been found that media consumers are moved by different drivers, from interpretative efforts to producing needs (Couldry, 2005; Gandolfi, 2016). This article aims to shed light on this phenomenon, which is still poorly analyzed, with a specific attention to how ID can incorporate its features and processes. Extended prototyping is indeed a thriving technique and affects how playing and game testing are widely perceived; it matches the call for understanding mainstream trends in digital entertainment in order to develop a better educational exploitation of the sector (from production to consumption) (Egenfeldt-Nielsen, 2007). It may be interpreted as a novel way to handle formative evaluation—assessment of the instructional materials and strategies during their development (Smith and Ragan, 1999)—in online contexts and learning.

Research design

According to the previous premises, two research questions lead the present article:

RQ1: how do testers perceive open development and which motivations lead them in being involved? From an ID perceptive, what learners can this technique attract and engage?

RQ2: what are best practices and weaknesses in staging open development sessions? From an ID perspective, what are the parameters to consider for staging an alike formative evaluation?

The resulting reflection aims to comprehend how these findings can be used to inform ID techniques, with constant references to potential applications. The objective is indeed to win “the challenge for (…) [ID] to learn from other disciplines as these new tools are developed, adopted, and systematically integrated into our practice” (Dieker et al., 2014: 31). Prototyping in ID works indeed as a formative step (Brown and Green, 2011: 245), and assessing its strong and weak points might help in better realizing if this phenomenon can (or cannot) work for instructional designers.

The present analysis will focus on four types of extended prototyping in gaming, selected for their popularity and reciprocal differences. The first two are closed and open testing, which are usually made accessible for download by the developer for a limited amount of time. In last years, million players have participated in these sessions (e.g. 6 million for The Division, 6.5 million for For Honor; see https://www.ubisoft.com/en-US/company/investor_center/annual_report.aspx). The former type, which relies on selected testers, tends to let participants assess relevant gaming features; the latter is usually consequential, open (and then broader) and focused on more marginal traits. The third addresses Steam Early Access (in the following, Steam EA); released in 2013, EA is a specific section of Steam, i.e. an online platform owned by Valve Inc with multiple functions, from online digital purchase to forums and UGC galleries, which count million users (Makuch, 2015). Publishers can sell unfinished versions of their games and ask funders to help them in finalizing the development process. At the current date (3 June 2017), almost 1700 games are enrolled in this system in several stages, from rough alpha versions to almost finished products. The fourth targets Kickstarter, which is the most famous crowdfunding platform with over $3.1 billion in pledges from 13 million backers (i.e. funders) supporting 257,000 creative initiatives. Digital games represent a leading component of such a landscape with 31,628 launched projects and $650.33 million raised (Kickstarter, 2017). Usually backers fund a project and then, if successful, they are updated about the development and sometimes asked to give opinions and even participate to testing phases.

Methods

A quantitative questionnaire (see Appendix 1) was designed to gather testers’, EA users’, and backers’ opinions and statements about their involvement. Aside from sociodemographic information (i.e. gender, age, nationality, education, occupation), the survey presented two parts.

The first included four sections about gaming habits and preferences (weekly hours, favorite genre, favorite gaming platform, motivation to play), and then one for gaming expertise (about favorite games, favorite genre, and digital entertainment per se) and affiliation as a gamer. The latter is a scale that was proved to be reliable (Gandolfi, 2016) about gaming identity. It is composed of four items addressing internal attribution, external attribution, priority of the attribution, and sharing of the attribution regarding the word “gamer.” Therefore, participants were asked to fill sections related to their concrete experience with closed testing, open testing, Steam EA, and Kickstarter. For each category, number of involvements and most targeted genres and platforms were reported. In addition, motivations to contribute, behavior, difference with nonparticipative players, and community feelings perceived during the support were addressed. The intent was also to understand if we can refer to a “community of practice” (Wenger, 1998), i.e. proactive learning environments based on a shared domain, a strong community, and reiterated practices (Buysse et al., 2003). If subjects reported good-high scores in stating that testers are not alike common gaming audiences, they were asked to write down the reasons. This first section aimed to understand testers’ characteristics and attitudes toward the domain (in ID, the content taught) targeted by their support. The objective was to address RQ1 by framing them as potential learners during a formative evaluation.

The second part targeted RQ2. Orientation, relation, and experience in testing were assessed (each with seven items) to uncover the bottom-up support itself. The first parameter concerned the perceived utility of such an effort spanning refinement, fixing, and innovation; the second regarded the relation with developers, which is indeed highly debated; the third focused on the experiential orientation of their activity. The rationale was to understand the main traits of extended prototyping, enlightening how subjects perceived their own efforts and deal with the producers. By adopting an ID perspective, orientation can be tied to the relevance of the problem solved by an instructional intervention; relation may be the one with possible ID managers and staff, which “are responsible for researching, designing, and developing the instructional product” (Litchfield, 2007: 117); experience is about the affection (Reeve, 2001) of the learning process, which can vary from engaging to stressful. Finally, best and worst testing experiences and titles were also asked with a free text entry for acquiring further and more spontaneous information.

Regarding items providing personal positions, 1–4 Likert Scales were adopted to prevent the tendency by subjects to check the median point (Marradi, 2007). For the rest (e.g. weekly consumption of video games) a more common 1–5 Likert Scale was preferred. Quantitative data were processed with IBM SPSS Statistics, while textual outputs (from open questions) have been analyzed with NVivo v.10. Regarding the latter, a “discourse analysis” (Gee, 2012) was staged to gather main criteria of difference from other players and strong and weak points of the targeted experiences. The related coding followed a two-cycle process (Saldana, 2016: 234). First, several tags were applied to map statements in detail (e.g. wrong policies by developers, controversial design choices, single experiences) and then a thematic summary was directed (e.g. coherence, community) to regroup results for a clearer overview. In conclusion, this inquiry has been designed for supporting an exploratory analysis—an investigation with “no initial hypothesis, even the initial selection of relevant variables becomes a critical step. (…) [that] aims at summarizing the main characteristics of a given dataset by inspecting it” (Canossa, 2013: 259). Due to the scarcity of studies on the topic, the main objective was indeed to outline a first overview of such a phenomenon for facilitating further researches and more structured arguments. The survey was spread online from October 2016 to February 2017 with the prerequisites of age (18 years old or older) and suitability (being involved with at least one of the four types of open development). Social media (e.g. Facebook, Reddit, Twitter) worked as main dissemination channels with an additional use of the academic mailing-list gamesnetwork.com. Moreover, the creative director of a game frequently mentioned by participants was contacted for an e-mail interview (made in January 2017) about his experience with Steam EA. The research was approved and monitored by the author’s Institution I.R.B. committee.

Results

N: 130 participants were recruited (age—m: 29,746; St. dev.: 8855/n:100 male; n:27 female; n:3 other). In Table 1 subgroups’ sizes are reported.

Table 1.

Sample composition.

	N: subjects recruited
Practice
Total closed-beta testers	74
Total open-beta testers	59
Total Steam EA users	50
Total Backers	32
Combinations
Closed+open+stem+kick	11
Closed+open	2
Closed+open+stem	20
Closed+open+kick	5
Open+stem+kick	2
Steam+kick	4
Open+stem	5
Open+kick	2
Closed+stem	2
Closed+kick	2

Subjects mainly come from U.S. (n:45–34.62%), United Kingdom (n:16–12.31%), Italy (n:16–12.31%), Canada (n:7–5.43%), and Germany (n:6–4.61%). Other geographic origins are Netherlands, Denmark, Switzerland, Sweden, India, Greece, Hungary, France, Malaysia, Indonesia, Norway, etc. The majority has a College degree (n:40–30.76%), followed by a Bachelor Degree (n:28–21.71%), Master Degree (n:23–17.69%), and High School degree (n:20–15.50%). Most diffused occupations include student (n:34–26.15%); computer and mathematical (n:23–17.69%); education, training, and library (n:13–10.00%); and unemployed (n:13–10.00%). Regarding their game habits, almost half (n:63–48.46%) of the sample reported more than 11 h weekly. N:33–25.38% spend 8–11 h with digital games, while n:24–18.46% 4–7 h and n:10–7.69% less. Favorite game genres widely span Western Role-Playing Games (RPGs) (n:45–34.62%), Turn-Based Strategy (TBS) (n:36–27.69%), and Simulations (n:31–23.85%). On the contrary, PC is the most favorite game platform (n:119–91.54%) with smartphones and PlayStation 4 stop at 21–16.15% preferences each. Regarding the orientation to play, fun (n:79–60.77%), challenge (n:47–36.15%), and immersion (n:44–33.85%) have a crucial influence; other important ones are dissociation from everyday routine (n:21–16.15%) and narrative (n:19–14.62%). Table 2 summarizes the activity outside the game per se. Among the gaming practices, just a minority is actively involved with the development of games, patches, or additional content, but there is a majoritarian tendency to discuss digital entertainment online (72.31% of average-high scores) as well as offline (66.16% of average-high scores), and purchase additional content (58.92% of average-high scores).

Table 2.

Practices.

	0		1		2		3		4
I create mods for extending existent video games	56.15%	73	23.08%	30	13.85%	18	6.15%	8	0.77%	1
I create patches for improving existent video games	79.84%	103	8.53%	11	8.53%	11	1.55%	2	1.55%	2
I watch Twitch.tv	32.31%	42	24.62%	32	26.15%	34	13.08%	17	3.85%	5
I watch YouTube Gaming	20.00%	26	23.08%	30	28.46%	37	23.08%	30	5.38%	7
I stream on Twitch.tv	79.07%	102	13.95%	18	5.43%	7	0.78%	1	0.78%	1
I stream on YouTube Gaming	91.47%	118	5.43%	7	2.33%	3	0.78%	1	0.00%	0
I develop my own video games	65.12%	84	15.50%	20	7.75%	10	6.20%	8	5.43%	7
I debate and discuss online about video games	8.46%	11	19.23%	25	31.54%	41	28.46%	37	12.31%	16
I create UGCs related to video games	46.92%	61	24.62%	32	19.23%	25	6.15%	8	3.08%	4
I debate and discuss offline about video games	13.85%	18	20.00%	26	33.08%	43	20.00%	26	13.08%	17
I buy game DLCs and Season Passes	10.08%	13	31.01%	40	26.36%	34	21.71%	28	10.85%	14

DLC: downloadable content; UGC: user generated content.

Taking a step forward, in Table 3 the first subgroups’ tendencies are reported. Kickstarter group leads the way in terms of projects funded despite the relevant standard deviation. Weekly consumption is tendentially high across the four categories. It is easy to notice a shared expertise in favorite games and genres with a glaring majority of good-high scores. Such a tendency is confirmed but weakened toward the sector. The tag gamer is intensively felt by each category; however, subjects consider its centrality average and sometimes marginal. PC is the reference testing platform, with Mac and Linux machines are reported as secondary highlights. The targeted genres are common between closed and open testing spanning MMORPG, FPS online, and Survivals. The latter comes first in Steam EA preferences, followed by Simulations and Creative games. Finally, Kickstarter is practically dominated by old-fashioned genres, from adventures to TBS and Western RPGs. It is interesting to notice how many of these genres are online and multiplayer, pointing to a shared and social testing experience. Closed and open testing are similar also regarding motivation; testing before purchase, being involved, genre, and playing for free are the reference drivers. Conversely, Steam EA attracts users with improvement opportunity, and then relying on genre, originality of the idea, and presence of friends. Kickstarter is affected by genre and originality as well, but also nostalgia and developers’ credentials play a key role.

Table 3.

Results (part 1).

			Closed	Open	Steam		Kickstarter
Number of projects			5.54 mean6.26 st	4.52 mean5.37 st	6.68 mean6.82 st		8.43 mean17.97 st
Weekly consumption			3.26 mean0.96 st	3.37 m0.88 st	3.36 m0.82 st		3.03 mean0.98 st
Expertise	Favorite game	1234	0.00% 05.41% 443.24% 3251.35% 38	0.00% 03.39% 242.37% 2554.24% 32	0.00% 04.00% 250.00% 2546.00% 23		0.00% 012.50% 434.38% 1153.13% 17
	Favorite genres	1234	0.00% 016.22% 1255.41% 4128.38% 21	0.00% 013.56% 859.32% 3527.12% 16	2.00% 120.00% 1054.00% 2724.00% 12		0.00% 025.00% 850.00% 1625.00% 8
	Digital entertainment	1234	4.05% 333.78% 2539.19% 2922.97% 15	1.69% 133.90% 2040.68% 2423.73% 14	8.00% 440.00% 2032.00% 1620.00% 10		3.13% 137.50% 1231.25% 1028.13% 9
Gamers	Self-assessment	1234	4.05% 310.81% 825.68% 1959.46% 44	5.08% 38.47% 528.81% 1757.63% 34	8.00% 412.00% 628.00% 1452.00% 26		12.50% 412.50% 425.00% 850.00% 16
	Other assessment	1234	4.05% 312.16% 932.43% 2451.35% 38	6.78% 410.17% 635.59% 2147.46% 28	14.00% 718.00% 928.00% 1440.00% 20		6.25% 218.75% 631.25% 1043.75% 14
	Centrality	1234	25.68% 1920.27% 1529.73% 2224.32% 18	28.81% 1716.95% 1032.20% 1922.03% 13	36.00% 1818.00% 934.00% 1712.00% 6		40.63% 1315.63% 521.88% 721.88% 7
	Friends	1234	5.41% 428.38% 2139.19% 2927.03% 20	6.78% 427.12% 1632.20% 1933.90% 20	10.00% 532.00% 1628.00% 1430.00% 15		21.88% 715.63% 528.13% 934.38% 11
	Closed		Open	Steam		Kick
Platf	PC 91.89% 68Mac 10.81% 8Smartph 5.41% 4		PC 93.22% 55Mac 8.47% 5PS4 6.78% 4	PC 96.00% 48Mac 4.00% 2		PC 90.63% 29Mac 18.75% 6Linux 18.75% 6
Genre	MMORPG 39.19% 29FPSOn. 37.84% 28Surv. 27.03% 20Act.-Adv. 22.97% 17RTS 21.62% 16		MMORPG 42.37% 25FPSOn. 30.51% 18Surv. 28.81% 17Act.-Adv. 22.03% 13Sim. 22.03% 13	Surv. 66.00% 33Sim. 38.00% 19Crea. 34.00% 17TBS 28.00% 14RTS 26.00% 13		Adv. 43.75% 14TBS 34.38% 11WestRPG 34.38% 11Act.-Adv 25.00% 8Survival 18.75% 6
Motivation	Testing before purch. 56.76% 42Being involved 52.70% 39Genre 41.89% 31Playing for free. 28.38% 21		Testing before purch. 52.54% 31Being involved. 35.59% 21Genre. 33.90% 20Playing for free. 28.81% 17	Improvement. 56.00% 28Genre. 42.00% 21Originality 42.00% 21Friends. 18.00% 9		Genre 56.25% 18Originality. 53.13% 17Nostalgia 34.38% 11Dev credentials. 28.13% 9

About the activities related to the extended prototyping (see Table 4), discussing the game in official online spaces and with peers is common, with the prevalence of good-high scores (i.e. 3 or 4) (the only exception is Kickstarter about online debates, with 56.25% of low-average scores). However, streaming was marginal despite it is getting popular with closed and open testing (see Overwatch and For Honor prelaunches, in which streaming by testers was highly supported by developers). Community feelings are widespread, while the hardcore orientation is relative (relevant in closed testing and Kickstarter) and the higher expertise is perceived only by closed testers (52.71% of good-high scores). Employment ambitions are low in each category, and additional content is considered compatible to the extended prototyping following the aforementioned subjects’ tendency to purchase it. Steam EA and especially Kickstarter users think that these platforms are a source of innovation in the game industry. Finally, participants from each category tend to see themselves similar to the average player.

Table 4.

Results (part 2).

			Closed	Open	Steam	Kick
Activity	Discuss the game in official forums/social media accounts.	12345	21.62% 1621.62% 1627.03% 2021.62% 168.11% 6	15.25% 920.34% 1232.20% 1928.81% 173.39% 2	24.00% 1212.00% 626.00% 1332.00% 166.00% 3	18.75% 637.50% 1215.63% 525.00% 83.13% 1
	Discuss the game in unofficial forums/social media accounts.	12345	36.49% 2731.08% 2318.92% 149.46% 74.05% 3	28.81% 1722.03% 1325.42% 1518.64% 115.08% 3	30.00% 1536.00% 188.00% 416.00% 810.00% 5	18.75% 646.88% 1512.50% 418.75% 63.13% 1
	Stream or upload videos of the game.	12345	79.73% 599.46% 78.11% 61.35% 11.35% 1	69.49% 4113.56% 811.86% 75.08% 30.00% 0	74.00% 378.00% 412.00% 66.00% 30.00% 0	75.00% 249.38% 39.38% 33.13% 13.13% 1
	Discuss the game with peers and friends.	12345	14.86% 1112.16% 927.03% 2028.38% 2117.57% 13	11.86% 711.86% 725.42% 1532.20% 1918.64% 11	20.00% 1014.00% 720.00% 1030.00% 1516.00% 8	15.63% 531.25% 1025.00% 821.88% 76.25% 2
Perception	X of a game represents a community.	1234	12.16% 929.73% 2244.59% 3313.51% 10	20.34% 1218.64% 1140.68% 2420.34% 12	18.00% 912.00% 642.00% 2128.00% 14	28.13% 918.75% 634.38% 1118.75% 6
	X are usually hardcore gamers	1234	0.00% 06.76% 536.49% 2756.76% 42	32.20% 1937.29% 2223.73% 146.78% 4	32.00% 1630.00% 1524.00% 1214.00% 7	25.00% 821.88% 746.88% 156.25% 2
	X are more expert than the average gamer.	1234	13.51% 1033.78% 2536.49% 2716.22% 12	44.07% 2632.20% 1916.95% 106.78% 4	40.00% 2032.00% 1618.00% 910.00% 5	34.38% 1131.25% 1025.00% 89.38% 3
	I participate to X with employment aspirations.	1234	64.86% 4820.27% 159.46% 75.41% 4	93.22% 550.00% 05.08% 31.69% 1	86.00% 46.00% 36.00% 32.00% 1	90.63% 296.25% 20.00% 03.13% 1
	DLC, season passes (…) are incompatible with X.	1234	31.08% 2322.97% 1716.22% 1229.73% 22	40.68% 2416.95% 1016.95% 1025.42% 15	44.00% 2214.00% 714.00% 728.00% 14	50.00% 1615.63% 518.75% 615.63% 5
	Game projects on EA/Kickstarter address lacks and deficiencies in current digital entertainment	1234	n/a	n/a	24.00% 1224.00% 1244.00% 228.00% 4	12.50% 49.38% 346.88% 1531.25% 10
	X are different than the average player.	1234	25.68% 1931.08% 2331.08% 2312.16% 9	50.85% 3030.51% 1813.56% 85.08% 3	36.00% 1836.00% 1822.00% 116.00% 3	28.13% 931.25% 1025.00% 815.63% 5

DLC: downloadable content.

About what differentiates a tester from an average player, Table 5 includes the main criteria of difference reported by the subjects with main themes (with n: of references) and indicative quotes from the survey. Closed beta testers would be characterized by an analytic approach (i.e. assessing the system, finding bugs and issues, giving feedback), and then competence (related to the overall experience), relation with the developer’s commitment (i.e. passion and attitude toward the project tested), and vision of the whole productive process. Open beta testers share commitment and competence, but not the rest. Especially concerning closed testing, the leading trait is the ability to debug the ludic system and improve it issue by issue. Such a process relies on patience but also excitement in observing the product getting “better in the future.” Such an opportunity decreases with open beta testing because “most [of them] are a chance to play the game early and input is rare” and “tend to be more stress-tests of the game servers or similar to make sure the game is ready.” Some subjects observe that sometimes they do not participate to open beta “because it is more difficult to communicate with developers, (…) the game is in an almost finished state (…) - while during CB there are usually more margins.” Steam EA adopters seem to have a peculiar approach (similar to the debugging one mentioned above), relevant commitment, and also the willing to support alternative products. Backers are fans who want to assist novel games too and play a role in the production per se, but their effort is especially economic.

Table 5.

Testers’ peculiarities.

Closed		Open
Topic(n)	Example	Topic(n)	Example
Approach(23)	– A tester is trying as many different things as possible and approaching from a design perspective	Commitment(9)	– They are more eager to try new content earlier, thus they express a higher degree of affection toward the game/games in general.
Competence(13)	– Because CB testers are generally more experienced with video games and how they work	Competence(4)	– Greater understanding of mechanics and programming usually.
Relation(12)	– Closed testers are different because they often have had (…) more direct contact with the developers for better knowing mechanics, internals, etc.
Commitment(4)	– They are more passionate, they what to be involved in the creation of the game and they do not accept to be only spectators.
Vision(2)	– They have a bigger perspective and knowledge about the milestones that the game had passed, what were initial ideas behind certain mechanics, and how those ideas have changed, mutated into other gameplay or story features, or were completely removed from the final product.
Steam		Kickstarter
Approach (11)	– A higher willingness to give feedback and tolerate bugs.	Fandom(8)	– obviously, backers are fans (…) and the average player is not.
Commitment (7)	– EA supporters are a form of player all to their own who enjoy seeing it before it is released (…) I believe part of their joy comes from opening a treasure chest to see what is inside.	Alternative (7)	– Gamers that play lots of games get disillusioned with mainstream content just like all kinds of art critics. So Kickstarter allows the kinds of games that major publishers ignore to get made.
Alternative (3)	– They yearn for something better than the triple A regurgitated crap like COD and faulty releases	Role(2)	– Because they take an active role in making a game possible even at the risk of losing the money they threw into the project.

The assessment per se of the extended prototyping is presented in Table 6. Closed testing’s orientation is broad, from fixing and refining to suggesting innovative ideas and augmenting bottom-up participation. Therefore, it is a service for both developers and future players. Open testing is similarly perceived but with a poorer autonomy by testers in influencing the outcome. Steam EA and Kickstarter scope is particularly wide too, but the latter presents lower scores in fixing and refining goals. Regardless, all four subgroups consider their experience affected by promotional drivers. The relation with developers is usually positive, with reciprocal understanding and respect (the only exception is open testers, with 52.54% of low-average scores). Most Steam EA users felt to be exploited by developers (54% of good-high scores), but also that they bare their work (58% of good-high scores); for the other categories, this situation is overturned. Only most bakers (59.33% of low-average scores) think that developers do not need them, but the feeling to be part of the team is marginal in each subgroup. Regarding the experience per se, extended prototyping is poorly perceived as a labor and a stressing experience. On the contrary, it is mainly considered funny, a hobby, and an opportunity by each subgroup. Only closed testers and backers mainly felt the game more theirs in being involved, while the challenging dimension was highlighted by EA users and closed beta testers.

Table 6.

Perception of the procedure.

			Closed	Open	Steam	Kick
X support is important to fix a game.	Orientation	1234	1.35% 16.76% 527.03% 2064.86% 48	5.08% 318.64% 1144.07% 2632.20% 19	0.00% 018.00% 938.00% 1944.00% 22	28.13% 921.88% 740.63% 139.38% 3
X support is important to refine a game.		1234	0.00% 06.76% 536.49% 2756.76% 42	5.08% 323.73% 1432.20% 1938.98% 23	0.00% 014.00% 736.00% 1850.00% 25	28.13% 921.88% 740.63% 139.38% 3
X support is important for suggesting new ideas and solutions for a game.		1234	2.70% 218.92% 1440.54% 3037.84% 28	11.86% 715.25% 938.98% 2333.90% 20	4.00% 26.00% 348.00% 2442.00% 21	21.88% 728.13% 937.50% 1212.50% 4
X helps me in driving game development for getting the final product that I want.		1234	9.46% 728.38% 2141.89% 3120.27% 15	23.73% 1432.20% 1930.51% 1813.56% 8	14.00% 728.00% 1444.00% 2214.00% 7	15.63% 528.13% 937.50% 1218.75% 6
X is a service for developers.		1234	0.00% 09.46% 763.51% 4727.03% 20	10.17% 623.73% 1442.37% 2523.73% 14	6.00% 310.00% 550.00% 2534.00% 17	3.13% 131.25% 1043.75% 1421.88% 7
X is a service for future players.		1234	4.05% 310.81% 843.24% 3241.89% 31	11.86% 713.56% 842.37% 2532.20% 19	10.00% 516.00% 836.00% 1838.00% 19	9.38% 318.75% 650.00% 1621.88% 7
X has promotional objectives.		1234	9.46% 724.32% 1840.54% 3025.68% 19	10.17% 613.56% 830.51% 1845.76% 27	14.00% 710.00% 542.00% 2134.00% 17	0.00% 03.13% 143.75% 1453.13% 17
The relation between developers and X is a “win-win” one.	Relation	1234	4.05% 314.86% 1140.54% 3040.54% 30	6.78% 416.95% 1045.76% 2730.51% 18	8.00% 424.00% 1238.00% 1930.00% 15	6.25% 218.75% 643.75% 1431.25% 10
Developers listen to X.		1234	2.70% 229.73% 2256.76% 4210.81% 8	10.17% 635.59% 2149.15% 295.08% 3	8.00% 428.00% 1462.00% 312.00% 1	0.00% 034.38% 1153.13% 1712.50% 4
Developers exploit X.		1234	25.68% 1941.89% 3122.97% 179.46% 7	28.81% 1737.29% 2220.34% 1213.56% 8	20.00% 1026.00% 1328.00% 1426.00% 13	21.88% 740.63% 1331.25% 106.25% 2
Developers bare their work to X.		1234	18.92% 1432.43% 2440.54% 308.11% 6	18.64% 1140.68% 2433.90% 206.78% 4	16.00% 826.00% 1352.00% 266.00% 3	28.13% 928.13% 931.25% 1012.50% 4
Developers need X.		1234	2.70% 212.16% 941.89% 3143.24% 32	6.78% 430.51% 1832.20% 1930.51% 18	18.00% 924.00% 1236.00% 1822.00% 11	21.88% 737.50% 1231.25% 109.38% 3
By participating to X, I feel to be part of the development team.		1234	28.38% 2128.38% 2133.78% 259.46% 7	45.76% 2727.12% 1622.03% 135.08% 3	44.00% 2224.00% 1222.00% 1110.00% 5	43.75% 1434.38% 1115.63% 56.25% 2
X are respected by developers.		1234	12.16% 924.32% 1854.05% 409.46% 7	25.42% 1527.12% 1645.76% 271.69% 1	18.00% 930.00% 1548.00% 244.00% 2	9.38% 318.75% 653.13% 1718.75% 6
I consider X a labor.	Experience	1234	28.38% 2135.14% 2629.73% 226.76% 5	47.46% 2830.51% 1815.25% 96.78% 4	48.00% 2426.00% 1320.00% 106.00% 3	68.75% 2218.75% 69.38% 33.13% 1
I consider X a hobby.		1234	4.05% 318.92% 1456.76% 4220.27% 15	13.56% 811.86% 754.24% 3220.34% 12	16.00% 812.00% 648.00% 2424.00% 12	25.00% 821.88% 734.38% 1118.75% 6
I consider X an opportunity.		1234	2.70% 210.81% 844.59% 3341.89% 31	11.86% 720.34% 1240.68% 2427.12% 16	10.00% 526.00% 1342.00% 2122.00% 11	12.50% 49.38% 362.50% 2015.63% 5
X is fun.		1234	0.00% 05.41% 451.35% 3843.24% 32	1.69% 16.78% 457.63% 3433.90% 20	6.00% 310.00% 560.00% 3024.00% 12	12.50% 46.25% 262.50% 2018.75% 6
By participating to X, I feel the game more mine.		1234	17.57% 1329.73% 2235.14% 2617.57% 13	40.68% 2425.42% 1523.73% 1410.17% 6	42.00% 2122.00% 1126.00% 1310.00% 5	25.00% 815.63% 543.75% 1415.63% 5
X is stressing.		1234	39.19% 2936.49% 2716.22% 128.11% 6	52.54% 3130.51% 1811.86% 75.08% 3	46.00% 2322.00% 1128.00% 144.00% 2	53.13% 1715.63% 525.00% 86.25% 2
X is a challenge.		1234	9.46% 729.73% 2239.19% 2921.62% 16	28.81% 1730.51% 1823.73% 1416.95% 10	20.00% 1030.00% 1536.00% 1814.00% 7	43.75% 1425.00% 825.00% 86.25% 2

The description of worst and best experiences allowed to frame extend prototyping’s strengths and weaknesses. Table 7 includes the main highlights reported by participants (main themes with numbers of positive—n+—and negative—n−— references and indicative quotes). The importance of communication is spread across the subgroups as well as the coherence between premises and final development. Therefore, a constant exchange with testers and a productive continuity should be a priority in planning such an involvement. This is not a surprise if we consider the importance of the analytic challenge experienced through testing, which requires an adequate feedback to be fully realized. Aside from Kickstarter, community plays a key role in both positive and negative terms, while the feeling of making a difference is relevant in closed and open testing. Other insights concern the limited time for closed testing and the opportunity to see the whole process of production associated with Kickstarter.

Table 7.

Pros and cons.

Closed		Open
Topic(n+/n−)	Examples	Topic (n+/n−)	Examples
Communication (24/31)	– Product was too late in development cycle for any of testers’ feedback to be taken seriously to make a real (…) change .– Devs never fixed many of the bugs found in beta.	Communication (17/25)	– The best is when the community and developers manage to work together despite a larger scale. The worst is when they don't talk much to the community. – Developers simply do not listen to the community and dismiss a lot.
Make a difference (13/0)	– Best experience involved finding a crucial bug which did get fixed before release– The best is when developers fix bugs you found or incorporate your suggestions.	Coherence/incoherence (5/4)	– best is overwatch so far. Its blizzard, they know what they did and we essentially played the finished game, it really was just a demo weekend instead of a beta. – the game released with less content than the beta itself, and (…) numerous “Pay to Win” options were added despite promises during the beta period that this would not occur.
Community (5/4)	– [changes] were run past a player development committee for balancing prior to release. – Have to deal with “leecher” testers.	Making a difference (5/0)	– where the caliber is constantly improving and new content being added resulted in some very rewarding gameplay and time well spent.
Coherence/incoherence (8/9)	– when they realized that (…) [the game] never would be a competitor for LoL or DotA, they implemented a cash shop, got all the money out of the testers they could and then shut down the whole project. – Was reincarnated multiple times, every time suffering from rather ill business practices and progressively worse design decisions.	Community (0/1)	– The community was the worst, full of entitled fanboys.
Time (0/1)	– testing time is too short (…) - you can’t have like a bunch of days to test the whole thing.	Community (0/1)
Steam		Kickstarter
Communication (19/16)	– The best is when developers fix bugs and incorporate suggestions. The worst is if they stop working on the game, or if updates and content become too infrequent or insubstantial. – best: devs usually listen a lot to supporters. worst: listening to supporters often means listening to game design illiterates with bad ideas.	Coherence/incoherence (15/8)	– got scammed, developer absconded with money– Worst is with games that either failed to deliver or are taking way longer than expected, without enough communication as to why. Best is the joy of knowing that I helped make said game.
Community (10/9)	– I went and linked that year old review on the Steam forums and was immediately met with overwhelming hostility from what I later learned were paid supporters. (…) They then stalked my social media accounts and began bothering people connected with me. – Became community hub moderator for said game on the steam and official forums.	Communication (4/18)	– Plus, he keeps a constant weekly flux of communication with his backers via Kickstarter updates and blog posts. – That has been a mess, a small team, poor communication including an offsite forum that got most backers locked out, very late and dubious on how “real” the game.
Coherence/incoherence (7/10)	– it has full mod support and by now the game is essentially finished, but we still get new content. – they all have some good ideas but development either dries up or the devs become unwilling to make needed changes to the game due to sunk-cost feelings.	Transparency (1/0)	– The rollercoaster ride is amazing—gamers need to take a chill pill and learn more about the business of development.

Table 8 lists best and worst projects reported by the participants. It is quite surprising that some appear in both the categories. Darkest Dungeon is one of the most referenced, but also Pillars of Eternity and Rimworld were highly debated.

Table 8.

Best and worst experiences.

	Closed	Open	Steam	Kickstarter
Best	Gwent; Poxnora; Darkest Dungeon; Rift; King of the World; Star Wars: The Old Republic; Warframe; Kenshi; Planet Coaster.	Darkest Dungeon; Beasts of prey; Overwatch; Empyrion; Warframe; Ultima Online; Crusader Kings 2; Cities Skylines; Prison Architect; Fallen Enchantress; Legendary Heroes.	Darkest Dungeon; Exanima; Project Zomboid; Factorio; Rimworld; Warcube; Kerbal Space Program; Boss defiance; star sector; Prison Architect; Endless Space 2; Galactic Civilizations 3; Folk Tale; ARK; 7 Days to Die.	Bard’s Tale IV; Infinite Space 3; Grim Dawn; Project Maiden; Haque; Pillars of eternity; Hyperlight Drifter; Star Citizen
Worst	Darkest Dungeon; Dead by daylight; SuperNova; Warhammer Online; Dead Island; Riptide; Elder Scroll Online; Tabula Rasa; NewZ; Cities XL; Castlestory.	Pillars of Eternity; ARK; Elder Scroll Online; Darkfall 2.	Darkest dungeon; ARK; Rust; Stash; Executive Assault Basement; StarForge; Spacebase; DF-9; Towns; Gnomoria; Stardrive 1 & 2; Legend of pegasus; The Dead Linger.	Elite Dangerous; War for the Overworld; The Mandate; Which Sleeps.

Moved by this outcome, the author interviewed via e-mail Chris Bourassa, the creative director of Darkest Dungeon—a turn-based RPG developed by Red Hook Studios and released on Steam EA in 2015. He observed that Steam EA was essential for this game because his team was entirely self-funded and in need of testers: “Early access enabled us to earn a sustainable living while continuing to enhance and grow the game.” Steam EA community was described as “enthusiastic, excited, and provided us with hundreds of pages of constructive critiques, feedback, and observations.” Moreover, the first testers served as the “bedrock of our community” supporting and helping new players. The range of refining was broad but especially focused on balance, tuning, and bugs, which are usually hard to gather for a small software house. In order to handle such a number of inputs, they hired a community manager, whose work is to monitor Steam forums, Reddit, Twitter, and Facebook and select insights with potential to share with the rest of the development team. “We try to address as many issues as we can, but we don’t always take feedback at face value (…) we use the community feedback to identify areas of concern - for example, a heated debate over a hero’s skills usually means something is either out of whack, or not being communicated clearly enough”. Bourassa and his colleagues have positive words about Steam EA. Coherently with the survey results, he claimed that “Regular communication from a developer to their community is crucial in early access, and it does place additional responsibilities/demands on the team. However, we felt it was worthwhile.”

Discussion

It is challenging to summarize such an overview, which was developed to provide a first picture of extended prototyping through an ID lens—an effort made by following the call to look at mainstream trends in digital entertainment to stage a more effective GD for learning (Egenfeldt-Nielsen, 2007). It can be argued that extended prototyping seems to work as a specific type of formative evaluation. Alpha and beta testing have already been described in these terms indeed (Smith and Ragan, 1999), and the techniques of field test and operational tryout (Smith and Ragan, 1999; Tessmer, 1993) share several similarities (e.g. the almost final state of the materials, the range of testing, the context of application). However, open development seems to be characterized by bigger audiences and a relevant social dimension, which developers are not always able to manage.

Answering RQ1, it is worth to begin by stating that extended prototyping was considered by testers an engaging opportunity. A social cognition fueled by a perceived efficacy (Bandura, 1997) seems a relevant driver, especially considering that instrumental expectations (Wigfield and Eccles, 1992) (e.g. of employment) are irrelevant and the promotional dimension of open development has been highlighted. Pursuing this line, a strong motivation of attending closed and open testing is to try the product before purchase, and subjects recruited tend to buy downloadable contents—i.e. additional upgrades (e.g. new missions, weapons, esthetics)—considering them compatible with extended prototyping. In essence, they are heavy consumers and then potential final users of the tested product; it is not surprising that the majority feels to be alike the average gamer and plays for fun and challenge. This result is aligned with ideal ID formative evaluations which tend to recruit normal learners rather than expert ones.

In allowing audiences to try, debug, and contribute to an evolving product, open development raised positive feelings. Being involved is a relevant reason for participating indeed, and the related motivational design relies on a passionate and analytical predisposition toward a system to improve. This importance is especially true for closed testing and Steam EA, which usually let players affect core game features. A correlation between commitment and perceived impact occurs, proving that online audiences may want to be involved with tasks aside from mere escapism. Open testing and Kickstarter are less felt in in this way; the former is considered a sort of stress test (e.g. for game servers), while the latter relies on an economic investment and then a variable involvement (often just waiting the final release with marginal online activities). In ID terms, it means that for testing an educational product/plan, testers need to be selected by their commitment and analytic attitude. Therefore and addressing RQ2, designers are asked to be up to this exchange by providing a constant feedback and listening to bottom-up suggestions.

In comparison with other types of formative evaluation, extended prototyping is affected by a specifically active role of the testing community, which demands to be informed about its own feedback. Therefore, (game/instructional) designers are also asked to explain, motivate, and reflect on the consequent implications within the development. Such a forced taking a stand (exasperated by the open development’s online dimension in terms of assessment itself and/or debating settings like forums and social media) may entail a more aware and iterative ID, which is an essential step to take for current instructional designers (Gibbons et al., 2014). Moreover, it can prepare this field to the challenges and pitfalls of online and blended learning, which is increasingly social and interconnected. Finally, it entails an emphasis on the role of the instructor per se, which has to be legitimized and explained especially when changes and updates occur.

Despite their own peculiarities, the four types of extended prototyping share this point. In ID terms, the challenge is to find a potential audience with notions of the field targeted but in need of advancement (conversely, the risk is to have a nonsupportive or misleading community). For instance, for an extended prototyping of a graduate course in ID, a productive choice would be to recruit students who have already dealt with it in their undergraduate program. Otherwise, older students can be used to contribute to new and updated editions of a course according to their final assessments. The same process may be applied to professional development with specific formative sessions related to targeted tasks (referring to the ARCS model; Keller, 1987) for testing broader interventions. Therefore (and referring to Bourassa’s words), ID experts must constantly follow the participants and explain the rationale of their own choices. It entails a matter of coherence, self-honesty, and continuity, which should be always followed and preserved. A tacit agreement occurs between designers and testers; ruining it means to weaken their involvement and trust. In addition, this engagement entails commitment and satisfaction, and promotes transparency, i.e. one of the pros reported by participants. Echoing Kemp and UbD models about students’ profiles and expectations, these testers should always know which specific audience will be targeted with the final product. If we want to overturn the exchange from GD to ID and reflect on how the latter can support the former, these insights fit into models like ARCS and Universal Design for Learning (UDL) (CAST, 2011), which stress the need to provide clear goals, milestones, and rubrics during the instruction for making its effectiveness more inclusive and significant. Several of their guidelines (e.g. UDL representation’s principles, the importance of Relevance while explaining changes, updates, and top-down decisions) can assist game developers in handling the communication with their testers. Indeed, a successful extended prototyping seems to rely on the concrete behavior of the producer rather than on its credentials which count but are informed by previous conducts (in ID, a sort of instructional history). Online learning may make this process particularly challenging due to the extended range of audiences and feedback, and hiring a community manager can be an effective solution (see Bourassa’s strategy) to filter suggestions and interact with student testers.

Finally, community seems a crucial parameter according to the results related to the tag gamer, the tendency to share game-related information with peers, and its importance in shaping the testing experience. Therefore, in ID the suggestion is to recruit cohorts of students/employees with shared experiences and objectives, and facilitate their mutual exchange. Addressing potential scenarios, extended prototyping can be deployed by making accessible instructional milestones or first modules for informing the whole educational process in the long term. This would allow a better knowledge and an improved quality rather than waiting for the first rounds of final assessments, which may come too late and without preventing significant critiques. In other words, pilots may be shared with selected individuals, whose involvement can provide formative rather than summative evaluations (Rossett and Sheldon, 2001) and trigger their commitment in promoting the final product—even peer tutoring/mentoring might be considered (Bowman-Perrott et al., 2013) as a potential addition. Collaboration and mutual understanding are productive instruments to enhance learning, social skills, and students’ behavior beyond top-down instructions (e.g. Jeong and Chi, 2007; Klavina and Block, 2008). Extended prototyping can provide not just a significant testing opportunity, but also a chance to improve collaborative thinking, knowledge promotion, and empathy among students (Stahl, 2006; Topping, 2008) in places like forums, chats, and virtual environments. These reflections should be kept in mind when technology issues are at stake, from improving an online infrastructure to refining the design interface in a learning-oriented software.

Regardless, this study presents three main vulnerabilities, which lead to possible future developments. First, its orientation is explorative and aims to outline a first overview of the topic. Further and more structured inquiries are required to take a step forward toward, and pilot studies are necessary to evaluate open development’s feasibility and scope. Second, the sample recruited is small and unequally distributed across the subgroups and provides just one perspective on the topic. Developers and platform owners should be addressed to get a complete picture of this phenomenon. Bourassa was contacted for starting to cover this dimension, which might provide further insights and indications. Third, this article is characterized by a speculative orientation. By depicting a potential bridge between two domains, it could raise mixed reactions due to the explorative findings gathered and leap from game audiences to learners.

Despite these deficiencies, the implications are noteworthy for both researchers and practitioners. The former group has now a starting point to develop in either GD or ID fields. The latter can take advantage of an array of insights and suggestions to follow in instruction and/or game development. Extended prototyping is increasingly popular and it will probably flourish beyond the game industry (e.g. Kickstarter hosts projects of several types). Moreover, it is becoming a familiar practice for new generations, who might be more receptive of its use in learning. Finally, it suggests strategies and malleable factors in designing and handling concrete applications, from management to community, which may support ID projects and performance systems on several levels. The desired outcome of this article is to offer a preliminary step in this direction by contributing to the exchange between ID and GD, which shows a promising potential but requires an ongoing attention to be effectively pursued.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biography

Enrico Gandolfi is an assistant professor in Instructional Technology at Kent State University. His research interests include span media platforms, game cultures, instructional design techniques, and educational technology.

Appendix 1

References

Adams

and Dormans

(2012) Game Mechanics: Advanced Game Design. San Francisco, CA: New Riders.

Badatala

Leddo

Islam

et al . (2016) The effects of playing cooperative and competitive video games on teamwork and team performance. International Journal of Humanities and Social Science Research 2(12): 24–28.

Bandura

(1997) Self-Efficacy in Changing Societies. New York: Cambridge University Press.

Banks

(2013) Co-creating Videogames. London: Bloomsbury.

Bateman

and Boon

(2006) 21st Century Game Design. Hingham, MA: Charles River Media.

Boller

and Kapp

(2017) Play to Learn: Everything You Need to Know About Designing Effective Learning Games. Alexandria, VA: Association for Talent Development.

Bowman-Perrott

Davis

Vannest

et al . (2013) Academic benefits of peer tutoring: A meta-analytic review of single-case research. School Psychology Review 42(1): 39–55.

Brathwaite

and Schreiber

(2008) Challenges for Game Designers. Hingham, MA: Charles River Media.

Brown

and Green

(2011) The Essentials of Instructional Design: Connecting Fundamental Principles with Process and Practice. 2nd ed. Columbus, OH: Pearson.

10.

Buysse

Sparkman

and Wesley

(2003) Communities of practice: Connecting what we know with what we do. Exceptional Children 69(3): 263–277.

11.

Canossa

(2013) Meaning in gameplay: Filtering variables, defining metrics, extracting features and creating models for gameplay analysis. In: Seif El-Nasr

Drachen

and Canossa

(eds) Game Analytics. Maximizing the Value of Player Data. London: Springer-Verlag, pp.255–283.

12.

CAST (2011) Universal design for learning guidelines. UDLcenter.com. Available at: http://www.udlcenter.org/aboutudl/udlguidelines (accessed 8 April 2017).

13.

Consalvo

(2007) Cheating: Gaining Advantage in Videogames. Cambridge: The MIT Press.

14.

Consalvo

and Begy

(2015) Players and Their Pets. Gaming Communities from Beta to Sunset. Chicago, IL: University of Minnesota Press.

15.

Couldry

(2005) The extended audience: Scanning the horizon. In: Gillespie

(ed) Media Audiences. Maidenhead: Open University Press, pp.183–222.

16.

Dale

and Shawn Green

(2017) The changing face of video games and video gamers: Future directions in the scientific study of video game play and cognitive performance. Journal of Cognitive Enhancement 1(3): 280–294.

17.

Dick

and Carey

(2014) The Systematic Design of Instruction. 8th ed. Columbus, OH: Pearson.

18.

Dieker

Rodriguez

Lignugaris/Kraft

et al . (2014) The potential of simulated environments in teacher education: Current and future possibilities. Teacher Education and Special Education 37(1): 21–33.

19.

Egenfeldt-Nielsen

(2006) Overview of research on the educational use of videogames. Digital Kompetanse 3(1): 184–213.

20.

Egenfeldt-Nielsen

(2007) Educational Potential of Computer Games. New York: Continuum.

21.

Flanagan M (2009) Critical play: Radical game design. Cambridge, MA: The MIT Press.

22.

Fullerton

(2008) Game Design Workshop. Boca Raton, FL: CRC Press.

23.

Gandolfi

(2016) To watch or to play, it is in the game: The game culture on Twitch.tv among performers, plays and audiences. Journal of Gaming & Virtual Worlds 8(1): 63–82.

24.

Gandolfi

(2017) Playing the post 9/11 on game service platforms: Premediation in The Division via Twitch.tv and Steam. Convergence. Epub ahead of print 23 November 2017. DOI: 10.1177/1354856517741131.

25.

Gee

(2007) What Video Games Have to Teach Us About Learning and Literacy. New York: St Martin’s Press.

26.

Gee

(2012) An Introduction to Discourse Analysis: Theory and Method. 3rd ed. New York: Routledge.

27.

Gibbons

Boling

and Smith

(2014) Instructional design models. In: Spector

et al . (eds) Handbook of Research on Educational Communications and Technology. New York: Springer, pp.607–615.

28.

Gustafson

and Brunch

(2007) What is instructional design? In: Reiser

and Dempsey

(eds) Trends and Issues in Instructional Design and Technology. Columbus, OH: Pearson, pp.10–16.

29.

Hannafin

and Hill

(2007) Epistemology and the design of learning environment. In: Reiser

and Dempsey

(eds) Trends and Issues in Instructional Design and Technology. Columbus, OH: Pearson, pp.53–61.

30.

Jeong

and Chi

(2007) Knowledge convergence and collaborative learning. Instructional Science 35(4): 287–315.

31.

Jonassen

Cernuca

and Ionas

(2007) Constructivism and instructional design: The emergence of the learning sciences and design research. In: Reiser

and Dempsey

(eds) Trends and Issues in Instructional Design and Technology. Columbus, OH: Pearson, pp.45–52.

32.

Keller

(1987) Development and use of the ARCS model of instructional design. Journal of Instructional Development 10(3): 2–10.

33.

Kickstarter (2017) Stats. Kickstarter.com. Available at: https://www.kickstarter.com/help/stats?ref=hello (accessed 17 February 2017).

34.

Klavina

and Block

(2008) The effect of peer tutoring on interaction behaviors in inclusive physical education. Adapted Physical Activity Quarterly 25(2): 132–158.

35.

Landers

(2014) Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming 45(6): 752–768.

36.

Litchfield

(2007) Instructional project management: Managing instructional design projects on site and at a distance. In: Reiser

and Dempsey

(eds) Trends and Issues in Instructional Design and Technology. Columbus, OH: Pearson, pp.112–122.

37.

Lombardi

(2012) Computer games as a tool for language education. GAME 1. Available at: https://www.gamejournal.it/computer-games-as-a-tool-for-language-education/ (accessed 1 April 2017).

38.

Makuch

(2015) Steam reaches new concurrent user record. GameSpot.com. Available at: https://www.gamespot.com/articles/steam-reaches-new-concurrent-user-record/1100-6431895/ (accessed 17 February 2017).

39.

Marradi

(2007) Methodology of Social Sciences. Bologna: Il mulino.

40.

Merchant

Goetz

Cifuentes

et al . (2014) Effectiveness of virtual reality-based instruction on students’ learning outcomes in K-12 and higher education: A meta-analysis. Computers & Education 70: 29–40.

41.

Merrill

(2007) First principles of instruction: A synthesis. In: Reiser

and Dempsey

(eds) Trends and Issues in Instructional Design and Technology. Columbus, OH: Pearson, pp.62–71.

42.

Merrill

and Willson

(2007) The future of instructional design. In: Reiser

and Dempsey

(eds) Trends and Issues in Instructional Design and Technology. Columbus, OH: Pearson, pp.335–351.

43.

Molenda

(2003) In search of the elusive ADDIE model. Performance Improvement 42(5): 34–37.

44.

Morrison

Ross

Kemp

et al . (2010) Designing Effective Instruction. Hoboken, NJ: John Wiley & Sons.

45.

Reeve

(2001) Understanding Motivation and Emotion. Fort Worth, TX: Harcourt College.

46.

Reiser

(2007) What field did you say you were in? Defining and naming our field. In: Reiser

and Dempsey

(eds) Trends and Issues in Instructional Design and Technology. Columbus, OH: Pearson, pp.2–9.

47.

Reiser

(2017) Eight trends affecting the field of instructional design and technology: Opportunities and challenges. In: Lai

and Lehman

(eds) Learning and Knowledge Analytics in Open Education. Berlin: Springer, pp.139–147.

48.

Ritchhart

Church

and Morrison

(2011) Thinking Visible: How to Promote Engagement, Understanding, and Independence for All Learners. San Francisco, CA: Jossey-Bass.

49.

Rossett

and Sheldon

(2001) Beyond the Podium: Delivering Training and Performance to a Digital World. San Francisco, CA: Jossey-Bass/Pfeiffer.

50.

Saldana

(2016) The Coding Manual for Qualitative Researchers. 3rd ed. London: Sage.

51.

Schell

(2008) The Art of Game Design: A Book of Lenses. Boca Raton, FL: CRC Press.

52.

Smith

and Ragan

(1999) Instructional Design. 2nd ed. New York: John Wiley & Sons, Inc.

53.

Stahl

(2006) Group Cognition. Vol.106. Cambridge: The MIT Press.

54.

Sugar WA and Luterbach KJ (2016) Using critical incidents of instructional design and multimedia production activities to investigate instructional designers’ current practices and roles. Educational Technology. Research & Development, 64: 285–312.

55.

Swain

(2008) Master metrics: The science behind the art of game design. In: Isbister

and Schaffer

(eds) Game Usability. Burlington, MA: Morgan Kaufmann Publishers, pp.119–140.

56.

Sylvester

(2013) Designing Games: A Guide to Engineering Experiences. Sebastopol, CA: O’Reilly Media.

57.

Tessmer

(1993) Planning and Conducting Formative Evaluation. London: Kogan Page Limited.

58.

Topping

(2008) Peer-Assisted Learning: A Practical Guide for Teachers. Newton, MA: Brookline Books.

59.

van Merrienboer JJG (2007) Alternate Models of Instructional Design: Holistic Design approaches and complex learning. In: Reiser RA and Dempsey JV (eds) Trends and Issues in Instructional Design and Technology. Columbus, OH: Pearson, pp.72–81.

60.

Wenger

(1998) Communities of Practice: Learning, Meaning, and Identity. Cambridge, MA: Cambridge University Press.

61.

Wigfield

and Eccles

(1992) The development of achievement task values: A theoretical analysis. Developmental Review 12: 265–310.

62.

Wiggins

and McTighe

(2005) Understanding by Design. Alexandria, VA: ASCD.

Playing,debugging,learning: A proposal between game and instructional designs via extended prototyping

Abstract

Keywords

Introduction

Seeking a crossroad between instruction and gaming

ID and GD

Combining perspectives

Research design

Methods

Results

Discussion

Footnotes

Declaration of Conflicting Interests

Funding

Author Biography

Appendix 1

References