Rethinking iterative design: A case study of a design team’s handling of user representations

Abstract

In design practice, iterative approaches are widely used as a type of organization that aims to facilitate user-oriented design by soliciting user feedback before or after product launch. However, despite their widespread use, iterative approaches do neither automatically nor in themselves lead to productive user involvement. The article presents a case study that contributes to an in-depth understanding of iterative design in practice. We followed a design team for a year to investigate how an iterative organization of the design work configured the possibilities for user involvement, enabling some kinds of user feedback at specific moments, while inhibiting others. We also point to key issues that appeared to escape the designer-user interactions that were set up by the iterative approach. All in all, this article offers a case study and a conceptual framework, shedding light on both intended and unintended consequences of how an iterative design approach unfolds in practice.

Introduction

Since the 1970’s terms such as co-design, participatory design and user involvement have been debated and developed in the field of design (Asaro, 2000; Törpel et al., 2009). At some point the idea of engaging future users in the design process may have appeared radical and new, but decades of experimentation have gradually turned user involvement into a widespread and normal practice (Wynn and Eckert, 2017). Today, arguments about user involvement seem to be kicking in an open door. The potential benefits of user involvement are well documented (e.g., Törpel et al., 2009; Simonsen and Robertson, 2013; Von Hippel, 2005), the methods for user involvement are countless and widely available (Hyysalo et al., 2016), and there are few sectors if any, public or private, where some leading actors would not somehow practice user involvement.

Alongside the rising commitment to user involvement, the design field has developed a plethora of practical tools for orchestrating the involvement of users. In terms of project design and practical organizing, a crucial development is the wide adoption of iterative approaches, where the idea is to develop the product in relatively small steps interspersed with occasions for soliciting user feedback. Reportedly, iterative styles of development have been used in IT design since as early as the 1950’s (Larman and Basili, 2003). In the 1980’s, the rise of user-oriented and participatory design drove a wide adoption of iterative approaches (Simonsen and Robertson, 2013). In this millennium, the IT industry has made it a common practice to deliver their products in the shape of a series of gradually updated versions, thus organizing a substantial part of their product development as on-going iterative work (e.g., Hyysalo and Lehenkari, 2002, 2003; Johnson, 2013; Pollock, 2003). In recent years, even manufacturing industries traditionally adhering to planned, stage gate models of product development are beginning to add more iterative or agile elements into their process (Ahmed-Kristensen and Daalhuizen, 2015). Another recent development is the association of iterative approaches with a surge of interest in the so-called minimum viable product approach—the strategy of releasing a product with just enough features to be usable by early customers who can then provide feedback for future product development (Frederiksen and Brem, 2017; Moogk, 2012; Ries, 2011). Varieties of iterative approaches thus appear to be spreading to an increasingly broad set of design and development contexts where emphasis is put on being responsive to markets and users.

The close connection between (a) the ambition of involving users and (b) the choice of iteration as the practical way of achieving this ambition has been evident for a very long time. In the pioneering work of participatory design, designers used low fidelity prototypes, such as cardboard boxes, that they and the future users could move about together to explore various design options and their consequences (Ehn and Kyng, 1991). In this way, the collective iteration of possible solutions was precisely the move that opened the designers’ reflection and decision process to user engagement. Today, the combination of iteration and user involvement is broadly considered to be an effective and appropriate way to organize a design process. Co-design and iteration have thus become essential features of prevailing design philosophies such as participatory design (Simonsen and Robertson, 2013), design thinking (Rowe, 1991), agile programming (Beck et al., 2001), and minimum viable product strategies (Ries, 2011). In actual design work, the formulaic prescriptions of design philosophies and methods are rarely followed to the letter (Jensen and Andreasen, 2010; Suchman, 1987). But vast numbers of contemporary design and development projects work have an affinity with idealized iterative approaches by working with project designs, plans and milestones that configure the design process into a series of iterations before or after the first product launch.

The empirical object of this article is iteratively organized design processes. By this we mean the designing practices—not the ideals formulated in design philosophies or principled design methods—but the actual work that is carried out by designers and developers as they maneuver their way through projects where feedback from users is solicited and incorporated at various points. Design practices are inspired by and drawing on design ideals and formulaic methods, but plans and methods are essentially weak resources for action, because any kind of actual designing requires a vast amount of work to interpret, specify, and make sense of pre-given plans and methods, not to mention the work needed to improvise and develop solutions to unforeseen problems on the fly (Suchman, 1987). For this reason, this article is not focusing on the questions that occupy zealous promotors of specific design recipes, such as whether a recipe contains the most optimal principles and whether people follow it faithfully and to the letter. Instead, we are interested in developing a close-up understanding of how iteratively organized design processes unfold in actual practice, and whether these iterative practices produce the constructive and beneficial form of user-engagement that is commonly cited as their purpose and justification.

To explore iterative designing in practice, this article investigates a specific project—the EMOVE project—that from the very beginning had clearly stated ambitions of engaging users in an iteratively organized design process. In the funding application the three partners of the project stated that the project would develop through “an iterative process in which knowledge from one work package will be utilized as a resource in other work packages, which will enhance the development of the IT system (emphasis added)” (see Figure 1).

Figure 1.

Excerpt from EMOVE’s project application describing the project as a series of work packages (WPs) forming an iterative loop.

The commitment to an organized iterative process was later formalized in an elaborate 3-year Gantt chart (Figure 2) outlining the key activities of the project: In the early phase of the project, several workshops with current users of a precursor system where scheduled. These activities could be seen as iterations of preliminary product ideas. In the next phase, a team of designers was given 6 months to develop a first version of a functioning product, which would subsequently be implemented in two user organizations. This part of the project appeared to be inspired by a minimum viable product strategy, where the implementation of a quickly-built, functional system was to be used to solicit feedback from people who use the system as a part of their everyday work. Finally, in the last couple of years of the EMOVE project, six iterations were planned. In each of these iterations the product would be evaluated, errors would be fixed, new features would be added, a new version would be released, user feedback would be solicited whereafter, yet another upgraded version of the product would be made and released. This, again, seemed to be broadly in line with a minimum viable product strategy. From reading the project application and the Gantt chart (Figure 1), it thus seemed clear that iteration was a pervasive principle in the organization of EMOVE.

Figure 2.

Excerpt from Gantt chart.

Our aim with analyzing the EMOVE case is to understand how an iterative organization of design work influences how a design team incorporates various types of “input” about future use (requirements, preferences, visions, etc.). The question might seem odd, or banal even, if one readily accepts the suggestion of idealized design philosophies that working iteratively will by itself lead to a sufficient and satisfying level of user involvement. However, decades of research have shown that user involvement in practice is not a simple thing. As Hyysalo & Hyysalo point out, user involvement is a practical accomplishment that includes mundane work as well as strategic actions (Hyysalo and Hyysalo, 2018). User involvement consists of a series of situated activities that somehow translate users into materials that somehow has a bearing on designs and that sometimes (and sometimes not) gain the approval of the so-called end users (Botero et al., 2020). Understanding user involvement as a practical accomplishment in real-life design projects therefore requires the development of a suitable theoretical and empirical sensitivity—a kind of sensitivity that has grown steadily in recent years with a stream of in-depth studies of design practice emerging in the intersection between STS and design research (e.g., Botero et al., 2020; Hyysalo, 2010; Jensen and Petersen, 2016; Kohtala, 2016; Pollock et al., 2016; Savolainen and Hyysalo, 2021; Wilkie, 2010; Wilkie and Michael, 2009). In this article, we contribute to this line of work with the added ambition of understanding how an iterative organization of design work factors into the already complex process of user involvement.

The structure of the article is as follows: First we introduce the ideas and theories that have inspired and framed our study. Second, we present the EMOVE case and our sources of ethnographic material. Third, we develop our analysis of how the design team—working under “iterative conditions”—handled and prioritized a variety of user representations during the project. In this part, we describe the design team’s work pattern as a series of different “modes of handling user representations.” Fourth and finally, we discuss how the possibilities for user involvement were configured by the iterative organization of the design team’s work.

Theoretical framings of the study

The starting point for our analysis of user involvement is the notion of user representations. User representations were introduced by actor-network theorists in the 1990’s who argued that studies of design work should focus on the specific materialities, media, and formats through which users are translated and brought into contact with designers (Akrich, 1992, 1995). This analytical move is important because it frees us from the rather ill-formed question of whether specific “real,” “in-the-flesh” people are in the design process or not. Instead, the focus on user representations directs our attention to the many materially heterogeneous ways in which user perspectives and information about users are established, translated and mediated between users and designers (Akrich, 1995; Cooper and Bowers, 1995; Hyysalo and Johnson, 2016; Wilkie, 2010). In the case of EMOVE, for instance, we have observed that the great majority of design work took place without the physical presence of people we might designate as users. But users were nevertheless continuously present, or represented, to the design team. For instance, we have observed a programmer who repeatedly argued that the new software his team developed must reduce the number of clicks necessary to perform a particular task to make it more user-friendly. In this way, the programmer rhetorically evoked a general user who would be pleased by simplicity and annoyed by too much clicking. Another member of the design team, a manager, also brought forth a specific representation of users. At one point, for instance, he stressed that the design team must prioritize the demands of two specific user organizations because they were “paying customers,” whereas a third organization would have to wait. Even in situations when no one speaks a word about users, the user representations made a forceful presence: When the project’s UX designer sat at her desk, engulfed in building the interfaces of the new system, she drew on a standard library of icons. Each of these icons was infused with cultural ideas about “what things normally look like” and “what users will recognize.” Representations of users were thus swirling around the design team in many different forms and shapes, from explicitly stated design principles to the nuts and bolts of software development. The question yet to be answered is how these many user representations became incorporated in the iteratively organized design process.

User representations in design projects and work

Hyysalo and Johnson (2016) have conducted a comprehensive review of the user representation literature. Similar to our own observations, Hyysalo and Johnson point out that design teams’ discussions, tables, computers, and workshops tend to be crowded with envisioned users originating from numerous sources. Business models or regulatory demands carry implicit or explicit ideas about users’ motivations and behavior (Hyysalo, 2010; Konrad, 2008; Oudshoorn and Pinch, 2003). Parallel technologies may suggest what future users will accept and appreciate (Hine, 2000; Pollock and Williams, 2009). The designers may also draw inspiration and knowledge from their own or their colleagues’ successes and failures in previous development projects (Hyysalo, 2010), or they may base their assumptions about future users on prevailing cultural ideas, “folklore” (Grint and Woolgar, 1997), or their own logical reasoning about users’ needs and wants (Akrich, 1995; Hine, 2000; Oudshoorn et al., 2004; Russell and Williams, 2002). Finally, adding to all the other sources, user representations may appear in the shape of explicit statements collected from users who have participated in design workshops, market surveys, or other activities aimed at gathering requirements for a future product or system (Heiskanen et al., 2010; Johnson, 2010).

The implication of Hyysalo and Johnson’s exposition of the multiple sources of user representations is that availability of user representations is generally not an issue. Rather, the issue is that the abundant user-related information does not fit together like puzzle pieces, simply because there is too much, it is too varied, and it is mutually incompatible (Hyysalo and Johnson, 2016; Jensen, 2013; Johnson, 2007). Furthermore, user representations are known to be objects of discussion, strategizing, and contestation among the actors within or surrounding design teams (Ross, 2011; Suchman, 1995). The questions of how to select, interpret, and apply user representations are thus heavily entangled within the internal and external politics of design organizations, as well as the broad variety of practical resource concerns. For design teams, user-oriented design is therefore less a matter of solving a puzzle than of addressing a pragmatic question of determining—sometimes by negotiation, sometimes by conflict—what to build with the information and resources available in the moment.

Hyysalo and Johnson (2016, pp. 85–86) argue that we only have “relatively scant” research on how and why particular user representations become important in the situated action of design. They nevertheless suggest that longitudinal or biographical studies of design may offer some important points on which future research may build. In the following, we review some of the key points made by these studies.

First, it is well-documented that the uptake of user representations is not determined by how “good” or “persuasive” they are in some abstract sense. Instead, uptake seems driven by the specific demands and concerns of the design teams. For example, in a longitudinal study by Hyysalo (2006), user representations that were aligned with business demands or pressing technical issues were considered by the design teams, while those that did not align with business or technological issues gradually lost priority, even if they were considered significant by all team members. In the case described in this article, we observe, similarly, that the design team’s attention is highly selective and pragmatic depending on where the team found itself in the iterative process of the project.

A second point from the longitudinal design literature relates to the importance of timing and co-evolution. As a project or business develops over time, the types of user representations adopted may change quite dramatically; early phases of welcoming a broad variety of ideas from users may give way to later stages with a strong push for standardization (Johnson et al., 2014; Pollock et al., 2016). Not only may the openness change dramatically but also the methods and types of user representations. One vivid example of changing methods is given by Johnson (2010), who studied the development of the Habbo Hotel, an online recreational game that became immensely popular around the turn of the millennium. Within the span of a few years, the game developed from a small hobby project with a few hundred users to a vast enterprise with versions in 11 languages and 15 million users per month. In the same timeframe, the company adopted and sometimes discarded close to 30 different methods for knowing its users. It moved from an initial phase with highly informal methods, such as designers’ personal sense of the game, to later stages, with large-scale systematized methods, such as usability testing, market surveys, and automated web analytics. Such studies as Johnson’s (2010) underscore that any understanding of how design teams apply user representations must consider the timing or evolution of the project or business. Accordingly, our analysis of the EMOVE case is anchored in time in the sense that we explain the design team’s handling of user representation in light of the evolving relationship between design team, product and user organizations.

A third point, repeatedly emphasized in longitudinal studies, relates to the spheres and domains through which user representations travel (Hyysalo et al., 2019). User representations are obviously important to design teams, but it is noteworthy that they also circulate among broader society. This point is well made by Du Gay et al. (2013), who, with their term circuit of culture, argue that the meaning of an artefact is realized through a series of interlinked processes: the representations of the artefact in media, its associated identities, its production, its consumption, and the mechanisms regulating its distribution and use. From this, it follows that the actions of design teams should be seen in the context of the previous, simultaneous, and subsequent actions of many other actors—consumers, mediators, regulators, distributors—who also take part in configuring the meaning of a particular artefact. Thus, rather than imagining a design team as a first mover and point of origin for new products and cultural meanings, we may think of a design team as merely one location in a broader circuit of culture, where other actors are continuously engaged in configuring products and users. In our case, we have attempted to depict the circuit-like nature of the design process by describing both how “external” concerns such as economic considerations “flow into” the design process and by describing how objects “flowing from” the design process, such as the first version of the system, generate responses from people external to the design team.

Shifting modes of handling user representations

We have so far indicated that we will focus on the handling of user representations by a design team, and we have made the point that this handling must be understood in the context of the iterative organization of the project as well as in the context of a wider circuit of culture. In practical analytical terms, this means that we need a concept to describe the “styles” of handling user representations that unfold as the project proceeds through its various phases. To this end, we have taken inspiration from the British STS scholar John Law’s notion of modes of ordering (Law, 1994). Law developed his notion as a part of an organizational ethnography, where he described how the organization appeared to draw on several quite different and mutually incompatible mini-discourses or strategies when conducting their daily business. The same organization might thus work in an “administration” mode focusing on legality and proper procedures but also in an “enterprise” mode emphasizing the need to break rules and do radical new things. Each of these modes of ordering, Law argues, comes with a set of practices, normative ideas, and characteristic tools. What we take from Law are not the specific modes of ordering he derived from his case study, but the more general idea that an organization can be described as an established disorder with multiple somewhat incompatible ordering efforts that take center stage at various points. Accordingly, we describe the unfolding of the design process as a series of phases, each of which is dominated by a particular mode of ordering, while other modes are lurking in the periphery or elsewhere in the organization. In each phase, the design team attempts to address certain problems and achieve a certain type of progress for the project. Integral to this, the design team prefers certain kinds of user representations and certain ways of handling these user representations. By describing the design process as a sequence of modes of ordering, we do not want to suggest that only one mode of ordering matters at a time. However, we do argue, based on our ethnographic study, that almost all the design team’s attention and efforts are centered on one mode of ordering at a time. Our analysis of the design process is therefore a phase-by-phase exposition of a series of dominating modes of ordering. For each of these modes, we describe what types of user representations the design team mobilizes, as well as what “shape” these user representations must take to have a decisive influence on the design process and decisions. In the final part of our account, we argue that the dominating modes of ordering should be regarded as a type of hard-won framing that the design team establishes at various points (Callon, 1998). However, these framings invariably overflow; at each stage of the project, we can point to types of user representations that tend to escape the dominating modes of ordering and therefore emerge or reemerge as unresolved issues at a late project stage. As Law puts it: There is no order, only ordering. What we describe is thus a pattern of work, where a design team moves swiftly through a design process—emphasizing one mode of ordering after another—while also generating overflows and deferring some matters for later. This overall pattern is in our view a fairly accurate depiction of how fast-paced iterative design work evolves. Building on this, our final discussion is an attempt to rethink the merits of iterative organization by reflecting on its practical consequences for the handling of user representations.

Case and methods

The case discussed in this article is derived from EMOVE, a four-year research and development project funded by a Danish research foundation that sponsors research collaborations between universities and companies. The EMOVE project included three partners: a private company that we will call MatchDesign, the University of Copenhagen, and Aalborg University.

The key goal of the EMOVE project was to expand a concept that MatchDesign had developed in the previous 3 years: matching older Danish citizens with “language students,” that is, foreigners who are learning Danish. The older person and the language student would meet regularly in the older person’s home. They would have conversations in Danish, and they would often develop a mutually beneficial companionship. MatchDesign had matched more than 700 pairs, and the company had been widely praised for its innovative approach to voluntary work. In the process, MatchDesign had developed considerable expertise in creating good matches and supporting the matched pairs with conversation-starting materials and supportive follow-up calls by the MatchDesign staff. MatchDesign had also developed an IT system to support the matchmaking process. The purpose of the EMOVE project (“Enabling the Matching Of VoluntEers”) was to transform the pioneering efforts of MatchDesign into a broader and more far-reaching business endeavor. The EMOVE partners would gather knowledge about matchmaking and use this knowledge to develop a significantly improved successor to the previous IT system. The new system would support and facilitate matchmaking not only for the specific purposes of MatchDesign but for many other kinds of voluntary organizations across Europe working with matching.

Ethnographic methods and data

The authors of this article are the two participants of the EMOVE project from Aalborg University. Informally, we described our role in the project as “midfield players” between, on the one hand, the partner from Copenhagen University, representing expertise in welfare states and volunteer research, and, on the other hand, the IT design team from MatchDesign. Our responsibilities were to conduct UX studies, facilitate co-creation workshops, take part in the onboarding of new user organizations, and provide insights to the programmers. All activities were carried out in close collaboration with the other project partners.

The empirical material for this article was collected while we engaged in our midfield player roles. We followed and collaborated with the six people on MatchDesign’s design team: a MatchDesign manager, MatchDesign’s chief technology officer (CTO), a UX designer, two programmers, and a co-founder of MatchDesign. AUTHOR 2 was physically present in the offices of MatchDesign 2 days a week, when not prevented by local COVID-19 restrictions. She participated in bi-weekly design meetings, meetings with customers, and a series of testing and onboarding activities with the two companies that were the first users of the new system. She was also given access to documents concerning the design, development, testing, and onboarding of new customer organizations. Throughout the first 12 months of the project, AUTHOR 2 wrote weekly field notes, collected work documents, took minutes from meetings and workshops, and interviewed design team members. Our insights into the system and its development were further supported by AUTHOR 1’s role as a member of EMOVE’s project management team. He participated in writing the project description and funding application. During the project he had continual contacts with the two other project managers from University of Copenhagen and MatchDesign.

The analysis of the material developed alongside the field work. We held weekly meetings to reflect on the ongoing field work, we discussed preliminary ideas with academic colleagues, and we gradually settled on the idea of analyzing on the design team’s different modes of handling user representations during the iterative process. To further clarify our analytical focus, we conducted a thematic analysis (Braun and Clarke, 2006) to identify the variety of user representations in the material and explore their function as a part of different modes of ordering.

Analysis

In the following, we present an account of the design process beginning with the work that took place before the launch of the project and ending with a series of concerns that emerged after the implementation of the system in two user organizations. In keeping with our analytical focus, we present the story as progression through a series of dominating modes of handling user representations (Table 1).

Table 1.

Three modes of handling user representations.

Mode of ordering	Normative aim of this mode	Preferred types of user representations
Collecting attractive visions	Create and energize a coalition behind a common project.	User representations that have broad appeal and no opposition.
Negotiating co-existence	Balance concerns and settle dilemmas about which elements to fit in and how.	User representations that can be used as resources for articulating and settling specific design dilemmas at hand.
Removing stumbling blocks	Launching and implementing the system without alienating customers.	User representations that point to specific problems that annoy users and that can be resolved in a short period.

First mode: Collecting attractive visions

The official launch of the project took place in May 2021, but before this, at least 6 months were spent by the company and two university partners on developing an application to a research foundation and a detailed project plan. This process took place in parallel with internal discussions within MatchDesign about the business potential of the new software. After funding was secured, MatchDesign hired a CTO, a UX designer, and a programmer, who made additional contributions to the ongoing discussion about the system to-be developed. Throughout this preparatory phase, a variety of user representations were articulated by the participants. These user representations were most often condensed versions of the promises or appealing features of the future system. We will therefore describe this mode of handling user representations as a process of “collecting attractive visions.” In the project plan, for instance, it was stated that the project would develop a “widely accessible Software as a Service (SaaS) tool” that would be accessible and attractive to NGOs across Europe, and that these organizations would be able to install and customize the system in less than 1 day. One MatchDesign manager coined the phrase that the EMOVE system would become the “Wordpress of Matchmaking.” The tool would also be compliant with EU’s General Data Protection Regulation (GDPR) and would “significantly improve how NGOs recruit and engage with volunteers, including vulnerable groups, while also improving the NGOs’ ability to exercise a human touch in the matchmaking process” (EMOVE funding application). In addition to the promises written into the research application and project plan, a discussion within the design team gave rise to additional visions of future users: the CTO evoked the ambition that the new system should require significantly fewer clicks to complete the same tasks and that the amount of shifting between screens should be cut down considerably compared to MatchDesign’s previous system. Some visionary representations of future use were also inspired by discussions with potential clients. In this vein, one of the managers articulated the expected ease and simplicity of the system by evoking the image of a future volunteer worker who could handle matchmaking on her phone while taking a forest stroll.

When looking at the collection of attractive visions it is striking that the visions all have the characteristic elusiveness and positive tone that one often finds in the early stages of a project where participants attempt to the gain internal and external support and momentum for a project. This also indicates the kind of progress that the project leaders and the design team are working to achieve in this phase. The project will be moving forward if people are willing to join and support it—and that is indeed what happened. MatchDesign and the two university partners agreed that this was a worthwhile project, Innovation Foundation Denmark agreed to fund the project, and new people were recruited and hired to the project. The kinds of user representation favored at this stage were user representation that appealed broadly to all potential partners. They were also user representation that did not bring up conflicts or dilemmas between different possible aims of the project. On the contrary, the collection of attractive visions suggested that a range of goals would all be achieved by the project: A widely available software as a service solution and great business potential and GDPR compliance, etc. In the next phases of the story, we shall see a shift to an entirely different mode of handling user representations.

Second mode: Negotiating co-existence

Around May 2021, the design team began building various parts of the system such as the basic structure of the underlying database and the windows where some of the key operations of the matchmaking work would take place. While engaging in this work, the design team quickly found itself addressing a series of questions about which elements could be fitted into the system and how to balance various concerns. As a part of this, the design team also had to reconcile different or even conflicting ideas of what the users might want. Some of these ideas came from the MatchDesign manager, who had 3 years of experience in using an earlier IT system to handle the matching of older Danes and language students. Other ideas came from workshops where representatives from the two first customer organizations had explained how they organized their current matchmaking processes, and yet other ideas were derived from the programmers’ and UX designer’s general thoughts about what a system should look like. At this stage, the design team was engaging in a mode of handling user representations that we will call negotiating co-existence, because the key issue was to relate and balance different user representations as a part of determining if or how specific features could be incorporated into the space of the same system.

One vivid example of this new mode of handling user representations was a discussion in the design team about a window where the user opens and juxtaposes the profiles of two people (volunteers). After viewing the two profiles, the user decides whether to match the volunteers. The design team had created a first version of this window, but drawing on their experience from previous projects and personal judgement, they quickly agreed that it was “too cluttered” due to the amount of text on the two profiles. The designers thus brought a specific user representation into the discussion (users don’t like clutter), which led them to articulate a dilemma between two courses of action:

Dilemma version 1:

Should we keep the current 'cluttered' version of the window

OR

should we change it?

In the subsequent discussion, this dilemma was further qualified and elaborated by discussing possible solutions and by bringing more user representations into the discussion. To solve the presumed problem with “clutter,” the team considered the idea of removing a substantial part of the text from the profiles. Instead, they would make a link on each profile that allowed the user to open another window with the full amount of information. But when discussing this solution, three additional user representations were articulated. Members of the design team argued that users would prefer to see the full amount of profile text that users would have a distaste for too much clicking and that users would have a distaste for too many windows. A new version of the dilemma, now qualified by the three additional user representations, was thus articulated:

Dilemma version 2:

Should we reduce the cluttered window

OR

Should we avoid limiting the amount of visible information, avoid introducing more clicks and avoid introducing more windows?

The odds seemed stacked against the reduction of clutter, but that was nevertheless the path that the design team decided on. The crucial step toward this decision was made when the team introduced a distinction between different kinds of profile information. On the one hand, they argued, there was “key information” on the other hand “additional information.” Based in this, the design team developed a solution where only ‘key information’ would be shown on the profiles, whereas the viewing of “additional information” required the users to open a new window. A third version of the dilemma was thus articulated:

Dilemma version 3:

Should we reduce clutter and show only key information with the option of opening another window with additional information

OR

Should we show all information and avoid additional clicks and windows.

The mode of handling user representations that we see in this unfolding discussion in the design team has a number of characteristics that differ from the previous mode of “collecting attractive visions.” At this stage in the project, the purpose of evoking user representations is not to gather a broad coalition of supporters, but to explore and solve a specific design dilemma at hand. User representations are evoked by the design team to articulate the consequences of alternative courses of action that the team must choose between. As we have seen, the team draws on previous experience and personal judgement, when weighing the consequences of alternatives. But often this is not enough to reach a decision, such as in this case where the reduction of clutter can only be achieved by introducing other unwanted features: less information, more clicks and more windows. To handle the dilemma, or break the stalemate, the team develops a new interpretation of the matter at hand. They introduce the idea that the (excessive) text can be divided into “key information” and “additional information.” With this distinction, it becomes more justifiable to remove a part of the text from the window where the matching will be taking place. We have followed several interactions in the design team that unfolded in a similar way. User representations were drawn into the discussion as a resource for articulating dilemmas, but additional interpretation and distinctions were needed to arrive at final decisions. In one case, the designers prioritized a list of wishes from user organizations by distinguishing between those coming from paying and non-paying customers. In another case, a dilemma between the simplicity of icons and additional guiding text was solved by introducing a distinction between experienced and inexperienced users, and then making the argument that the system was intended for experienced users. From this premise, the design team reasoned that additional guiding text was less important than maintaining the simplicity of the interface.

The design team’s deployment of user representations as resources for handling design dilemmas also meant that most of the user representations from the previous mode of “collecting attractive visions” slid out of focus. When handling design dilemmas in this phase there was no reference to broad ideas of what the system would achieve such as GDPR compliance, significant improvements in how NGOs engage volunteers, facilitation of “a human touch” in the matchmaking process, or ease-of-use to the degree that future matchmakers could be taking a forest stroll while doing their job. All these broadly stated ideas about future use had contributed to gathering the coalition of supporters around the project, but none of them were translated into versions that might have a direct bearing on design dilemmas that emerged while building the first version of the system.

In the previous phase of the project the participants also formulated the vision that the system would be a “widely accessible Software as a Service tool.” This vision also slid out of the talk of the design team, but apparently not because it was unrelatable to the design teams work, but rather because it had now become the premise of the design team’s work. The team was indeed building a SaaS system. A final user representation from the previous stage continued to play a role: the idea that users would want a system with fewer clicks and fewer windows. However, while this user representation figured as a goal in itself when working in the mode of “collecting attractive visions,” it now became merely one desirable feature that had to be balanced with others.

To sum up, the second mode of handling user representations—negotiating co-existence—is a process where the design team attempts to bring the project forward by articulating and settling specific design dilemmas. In this context, they attend to those user representations that they can use to articulate the consequences of alternative courses of action and weigh in on a decision. Sometimes user representations immediately suggest solutions, sometimes dilemmas are settled by the design team’s introduction of additional interpretations and distinctions. In either case, the focus for the design team shifted from broad ideas about future usage to the specific challenge of building what they considered to be a “good system.”

Third mode: Removing stumbling blocks

The work of fitting and balancing a variety of concerns into the system lasted roughly 6 months. After this, a 2-month period followed where the system was implemented in two user organizations. In this part of the project, the user representations were “in the flesh” so to speak. The design team engaged in more than 30 hours of testing and onboarding sessions with users during which the designers received praise and complaints as well as suggestions for additional features. In our observations of how the design team handled user representations, we noticed that the previous focus on articulating and settling dilemmas now moved to the background. Instead, the overriding concern now was to identify and remove any kind of stumbling blocks that would jeopardize the onboarding of the user organizations. At this stage, the design team appeared to be in a high alert mode, and they were willing to move very quickly and assume costs that they would have been more reluctant to accept in earlier stages. In one situation, a member of the design team, who was also manager of MatchDesign, believed that one of the customers was unwilling to pay the full price for the system, and so he extended them a discount. In another situation, the programmer and UX designer realized that the emails sent by users through the platform looked “odd” compared to other standard email programs, and so the design team quickly changed the system to adhere to the prevailing standards.

We have chosen the term “removing stumbling blocks” to describe the mode of handling user representations that prevailed at this stage. The key concern for the design team was to get the system up and running and not to lose any customers. To further this aim, they focused on specific solvable problems that would clearly annoy the users or would appear eye catching or embarrassing. At the same time, other kinds of input from users slid out of focus, especially those indicating problems that could not be fixed quickly, such as some user’s broadly stated demand for more features they could use to generate overviews and reports. These kinds of issues were postponed to the versions and updates of the system that would or might happen in the future. Problems that reopened the balancing acts achieved in the earlier stage were also not in focus; the ambition was to get a version of the current version of the system up and running.

Emerging issues in the implementation phase

When introducing our inspiration by Law’s notion of modes of ordering, we emphasized that a mode is not a perfect order, but merely an attempt to frame and manage certain issues; each mode invariably pushes certain matters out of focus, generates overflows, or defers problems until later. Our account of the dominating modes of handling user representations in the three phases of the project must therefore be supplemented with an account of what was unaddressed or “left over” after the design team had produced the first version of the product. In this section we pursue this line of thought by describing two significant discussions that arose when the system was first implemented in two customer organizations. Both discussions provide us with a sense of what was not handled by the previously mentioned three modes of handling user representations.

The first discussion was about what motivates the users of the system. It appeared that there were two very different ideas about this. From the early beginnings of building the system, the design team had worked from the assumption that users would want a clear list of tasks with all the necessary information, and that the users wanted to finish all their tasks by the end of the day. The design team referred to this as the dream of “inbox zero.” However, when users tried the system several of them complained that the task list made their work too fragmented and that they couldn’t see the big picture of what they were working with. When the design team and the university partners talked to these users, it became clear that they had a different motivational theory. These users saw their job as taking care of a population of matches and they felt a sense of pride when many matches were going well. They focused on contacting and helping the matches that needed help or creating new matches. For them, the work was never-ending and therefore the idea of having a clean inbox at the end of the day made little sense. Instead, they were interested in overviewing the whole population and direct their efforts to where it was needed the most. Unfortunately, the new system didn’t offer much of an overview. In fact, it made it harder for users to see the big picture compared to some of the systems, they had worked with before. The people who adhered to the caring-for-the-population motivational theory were frustrated because of this.

The second discussion centered on the users’ somewhat frustrated sense of being able to influence the development of the system. In the early stage of the project, before the first version of the system was build, the user organizations were invited to “design workshops.” At this stage the designers sketched their preliminary ideas for the system and the users explained their current match-making practices. However, it was very difficult for the users to understand the implications of implementing the new system their organizations. After the design workshops, the design team had 6 months to build the system according to the EMOVE project plan. To reach this deadline, the design team worked intensively and deployed state-of-the-art tools such as “out-of-the-box” features that made it possible to speed up the building of a viable system. When the first version of the system was ready, it was launched and implemented in two user organizations. Many of the system’s features were immediately liked and appreciated by the users, in particular the fact that all types of communication (phone calls, text messages, emails) were now integrated in one system rather than scattered on several devices. But there were also aspects of the systems, in particular the lack of overview that left users with a sense that the system didn’t work properly. Several users suggested that they would have appreciated more opportunities to influence the system. This frustration seemed to be linked to a sense that it was difficult to persuade the design team that specific features were wrong. In our observations and interviews with users, we noticed three reasons for this difficulty: First, the fact that the system was implemented in two organizations at once made it difficult for users to argue against the suggestion that a feature, they wanted changed, might not be useful for someone else. Second, the fact that the system was already under implementation made it difficult for users to place the full blame on designers, since it could be argued that the users themselves now had the responsibility for making the system work better by learning how to use it and adapt their previous work habits if necessary. Third, the delivery of the system in a series of updated versions, created a situation where criticism of the system, could be answered by the promise that issues could be solved in one of the later updates. Such promises could be very specific, but they could also be more elusive thus deflecting the users’ criticism rather facing it directly. Several users thus said that they were “listened to,” but that it was very unclear to them if or how their viewpoints would be translated into features or changes in the subsequent versions of the system.

When looking at how the project had progressed after 12 months, it was evident that the project plan (milestones) for this iterative, user-engaging project had indeed been followed. But nevertheless, we could observe that several users were dissatisfied with their opportunities for influencing the design process and we also saw users complaining that the designers had built a system based on a motivational theory they did not share. If the expectation was that iterative organizing would somehow guarantee that the users would be fully satisfied with the content and the process of the project, then that expectation was clearly not met.

Discussion and conclusion: The mixed blessings of iteration

The design field’s many advocates of iterative approaches have emphasized that iteration allows designers to learn more from users, because prototypes and early versions create opportunities for focused discussion and assessment in relevant use contexts (Ehn and Kyng, 1991; Kolko, 2015; Sanders and Stappers, 2014). Although this may be true to some degree in many cases, our in-depth study of the EMOVE case gives us reasons to doubt the assumption that an iterative organization of design work necessarily or in itself leads to constructive user involvement. The key to this doubt is the observation that actual design work unfolds as a series of phases where different modes of handling user representations alternate in taking center stage. This sequential move through different dominating modes drives the project forward and gets the work done, but, inevitably, it also creates disconnects and deferral of issues. In practice, iteration is therefore not a process where all user representations are continually related to and incorporated into the evolving version of a system; instead, iteration is series of moves pragmatically aimed at driving the project forward in the context of time pressure, immediate problems and readily-available resources (Hyysalo, 2010). In the following, we will retrace some of the steps of the EMOVE project and bring attention to three specific issues that complicated the expectation that an iteratively organized design process would lead to a constructive and beneficial form of user engagement.

In the milestone plan of the EMOVE project, iteration was effectively operationalized as the obligation to deliver a system within 6 months followed by a series of updates on a tight schedule. For this reason, the design team was already under time pressure when it began building the first version of the system. As we have shown, the design team moved from a mode where conflicts between user representations were avoided (“collecting attractive visions”) into an entirely different mode where focus was on settling a variety of specific design questions (“negotiating co-existence”). When building the system, the team was clearly addressing tensions, dilemmas or even conflicts between user representations. But all these were negotiated within the design team, by drawing on a broad spectrum of user representations and by the team’s efforts to interpret and elaborate the design problems at hand. When the project reached the implementation phase, the design team solicited feedback from users that would guide the team to the “removal of stumbling blocks,” that is, minor problems that could be fixed quickly. At this stage, however, it became evident that while the design team settled a long list of specific design questions when building the system, it had also ignored fundamental question about the system’s implicit motivational theory. Here, the design team, had relied on its own assumptions about the users’ motivation and built these assumptions into the system with the help efficient tools such as of libraries of code that allowed the team to produce a functional version within only 6 months.

What these events show is that although the project had an iterative organization, there were in practice strong limitations on the topics or questions that were iterated. As soon as pressing deadlines were approaching, the crucial discussion of alternative motivational theories was left unexamined, and the design team rushed to produce a system that materialized some of its own fundamental assumptions.

When the system was launched, the stage was presumably set for involvement of users in a process of testing, evaluating and reiterating the system (cf. Hartswood et al., 2002; Botero and Hyysalo, 2013). More than 30 hours of meetings with users from the two customer organizations did indeed take place, and the design team adjusted and changed several features that might have been stumbling blocks for the implementation. But the users’ objection to the organizing principle of the task list was not accepted by the design team, and as we have described previously, the users sometimes found it difficult to object to features of the system: Faced with an already-implemented system, the users themselves could be blamed for not putting enough effort of learning the new system, they might be told that features they disliked were beneficial to other organizations, or the users might be met with the sometimes-elusive promise that future updates would solve the current grievances. The case thus demonstrates that the type of iteration that hinges on a series of quick releases and updates (often known as a minimally viable product strategy) is a highly ambivalent device for user involvement. On the one hand it provides a tangible product that users can assess in context, but on the other hand it may overwhelm the users and deflect their objections.

In our discussion so far, we have pointed to two major factors in the EMOVE project that complicates the idea that iteration leads to constructive user involvement. First, that fundamental discussions between designers and future users were left unexamined when the design teams rushed to build the system. Second, that the launch and implementation of the quickly developed system may have turned the scales against the users making it difficult for them to raise doubt about specific features as well as fundamental premises of the system.

With these observations, one might be tempting to draw the conclusion that the design team to some degree simply ignored the demands and wishes of the users. Our case study, however, suggests a more nuanced interpretation. First, we know from our observations that the design team had a strong commitment to user-oriented design and that it went to great lengths to solicit input from users. Second, and more importantly, the assumption that “the users” were ignored necessarily builds on the assumption that one clearly knows who “the users” are. It is striking that many of the iconic examples of iterative organizing appear to presume that it is perfectly clear who the users are. In participatory design, Ehn & Kyng describe how designers and factory workers meet in person and used cardboard boxes to explore different options for arranging the machinery that configured those factory workers’ future workplace. In agile programming, it is suggested that a person known as the “user representative” should be present at the meeting where it is decided which features to develop in the up-coming sprint period.

In the EMOVE project, the question of who the users of the system are remained elusive. MatchDesign’s business goal was to develop system that could be used by a wide variety of organizations, but it was yet to be figured out which types of voluntary organizations, or other organizations, were willing to buy and use the system. For the managers in MatchDesign, it was therefore not a given that all design issues, big or small, could be resolved through discussions with the two user organizations that first adopted the system. Contrary to larger and wealthier software developers (Pollock et al., 2016), MatchDesign did not have the time to first develop a range of features for selected companies and only later turn these into a generic product. MatchDesign had to do both at once, navigating a situation where its understanding of the product, the users, and the market were co-evolving (Hyysalo and Hakkarainen, 2014). In this context, the question of the designer’s implicit motivational theory, becomes murkier. If EMOVE had asked its present users whether they agreed with the dream of “inbox zero,” some of them would have answered no. But if the question is whether a system build on this assumption may find users and markets in the future, one might argue, that this is a bet worth pursuing. Bringing this back to the discussion about iterative organizing, we can summarize our discussion of the EMOVE case by pointing to three issues that complicated the clear path between iterative organization and user involvement.

Iterative organizing—which in practice can be depicted as a movement across alternating modes of handling user representations—does not prevent fundamental questions from being left behind or left out of the discussions between designers and users.

Quickly built materializations of a system (e.g., minimally viable products) that presumably enable user feedback simultaneously run the risk of overwhelming the user and deflecting their critique.

While iterative organizing within a project may be a (highly imperfect) device for engaging specific, known users, it does not address the more complicated, but very real problem of attuning a design project to emerging markets and future users.