Development method in a large-scale public sector IT programme

Abstract

The Norwegian parental benefit programme started as a large-scale information technology operation to automate and provide self-service solutions for parental benefit applications. This benefit replaces lost salaries for about 100,000 parents every year while parents are home with new-born children. A previous programme to modernize IT solutions was stopped. The new programme replacing legacy solutions from the 1970s was seen as a large, complicated, and risky undertaking with a cost limit of USD 130 million. Initially, this programme had three main releases based on an internal development method combining project management and agile methods. During the second release, the programme experienced challenges with coordinating work across the business and development projects. The programme manager established a task force to propose changes in the development method for the last release. We then ask the question: If you were part of this task force, what should the programme manager do to remedy the coordination challenges for the last release? This teaching case is based on a longitudinal case study and is relevant for graduate or continuing education courses in information systems development or IT project management during which students learn how to organize efficient software development at scale.

Introduction

In 2016, the Norwegian Parliament voted to initiate a large information technology (IT) programme to modernize solutions for applying for parental benefits, which is a benefit available from the Norwegian Labour and Welfare Administration. ¹ The programme had a cost limit of about USD 130 million and was the second of four programmes to modernize a legacy IT infrastructure from the 1970s. The aim of the programme was to develop a reusable decision solution for individual benefits and use this solution to provide new solutions for parental benefits.

Parental benefits replace lost salaries when parents are home up to 12 months with new-born children. Norway has about 5.5 million inhabitants, and parents submit about 100,000 parental benefits applications annually. The Norwegian Labour and Welfare Administration used to receive 280,000 inquiries about parental benefits and applications every year. The existing solution was described by media as ‘complicated’, ‘time-consuming’, and ‘incomprehensible’.

A new solution would enable advanced self-service solutions for parents, automate most of the application management, and make it easier to manage future changes in regulations. It was estimated that the programme would be socioeconomically profitable. Reuse of previous solutions or development of a solution based on standard components were considered, but development of a new tailor-made solution was seen as the best alternative for such a new solution. However, it was acknowledged that it would be a large, complicated, and risky programme ² (Flybjerg, 2014; Schmidt, 2001). A long start-up phase with gradual build-up of the programme organization was planned so as to reduce risks. One provider would be selected to support the business side of the programme and another to develop new software solutions. Thorough proof-of-concept analyses would be carried out and efforts made to develop and test the software in an efficient manner. ² It was decided to organize the work as a single programme because it was viewed as difficult to divide it into a series of programmes. ²

A previous programme to modernize the IT solutions with an estimated cost of USD 330 million at the Norwegian Labour and Welfare Administration was stopped after 9 months due to a lack of progress after about USD 70 million had been spent. A parliament hearing was held, and the Director of Modernization, the IT Director, and the Director of the Norwegian Labour and Welfare Administration resigned. Trust in the ministry was ‘totally broken’ according to one informant, and new programmes were planned as smaller programmes using known technology and a known development method. Because of a new reform, the old system was to be replaced by 1 January 2019.

The new solution aimed to reduce the number of inquiries by 25%, achieve a self-service rate of 80%, and decrease incorrect payments by 10%. The Norwegian Labour and Welfare Administration described the goals as described below:

‘(1) automatic application processing,

(2) users could manage their application through the self-service solution and

(3) electronic collection of information from caseworkers would provide better quality and more efficiency in application processing’. ³

The parental benefit programme was planned to last from October 2016 to June 2019. At its peak, over 200 people worked in the programme with about 130 people on 10 development teams of whom 100 were external consultants from Alpha ⁴ and Sopra Steria. Both companies had experience with large-scale IT programmes. Alpha was chosen to support the business- and requirement-related work, while Sopra Steria was chosen for development. The programme manager was employed by the Norwegian Labour and Welfare Administration. The programme depended on functionality in about 20 other internal systems.

This teaching case first provides theoretical background concerning development methods and tailoring of methods to a context. Second, it describes the organization and development method chosen for the programme and challenges with coordinating work across the business and development projects identified during the work for the second release. The programme manager established a task force with a mandate to propose changes in the development method for the third and final release. Third, we ask you to imagine that you are part of this task force. You will find questions for discussion and tasks on providing advice to the programme manager. Finally, the teaching case is summarized.

Background: Development method and method tailoring

A development method is ‘an approach to perform a systems development project, based on a specific way of thinking, consisting of directions and rules, structured in a systematic way in development activities with corresponding development products’ (Brinkkemper, 1996). A development method has several purposes: (1) It establishes ‘common ground for understanding and coordinating development’, (2) ‘normative principles (who should do what, when) that coordinate work dependencies’, and (3) ‘impose standards for evaluating design decisions’ (Thummadi and Lyytinen, 2020). Development methods were previously based on the waterfall model, ⁵ while agile development methods, such as Scrum ⁶ and eXtreme Programming (XP), ⁷ are prominent today. Brinkkemper (1996) distinguishes between development method and techniques and tools in which a technique is ‘a procedure, possibly with a prescribed notation, that is used to perform a development activity’. An example of a technique is the practice of pair programming in the XP development method. A tool is ‘a possibly automated means to support a part of a development process’ and includes support for testing, code analysis, and generation of code or documentation.

Making choices on the development method

How do organizations decide on which development method to use? Fitzgerald et al. (2006) describes two main approaches: The contingency factor approach and method engineering.

The contingency factor approach involves selecting a development method from a range of methods based on mapping the specific features of the development project to features of the development method. Typical challenges involved with this approach include the problem that organizations are not intimately familiar with many development methods, and it is also a considerable cost in training employees in a new method (Fitzgerald et al., 2006). Many suggestions as to which features to focus on are available. Boehm and Turner (2003) advise using several features to balance agility and discipline:

• Personnel (many with high competence favouring an agile approach)

• Criticality (less safety-critical favouring an agile approach)

• Size (many people favouring a traditional approach)

• Culture (thriving more on chaos than order favouring an agile approach)

• Dynamism (many requiremental changes favouring an agile approach).

Method engineering focuses on adaptation or ‘tailoring’ of development methods. This concept is sometimes referred to as ‘situational method engineering’ which focuses on adapting a standard method to the specific context of the development effort. A study of customization of agile methods at Intel Shannon shows how this organization selected six of twelve techniques from XP and combined these techniques with local Scrum tailoring. Scrum tailoring involved two stages in planning: (1) at the beginning of an iteration (‘Sprint’) and (2) at the beginning of the project. These steps led to benefits, such as reduction in code defects and projects delivered ahead of schedule.

Does method matter? Method in the small and the large

Does the method matter? What is the relationship between methods as described in literature or internal documents and the ‘method-in-use’ in practice? A recent study found that about 40% of the performed design activities are rooted in development method (Thummadi and Lyytinen, 2020). The remaining activities are chosen as the method does not describe the specific area, does not fit, and are chosen due to skills and habits or ‘organizational noise’. Software development ‘is continually socially constructed, and, accordingly, local abstractions are influenced by multiple, often contradictory and incomplete extrasomatic sources’ (Thummadi and Lyytinen, 2020).

In recent years, most environments that are involved in software development have been using agile approaches for such development. These methods were originally thought to fit small co-located development projects and develop software that is not life-critical (Williams and Cockburn, 2003). The most popular method is Scrum. The Scrum method recommends development of a product in iterations with a description of what is to be developed in a product backlog managed by a product owner and a team facilitated by a Scrum master. The product owner, the Scrum master and the team plan each iteration as a sprint backlog. The team demonstrate the product at the end of the iteration, and conduct a retrospective meeting to learn and adjust the development process. What is to be developed is typically described in overall epics, which are then decomposed into ‘user stories’ and then further broken down into tasks in the iteration planning. Scrum can be used for larger projects that involve several teams with joint coordination of work at Scrum-of-Scrum meetings, typically with one participant from each team.

From the mid-2000s, many organizations started using agile approaches, not only in single teams, but also for larger projects and programmes (Hoda et al., 2018), which typically consists of many development teams that work together for a long period of time. Practitioner environments have suggested several methods, such as Disciplined Agile Delivery (DAD), ⁸ Large-Scale Scrum (LeSS), ⁹ Scaled Agile Framework (SAFe) ¹⁰ and the Spotify model, ¹¹ for large-scale agile development (Dingsøyr et al., 2019). These methods prescribe agile methods at the team level combined with new roles, artefacts, and work processes to coordinate the work of many teams. LeSS, for example, introduces a joint iteration planning for all teams before teams plan their own work. LeSS comes in two versions, one for 2-8 teams and the ‘Huge’ for more than eight teams. LeSS Huge introduces additional roles, such as the ‘area product owners’ responsible for a requirements area for the product under development.

Researchers have investigated the use, efficiency, and challenges of these methods. A study found that developers often misunderstood the concepts and routines described in the methods, and it was problematic to choose between different methods (Conboy and Carrol, 2019). Some developers felt that decisions were made ad-hoc, some development teams were frustrated by frequent changes in methods, and some teams did not change their protocol after a new method was introduced. The researchers found that while agile methods at local levels allowed flexibility, the new large-scale methods came with predefines structures, routines, and tools, which were much more dominant, and that ‘small realignments can cause significant disruptions across units of an organization’ (Conboy and Carroll, 2019).

Case: The parental benefit programme

First, we describe the organization of the programme, then the chosen development method, and finally, the coordination challenges identified during work on the second release. This led to the programme manager establishing a task force to put forth recommendations for possible changes in the development method for the third and last release.

Programme organization

The programme was planned with three main releases consisting of 50,000 to 75,000 h of estimated work, which were named ‘the baseline’, ‘the settler’, and ‘the digital’. The releases were planned based on the timeframe shown in Figure 1.

Figure 1.

Programme timeline (modified from Dingsøyr et al., 2023).

The baseline release was a digital application processing system that automatically processed applications for one-time benefits. The settler release expanded the application processing system to include all types of parental benefits and integration with employers’ wage systems. This release aimed to develop a complete decision-making system adapted to the requirements of calculation in the law. The aim of the last release (the digital) was to create a self-service solution integrated with an extended application processing system and to support integration with health actors. The goals for the release included creating a complete integration between a planning calendar and a dialogue about benefit applications with users and conducting a digital dialogue between the user and the application caseworker. The previous release created a minimum viable product that was to be further developed. The main difference of this last release when compared with the first two was the programme was to extend a solution that was in use and add new functionality in a domain that was less well explored.

The programme had 16 roles, such as a construction manager, controller, and customer manager, at the programme level (Table 1). The senior solution architect at programme level would work with the application architects at team level. Similarly, the quality assurance and test roles at programme level would work with the testers at team level. The work was organized into four main projects: (1) business, (2) development, (3) test, and (4) change management (Figure 2). The three releases had four phases (Figure 3): (1) analysis of needs, (2) solution description, (3) construction, and (4) test.

Table 1.

Roles at the programme level and roles on development teams during the first phase (from Dingsøyr et al., 2023).

Roles on programme level	Roles in development teams
Construction manager (1), controller (1), customer manager (1), environmental manager (1), functional architects (7), functionality responsible (1), performance test manager (1), project manager (1), project support (1), PMO (1), quality assurance (1), solution manager (1), senior solution architect (1), test automation (1), test data responsible (1), test manager (1)	Scrum master (1), application architect (1 - 2), developers (5 - 6), testers (1 - 2)

Figure 2.

Organization of the programme with four main projects (from Dingsøyr et al., 2023).

Figure 3.

Development phases (from Dingsøyr et al., 2023).

The whole programme was physically collocated in the same work area on two floors as shown at one time in Figure 4. Some participants in the programme also had experience from the large-scale Perform programme (Dingsøyr et al., 2018) and had a background in a similar development method.

Figure 4.

Physical work area where the programme was located (from Dingsøyr et al., 2023).

The business project and the development teams were, however, located in different parts of the work area. The functional architects were located with the business project but prepared solution descriptions of user stories for the development teams. These were documented in the programme wiki.

‘every product owner had a functional architect who would ensure wholeness, they had meetings across to ensure coordination’ (group interview).

Development method

The development method was an internal method that had been used for previous large-scale programmes. This method was a combination of a project management framework (based on PRINCE2; Bentley, 2010) and agile methods Scrum and XP (Baham and Hirschheim, 2021; Project Management Institute and Agile Alliance, 2017). This is often referred to as large-scale agile development methods (Dingsøyr et al., 2019; Edison et al., 2021). The contract was a target price contract that emphasizes shared responsibility and risk.

For each release, the business project was responsible for the analysis of needs phase. In this situation, requirements were first described as epics and then broken down into user stories (Cohn, 2004). In collaboration with the development project, such requirements were given a more detailed solution description before being assigned to a development team in the construction phase. According to informants, the solution descriptions were ‘very extensive’ and ‘detailed’. After development, the approval phase organized by the test project followed. The programme would then, at a particular time, be in the production phase of ‘the baseline’ release while being in the construction phase of ‘the settler’ and undertaking an analysis of needs for ‘the digital’ (Figure 3). The change management project introduced new solutions to the main user groups, namely, the end users seeking parental benefits and caseworkers at the Norwegian Labour and Welfare Administration.

As shown in the timeline in Figure 1, the programme started with one development team and gradually increased the number of participants to 10 teams, which we consider a very large programme (Dingsøyr et al., 2023). Development started in February 2017. One team was allowed to continuously add (‘deploy’) new functionality starting in October 2017, while the other teams followed Scrum iterations towards the main releases. The first release was approved in November 2017.

The development teams worked in 3-week iterations according to Scrum with the four roles on each team described in Table 1. Application architects and testers were main responsible for software architecture and software quality for their teams, but all team members would work on tasks also related to these areas. As described, teams obtained solution descriptions from the business project. Teams then underwent sprint planning, followed by daily meetings during the iteration, review and retrospective meetings at the end. A release would have several iterations in the construction phase. Sprint reviews were conducted across all teams, and sprint planning started with overall planning for all teams. Teams would use XP practices, such as pair programming, while some also used test-driven development. Testers worked in collaboration with the test project. Furthermore, teams used feature-branching in that they would make a new codebase branch when working on a user story, which was then merged with the main branch when the user story was ready. The development teams would coordinate in Scrum-of-Scrum meetings two to three times per week. Generally, dependencies between teams were initially handled by the Scrum masters but then turned over to developers who would talk directly to others, hold ad-hoc meetings, and/or use instant messaging or emails. When two teams worked on closely related user stories, they were sometimes physically moved to the same location.

Coordination challenges during ‘the settler’; task force established

The programme delivered according to plan in the first release. In focus group interviews, informants described work in ‘the settler’ as being characterized by not only time pressure but also a meeting culture in the programme when the organization had grown to seven development teams. Such a culture made it time-consuming to make decisions:

‘It was a constant pressure to deliver. We had six to seven development teams that should continuously be fed tasks for their sprints. And that is quite a number of people and quite a lot of power in consuming user stories’ (manager, development project, group interview).

‘… people were in meetings the whole time, and you’d never find anyone by their desk; because you didn’t find a person there, you had to invite them to a meeting … And when first inviting, you’d also invite more people to make sure’ (business analyst, business project, group interview).

An informant stated that because people tended to have full schedules, calling for a meeting often would delay decision-making by more than a week.

Interviews with key persons in the programme indicated that internal coordination in the projects was perceived to be working well:

‘Coordination internally in the business project and internally in the development project worked well’ (manager, development project, group interview).

The development project used a total of 18 different coordination mechanisms. Mainly scheduled group meetings, such as the demonstration at end of each sprint, planning, and Scrum-of-Scrum and daily meetings. The daily meetings were staggered so that meetings were open to all project participants. Ad-hoc meetings, which were easy to organize as the participants were physically close to each other in the work area, were also conducted. Furthermore, team members were rotated and pair programming and instant messaging directly between individuals and to larger groups were used. The project also used physical artefacts, such as architectural guidelines, maps of dependencies, and team whiteboards visible to all. When teams were working on interconnected tasks, they were moved closer to each other as previously mentioned.

Nine coordination mechanisms between the development and the business projects were established. A special role, namely, the ‘functional architects’, was established to handle contact between the business and development projects. They were initially asked to divide work equally between activities in the business project and be present on the development teams to answer questions and make clarifications, but in actual practice, most time was used for customer meetings. In addition, several meetings, such as the demonstrations, meetings to approve user stories, working meetings on solution descriptions, and what they called ‘user story meetings’, which were informal meetings to clarify issues in user stories, were also conducted.

Additional dependency maps, an issue tracker (Jira) for transferring user stories from the business project to the development project, and user stories themselves with a solution description attached were also incorporated.

Despite these mechanisms, all parties expressed frustration with the coordination between the business and development projects:

‘The coordination between projects was more demanding’ (manager, development project, group interview).

‘In the business project, it was impossible to get insight into and obtain an understanding of what was happening and how they were working in development. You described needs, and it was like delivering to a black box’ (business analyst, business project, group interview).

The timespan from approval of a user story until the initiation of the development process could be months, and when needs for clarifications arose, the people who had written the solution description for the user story were often working on new tasks and found it difficult to recall what they had meant.

Informants stated that the handover from solution description to construction was a formal transfer of tasks and was regulated in the contract. This type of transfer led to much negotiation over the status of user stories and solution descriptions, namely, ‘many steps and much formalism’ (group interview).

A retrospective evaluation in January 2018 focussing on the delivery method identified the ‘transitions between [the] phases [of] analysis of needs, solution description, and construction’ (Figure 3) as one main challenge. This transition also included a handover of tasks from the business to the development project.

Many programme participants expressed concerns that the development method might not work well when used at the ‘digital’ release at a time when the teams would be maintaining a system that was running while developing new functionality in an area that was much less explored. The programme was also expanding with plans for 10 development teams to work in parallel at the end of the ‘settler’ release. The plan with three main releases implied long delays in feedback from users. In the IT department at the Norwegian Labour and Welfare Administration, a general change to agile development methods was made.

The programme manager established a task force with broad representation from the programme that was given a specific mandate:

• ‘The task force shall evaluate if the programme can work more efficiently in the “digital” and advise the programme manager. The programme manager will then put forth a recommendation to the programme owner for final decision’.

• The scope for the work group is the ‘digital’ and the ‘settler’ deliveries; however, the [next large programme] is beyond the scope of the discussion.

• The recommendation shall not increase the risk for the ‘settler’ or the time of the reform. Time-criticality for functionality shall be maintained. ¹²

Suggested case questions and assignment tasks

Questions to be answered individually or discussed in a group

1. What project management practices can you identify in the programme?

2. What agile development practices do you recognize?

3. How crucial do you think addressing the coordination challenges identified in the case are in terms of the success of the programme?

4. What do you think are pros and cons of a contingency factor approach and method engineering approach to changing development method?

Tasks to be given to a group for discussion and presentation

Given the problems of coordination towards the end of the second release, what should the programme manager do? Imagine that your group was the task force asked to advise the programme manager on development method in the last phase of the programme:

Task 1. What are the main arguments for changes, and what are the main arguments against such changes? How should the programme change development method if changing? What would you recommend?

Task 2. Given that the task force recommended a change in development method, how would you introduce/implement that change?

Conclusion

The parental benefit programme was viewed as a large, complicated, and risky programme that was initially planned with three main releases. It was initiated by the Norwegian Labour and Welfare Administration as a large-scale IT programme aiming to enable advanced self-service solutions and to automate applications for parental benefits. This benefit replaces lost salaries when parents are home up to 12 months with new-born children, and about 100,000 parents apply every year. A previous programme to modernize the IT solutions at the Norwegian Labour and Welfare Administration was stopped due to a lack of progress. The parental benefit programme would replace legacy solutions from the 1970s and had a cost limit of USD 130 million.

To reduce risks, the programme was planned with the incorporation of known technology and development methods. The programme was organized into four main projects: (1) business, (2) development, (3) test, and (4) change management, and three releases would go through the phases of analysis of needs, solution description, construction, and test. The development method combined practices from project management, Scrum, and XP.

While experiencing challenges with coordinating work across the business and development projects, the programme manager established a task force with the mandate to propose changes in the development method for the last release. The teaching case provides a background for development method, methods tailoring, and methods for small and large development efforts. The teaching case is based on a case study of a context to which few have access and focus on choosing and tailoring development methods, which is a topic many find crucial for efficient software development, particularly on a large scale.