Abstract
Large IT projects are challenging as they are complex projects with many stakeholders involved. IBM won a project tender for a toll gate solution for vehicles for the Norwegian Road Authority. The project was planned with about 77 000 hours work, in four phases, and as a ‘global delivery’ with a local centre in Norway and five delivery centres in India and China. After signing the contract, there were three main milestones. A bit more than a year after initiation, the Norwegian Road Authority chose to terminate the contract due to lack of progress. This was not accepted by IBM, and the parties met in court in a case that would finally end by a rejection in the Appeals Committee of the Supreme Court of Norway. This teaching is based on the court case and focuses on the following learning objectives: Understand challenges with requirements engineering approaches, understand organisation of large, global, outsourced software development projects and understand IT project management challenges and risks as faced by the customer and the provider. The teaching case can be used in graduate courses and continuous education in information systems development, requirements engineering and IT project management. The suggested assignments include questions for discussion on organization of the project and managing requirements work.
Keywords
Introduction
Large IT projects are complex in that they typically involve many people who are to jointly make a product, often consisting of a number of technical components. The product has often several user groups, and thus there are many stakeholders in the project. There has been much discussion in the information systems, project management and software engineering literature on approaches to manage such projects – ranging from managing IT projects as any other project, to choosing approaches especially designed for digital product development.
The outcome of an IT project depends on a number of factors, such as the skills and interaction of the participants, available technology and resources and also the way the project is managed (McLeod and MacDonnel, 2011). Achieving successful large IT projects are critical for companies and the society. Learning about practices and organisation of large IT projects can help also in planning and conducting smaller IT projects.
A particular challenge in large IT projects is to make sure the project develops a product which satisfies the requirements of main stakeholders and user groups. Traditional approaches to requirements management typically involves analysing, defining and specifying requirements which are later verified when testing the product (Jarke et al., 2011; Sommerville, 2016). Outsourcing software development introduces a number of challenges (Lacity et al., 2016), in particular when developing software offshore (Smite et al., 2010). Agile approaches prescribe an iterative development of understanding of requirements and of the product (Project Management Institute, 2017).
This teaching case illustrates challenges in managing a large IT project, particularly focussing on managing requirements, but can also be used as a teaching case to understand organisation of a large, global and outsourced IT project, or to understand project management challenges and risks faced by the customer and the provider. This teaching case is based on the court case decision in the Oslo City Court. The court of appeals reached a verdict very similar to that of the district court, and the case ended with a rejection in the Appeals Committee of the Supreme Court of Norway.
The tollgate project
Norway has had a national system for automatic tolling of vehicles. Over 50 tolling companies financed road infrastructure and sometimes local public transport through invoicing of owners of passing vehicles. In 2012, it was estimated that there were 300 million passages yearly, and the tolling infrastructure had a turnover of around 600 million USD. 1
The system was developed for the Norwegian Road Authority (‘Road Authority’) by Q-free from 2006. It was owned by Q-free, but the Road Authority was licensed to use. This agreement was about to end, and in 2011 the Road Authority decided to procure their own solution. There were four companies bidding for the project. The Road Authority wrote a contract with IBM in the end of 2013 for delivering a new ICT solution for tolling in Norway, and also operation, maintenance, further development and consulting. The new project was named AutoPASS Tollgate (Figure 1). An AutoPASS tollgate. Picture: (Photo: Gro Lillebø).
IBM won the bid with a solution which combined their Road User Charging Asset which was in use in Stockholm and Brisbane based on products which had been tailored to the project: the customer relations management system SugarCRM, Cognos for reporting and SAP FI-CA for accounting. Finally, IBM would use IRIS, a system specially designed by IBM for identification and prizing and IBM Websphere to manage the dataflow between the components.
The project was planned as a ‘global delivery’ with a local delivery centre in Norway and five delivery centres in India and China, with 70% of the work offshored. The time schedule was tight, as the existing contract with Q-free was to expire.
Contract and project model
The contract model was developed for contexts where it is not possible to specify precisely or in detail what is to be made, and where there is uncertainty with respect to the project execution. The model includes phases and milestones as shown in Figure 2. The contract included the analysis of needs from the Road Authority and a rough solution description and uncertainty matrix from IBM. The contract emphasized timely delivery, with daily fines imposed on IBM for any delays. The project model with main phases, milestones and control points (CP1 - CP3) for each iteration in the construction phase. Source: Oslo City Court (2020).
The work of the Road Authority was organised into subprojects procurement, delivery and deployment. There was a joint steering group consisting of leaders from the Road Authority and representatives from the tolling companies, the Tollgate project management from the Road Authority and KPMG as quality assurers. This group was reporting to the director of the Road Authority. IBM was to work in the delivery subproject and were sometimes invited to the meetings in the steering group, most often to report on specific items on the agenda.
Further, there was a joint coordination group with representatives from the Road Authority, IBM and the consulting company A-2, which acted as an independent quality assurer in the project. This group met monthly. The Road Authority and IBM each had their own project manager. The project manager from IBM had the daily responsibility for planning, organising and leadership so that work was executed according to the contract. There were monthly progress reports. The project manager from the Road Authority was responsible for leading the work with the activities Road Authority was responsible for and coordinate with IBMs project manager. There were weekly joint project management meetings, where also A-2 as the quality assurer took part.
The project was organised in 13 work streams: Customer relations management, Accounting, Integration, IRIS (Identification, Rating and Interoperability Services), Maintenance, Architecture, Infrastructure, Migration, Reporting, Security, Self-service, Test and Training.
The main phases (see Figure 2) were solution description, then an iterative construction phase (three iterations) and finally an approval and termination phase.
There were three main milestones in the project after the contract was signed: • Milestone1: Approval of solution description • Milestone2: Delivery ready for approval • Milestone3: Approved delivery
Eliciting requirements
To open the tender for a diverse range of software suppliers, the Road Authority intentionally kept the tender requirements broad and general. This approach was taken with the hope that more suppliers would participate, given the flexibility in interpreting the best-suited solutions. While the requirements tables specified certain functional and non-functional needs, there was still room for varied approaches in addressing them.
Hour Estimates for Phases and Risk Premium From the Four Companies.
Source: Oslo City Court (2020). BT Signaal and Q-free are Norwegian Companies, Kapsch Based in Austria and IBM an American Multinational Company. BT Signaal is now Called Skyttel AS.
In the requirement tables, specific details were outlined for many functional needs. For example, one specification related to image processing for a vehicle passage was distinctly defined for particular irregularities: R1: If there is only one image of a passing [vehicle], and this image doesn’t have an automatic reading, then it should be processed at least twice by two different image processors.
Conversely, some requirements merely outlined essential functionalities without providing explicit instructions on their implementation. These included: R2: All contract creations, changes and terminations should be easily and efficiently executed by customers via self-service and by internal users of the solution. R3: It should be possible to generate a credit note for sending to a customer. R4: It should be possible to change the registration number on a chosen image [of a vehicle], and the passing must then be reprocessed.
What happened in the project
The solution description phase which started in January 2014 took longer than expected. Milestone1 (HMP1) was by agreement moved from February to April 2014. The solution description delivered was not approved by the Road Authority until July, when the project entered the construction phase.
The actual end users, the more than 50 local toll companies, conveyed their requirements to the Road Authority. Figure 3 illustrates the sudden increase in estimated hours shortly after project initiation. In response to these new estimates, the Road Authority accused IBM of ‘lowballing’ – a strategy where a company underbids to win a tender, only to later seek additional compensations through change requests from the customer. IBM countered by stating they were merely trying to cater to the customer’s desires and develop software that satisfied everyone’s wishes. Contracted hours versus estimated hours from monthly progress reports from IBM. Source: Oslo City Court (2020).
IBM frequently encountered delays when seeking clarifications from the Road Authority. Clarifications could take weeks. For example, on December 5th, 2013, IBM requested detailed pricing rules from all the local toll companies. However, they had to wait until February 12th, and even then, they only received information from one company. These pricing rules are crucial as they show local variations, such as higher rates during rush hours and different pricing models for public transport such as buses. The Road Authority was responsible for documenting these local models to IBM. Typically, the Road Authority representative would consult the toll companies for their input before responding to IBM’s inquiries, leading to a time-consuming process.
The Road Authority argued that the delays during the first year were caused by IBM assigning too few people to the project in order to save money. The Road Authority argued that this was why they were not sufficiently involved by IBM in the work. The Road Authority also claimed that they did not have the basis to understand the functionality available in IBM’s standard products: what could be solved through configuration, and what required customization. IBM, on the other hand, argued that the delays occurred because The Road Authority changed the requirements when the project began, compared to what was described in the tender. Based on the tender, IBM believed they could deliver standard features ‘out of the box’, meaning that the solution was largely pre-made and could be implemented quickly. However, when the project began, IBM felt they had to do much more customization to the standard products than they had proposed in the tender.
After the initial disagreement on why there were delays, the Road Authority started to closely monitor the project execution, asking for more detailed progress reports. IBM argued that this drained its management capacity and created delays because they had to spend time on writing progress reports instead of managing the development process.
To meet the requirement for a ‘uniform user interface’, IBM proposed gathering three web applications into one view using tabs in a web browser. Figure 4 shows how the IRIS application appears as a tab within the SugarCRM application. IBM planned to integrate all applications in a similar manner to present them within a single view. The proposal to consolidate all the applications into one screen view was developed during the ‘Solution Description’ phase. The Road Authority believed that navigating between tabs was not user-friendly and therefore did not meet the non-functional requirement for the user interface: R5: The solution’s user interface/user interaction should be designed to support work processes efficiently and ensure good ergonomics. Proposed integration of user interfaces in a single view (right). Source: Oslo City Court (2020).
At the start of the construction phase, IBM did not have the expected productivity, and content from iteration 1 was moved to iteration 2. The deliverables in iteration 1 was demonstrated for the Road Authority in October 2014, and included about 15% of the total deliverable, half of what was planned. Each iteration ended with a checkpoint. The checkpoint after the first iteration was not approved by the Road Authority. IBM, however, received payment for this iteration. Checkpoint 2 in April 2015 was not approved by the Road Authority because of lacks in a test report, that IBM had delivered less than agreed, and because the Road Authority found the progress plan to be unrealistic.
In June 2015, the Road Authority terminated the contract. IBM thought the termination was unjustified and the parties met in a court case that would finally end by a rejection in the Appeals Committee of the Supreme Court of Norway in 2023.
Assignment
The case description gives information on organisation of the project, work with eliciting requirements, and a description of main events which led to the Road Authority terminating the contract. The following questions for discussion focus on learning objectives on understanding challenges with requirements engineering, understanding organisation of a large, global and outsourced software development project and understanding project management challenges as faced by the customer and the provider: (1) How was the work with requirements organised in the project? (2) What challenges do you see with the approach to requirements work chosen? (3) What are alternative approaches to managing requirements? (4) What problems typically occur in globally outsourced software development projects? Do you see any signs of such problems in the case? (5) If you were the project manager from the Road Authority, what would you do to mitigate the challenges described in the teaching case? (6) If you were the project manager from IBM, what would you do to mitigate the challenges described in the teaching case? (7) Was it reasonable of the Road Authority to terminate the contact? Why or why not?
Footnotes
Acknowledgements
We are very grateful for a detailed verdict from the City court of Oslo.
We are further grateful to Magne Jørgensen at SimulaMet, Vedran Zerjav at the Norwegian University of Science and Technology as well as students at the Norwegian University of Science and Technology, the University of Oslo and at Kristiania University of Applied Sciences for comments on an earlier version of this teaching case.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The work was funded by the Norwegian University of Science and Technology and by SimulaMet.
Ethical approval
Article based on court decision.
