Boards of Directors and the Governance of Large IT Investments: They Don’t Know What They Don’t Know

Abstract

Information technology (IT) is central to most organizations’ success. As IT systems age, organizations replace them, yet many of those replacement projects fail to achieve expected outcomes. This article explores what boards of directors can do to prevent such failures, increasing the chances that these replatforming programs succeed. Boards are typically affected by seven blind spots, including misplaced optimism, an abundance of data but too little information, and too much focus on technology. This article describes all seven blind spots and provides four recommendations to overcome them: developing a shared mental model, ensuring organizational readiness, agreeing on a reporting scorecard, and focusing on benefits and pivoting when necessary. The article also includes a framework for developing a shared mental model as well as a set of key questions for board members to ask about replatforming programs.

On April 22, 2018, following a board of directors’ decision two days earlier, TSB Bank went live with its new information technology (IT) platform. The bank was attempting to pull off one of the most ambitious IT transformations in British banking history, moving data about its 5.5 million customers from an old platform, managed by its previous owners Lloyds Bank, to one created by its new owner Sabadell. ¹ In the days and weeks that followed go-live, the internet and mobile app channels were unstable and almost unusable, and problems with the new platform caused significant issues in branches and call centers. The ensuing widespread outages affected millions of TSB Bank’s customers and led to the bank being investigated by the Financial Conduct Authority ² and Bank of England, ³ as well as having its board of directors and senior executives hauled before Parliament. ⁴ The debacle ultimately cost the bank more than £350 million ($500 million) in fines and compensation, with the chief information officer (CIO) personally fined over $90,000. ⁵ One of the many findings of an independent review by law firm Slaughter & May was that TSB Bank’s Board did not sufficiently understand the complexity of the data transition task. ⁶

The TSB Bank case is just one of many technology investments that we have studied over the years where boards of directors had been heavily involved in governance and oversight, yet the program’s outcome still had disastrous consequences for the organizations involved. (See Appendix for an overview of research.) Information technology investments like the one at TSB Bank are distinctive because, while they are a central element in helping an organization achieve its digital transformation ambitions, they are often couched as modernizing, upgrading, or replatforming programs or framed as a systems consolidation exercise or a technology replacement initiative. Consequently, these once-in-a-generation investments may not even make it onto the board’s agenda for scrutiny. As a result, not just the scale and scope of these investments, but also the considerable risks associated with them are misunderstood. ⁷

A defining characteristic of technology is that today’s leading edge is tomorrow’s legacy; investments made in the past eventually must be either upgraded, modernized, or simply replaced. What most organizations refer to as their core systems or operational backbone are the result of technology assets, data, processes, and applications that have been accumulated over many years, often because of merger and acquisition activity. These systems are frequently kept together with little more than a Band-Aid. When organizations finally decide to replace them, the resultant program is a significant organizational and technical change initiative that has been likened to swapping out an airplane engine while the plane is still flying. No part of an organization is likely to be untouched by a replatforming program. Moreover, as the organization is fundamentally dependent on these systems, any outage will significantly affect customers and may have regulatory implications. Because of the scale and scope of the work involved, these programs are likely to require significant investment ⁸ and run over multiple years. Boards should be concerned because the failure of these IT investments can have a material impact on the business, sometimes even bringing a company to collapse. ⁹

While few, if any, organizations could survive for very long if they have an IT outage, it is only relatively recently that technology began appearing on board agendas. ¹⁰ Many boards have added technology to their agendas in response to legislation, regulation, and/or the requirements of a securities commission rather than out of a concern for risk mitigation. However, there is strong evidence of the performance benefits of board-level governance of IT. ¹¹ Recently in Europe, a slew of legislation—such as the General Data Protection Regulation (GDPR), ¹² the Digital Services Act, ¹³ the AI Act, ¹⁴ the Network and Information Security (NIS) Directive, ¹⁵ and the Digital Operational Resilience Act (DORA) ¹⁶ —has elevated technology as an issue for boards. For example, under DORA’s mandate, boards of financial institutions are now responsible for digital resilience and ensuring that there is an effective framework in place for managing and mitigating IT risk.

In this article, we seek to help boards mitigate the risks of large IT replatforming investments. ¹⁷ We move beyond existing arguments that point to the weak digital savvy of boards ¹⁸ and recommend concrete actions. We first summarize the key challenges all boards face in governing large, risky investments. From our data, we then describe the specific blind spots that we have identified in governing replatforming investments. ¹⁹ We then present actions to tackle these blind spots to mitigate the risks. We draw on research and our personal experiential data set, both of which are considerable; for continuity and coherence in the early part of the article, we focus primarily on a subset of that research—specifically, the cases of TSB Bank, the broadcaster BBC’s Digital Media Initiative (DMI), and the Co-operative Bank. (The latter two are summarized in Box 1.) ²⁰

Box 1.

Background Information on BBC and The Co-Operative Bank Cases.

The BBC The Digital Media Initiative (DMI) was a major technology-enabled transformation program that was conceived to allow BBC staff and partners to develop, create, share, and manage video and audio content and programming on their desktop computers. The objective was to consolidate what were separate systems and bring them together on a new platform. The BBC decided at the outset to build a custom platform as commercially available products at the time did not support its vision. The work to develop the platform was initially contracted to an existing IT contractor, Siemens. After seventeen months, both parties agreed to cancel the contract, with the BBC deciding to take the work in-house. Having run for seven years, the program was eventually canceled after the BBC Trust (the BBC’s board) concluded that most of the £125.9 million spent had been wasted. One of the conclusions from a report by the UK government’s comptroller and auditor general, presented to the Trust, was that the level of assurance and scrutiny applied to the DMI was insufficient for a high-value and strategically important program that involved significant risks. “The BBC executive did not have a sufficient grip of the program and did not appear to appreciate the extent of the problems until a late stage. If the BBC had established better governance and reporting for the DMI, it would likely have recognized the difficulties much earlier. . . . The BBC Trust failed to exercise sufficient oversight of Executive Board’s delivery.” a A statement from Margaret Hodge, Member of Parliament and Chair of the House of Commons Committee of Public Accounts, on the publication of this report, remarked that “this report reads like a catalogue of how not to run a major program. I was shocked to learn how poor the BBC’s governance arrangements for the Digital Media Initiative were.” The Co-operative Bank In June 2013 the UK’s Co-operative Bank reported a capital shortfall of £1.5 billion, leading subordinated bondholders to convert some or all of their debt to equity, which gave a controlling interest to the U.S. hedge funds holding the bank’s debt. The bank’s attempt to replace its core banking systems, a program that was canceled at a cost of almost £300 million, contributed to the shortfall. Seven years earlier, the bank had started investigating options for upgrading its core banking system and had extended the scope a year later to be an enterprise platform that would include not just its core banking systems but customer relationship management, data management, and business process systems. The bank expected this larger scope to be simpler and cheaper to implement than if it used multiple suppliers for different applications, although the bank recognized that the larger scope would mean less functionality. During the build, the bank merged with a much bigger competitor and decided that it too would migrate to the new platform. This increased not only the scope of the replatforming program, but also risk. Sir Christopher Kelly, a former British civil servant, was commissioned to conduct a review of the events that led to the capital shortfall. Regarding the IT replatforming program, he wrote, “It is difficult to avoid the conclusion that the board failed to interrogate the program sufficiently closely and paid inadequate attention to its obvious difficulties until it was too late.” Moreover, he found that former members of the board’s Banking Transformation Program Subcommittee who, he noted, “should have been better placed than other directors to understand the program, described being surprised that it failed.” His damning critique: “They should not have been.” b

a

British Broadcasting Corporation: Digital Media Initiative, Comptroller and Auditor General, National Audit Office, UK, January 27, 2014.

b

Sir Christopher Kelly, “Failings in Management and Governance: Report of the Independent Review into the Events Leading to the Co-operative Bank’s Capital Shortfall,” April 30, 2014, https://assets.ctfassets.net/5ywmq66472jr/3LpckmtCnuWiuuuEM2qAsw/9bc99b1cd941261bca5d674724873deb/kelly-review.pdf.

Challenges All Boards Face

All boards face challenges when governing risky initiatives like mergers, acquisitions, or when opening new plants, launching new products, and entering new markets and regions. Research ²¹ has identified concerns that all boards need to be cognizant of that can impact their responsibilities in these matters; in our research we also observed these.

• Asymmetry of information: Management collates and presents information to boards that is necessarily a subset of all the available data. Management may also conceal relevant information.

• Episodic engagement: Only meeting a few times per year, boards effectively dip in and out of what is happening in an organization and may therefore not have a continuous understanding of critical issues and decisions.

• Attention distraction: Within a board’s purview, there may be many critical issues. Moreover, some board members may also hold roles on multiple boards and could struggle to clearly separate and fully focus on the issues of each.

Board dynamics also influence a board in performing its responsibilities. Boards’ effectiveness results from having the right mix of personalities and expertise, facilitated by a culture of mutual respect, trust, and (importantly) constructive dissent. ²² Particular dynamics can impact board performance ²³ :

• Support over scrutiny: Within boards, there can be a preference to support, rather than continually challenge, the executive team. Not wishing to bicker, board members tend to prefer consensus, even if it means some directors don’t speak up or are silenced.

• Groupthink: Board members with similar backgrounds or dominant board members may sway others around the table; for example, an authoritative and powerful board chair with a particular—but ultimately inaccurate—perspective may dominate others, or the board may fail to identify all the options available.

• Lack of expertise: It is only relatively recently that technology has been recognized as a key area of competence for board members; many director regulations and courses still only give it cursory attention. It is perhaps for this reason that the latest research shows that only a quarter of boards have a director with expertise in technology. ²⁴ Board members are not expected to have a working knowledge of what it takes to be successful with information and technology, yet no board member would ever consider it acceptable to lack basic financial competence. This lack of knowledge about IT only exacerbates the risk related to asymmetrical information because few directors will fully understand the implications of the information being presented or know if it is misleading or incomplete.

• Escalation of commitment: Instead of altering course or canceling the project or initiative, boards (often unreasonably) make decisions in the hope that the next iteration will be more successful.

Boards engage in a number of activities to help reduce the negative consequences of these issues:

• Hire advisers (e.g., external auditors, external consultants) to check decision criteria and risk and to opine on the veracity and completeness of the information presented to the board.

• Create a subcommittee to focus on a specific issue or project. This group often meets more frequently than the board, scrutinizing matters related to an investment or issue more deeply. ²⁵

• Bring on new board members to augment the board with specific knowledge and experience related to the risky investment.

• Rotate board membership to bring new perspectives and expertise. ²⁶ Company law and national corporate regulations for boards often suggest tenure lengths for directors.

• Conduct governance reviews to assess and continually improve decision processes, skills mix, and board performance.

Each board of the three organizations that we use as a foundation for our narrative engaged in most of these activities. For example, the board of TSB Bank appointed two new non-executive board members during the program, one of whom had considerable experience with large IT transformations, and hired external advisers to provide independent advice. While the Co-operative Bank created a new board committee focused exclusively on the replatforming program (the Banking Transformation Subcommittee), both the BBC and TSB Bank scrutinized the program in existing board committees (finance committee and audit committee, respectively). Despite these actions, the replatforming programs at all three organizations had significant negative impacts on their businesses. Our research revealed the reason: the unique nature of IT replatforming programs leads to blind spots that can pose considerable risks and render a board impotent.

Blind Spots

The migration of the banking platform at TSB Bank didn’t lack board attention. As the bank’s Chairman told the UK Parliamentary investigation: “There was a robust framework. There were independent experts throughout the program giving us technical reviews and audits . . . We had independent experts sitting alongside the board . . . saying, ‘Is the board asking the right questions? Are we getting the right information?’ The board and the audit committee between them met thirty-four or thirty-five times.” ²⁷

This IT migration was the TSB Bank CEO and leadership team’s fifth. ²⁸ During his response to questioning from UK lawmakers about the debacle, the CEO emphasized that “Migration was the number one corporate risk since the bank commenced the program . . . It has been right at the top of our risk register. . . . We ran nine migration acceptance cycles and four dress rehearsals for the weekend. We ran numerous volume tests. Through the tests and through refinement through the tests, we were given the belief that we were ready for the migration.” ²⁹ Yet, immediately after the system went live, there were serious outages and problems that lasted for many weeks and months, with massive implications for customers. ³⁰

Information technology replatforming programs have lots of moving parts. Many decisions relating to the program will be made every week. The organization lacks experience with the new platform; the accompanying change will impact many employees and customers; the scale and length of the initiative is unprecedented. Research and our own experience as consultants and board members have identified seven of the most dangerous blind spots that replatforming initiatives present for boards.

Misplaced Optimism

By the time that a replatforming initiative is proposed and planned, the organization has lived for many years with systems that are expensive to maintain or difficult to change to accommodate new products or services, lack documentation and institutional knowledge, and can contain cyber risks. Management will present persuasive reasons to upgrade the technology infrastructure. However, boards often fail to consider that the organization may not have the capability to execute the transformation program, interpreting senior executive commitment and enthusiasm as a proxy for the organization’s ability to deliver.

The Co-operative Bank, for example, lacked the capability to manage a large replatforming program and this, according to a review led by Sir Christopher Kelly, “played an important part in setting the program up to fail. No one should have been surprised that costs escalated and the project proved unsuccessful.” ³¹ Presentations, reports, and board papers all described the new platform as a commercial off-the-shelf package, a bank in a box, giving the board the impression that it was ready to be implemented. However, the product required significant configuration to make it compliant with UK regulatory requirements and UK banking practices, as well as to meet the bank’s own specific needs. It also required significant work from the vendor and bank staff to migrate data from the legacy systems, to build interfaces to other applications, and to train staff.

At TSB Bank, the post-implementation report of Slaughter and May noted that the board did not make “common sense challenges” to executives as to why, despite some workstreams being seven months behind schedule, the bank would be ready for migration in four months. ³² The National Audit Office (NAO) report into the DMI program concluded that it “was a major technology-enabled transformation program for the BBC. The BBC was too optimistic about its ability to implement it and achieve the benefits.” ³³

We also found other causes of this overoptimism, including the following:

• Boards are often comfortable relying on the conventional three lines of defense to monitor risk (front-line operations, risk management, and internal audit). However, the scale and novelty of these large replatforming programs can mean that the second line (risk function) and third line (internal audit) may not have the required capacity or capability to oversee programs of such scale and complexity. At TSB Bank, the risk oversight and internal audit teams carried out an extensive program of reviews. However, these reviews were at times inadequate, meaning issues were not identified and actions were not completed. Moreover, these teams did not ensure that the board understood all the material caveats and limitations included in their opinions. ³⁴

• An organization may have previously outsourced IT and later realized that technology was now a competitive necessity. As organizations looked to bring IT back in-house and take control, often as a part of a replatforming program, they often assumed that the process would be easier than it ultimately proved to be. As the CIO of a global energy company said to us, “we had outsourced the brain and totally underestimated the scale of knowledge we had lost due to two cycles of outsourcing.” ³⁵

Lots of Data but Little Information

Reports from management outlining the progress of a replatforming program, whether delivered to an audit committee, a technology subcommittee, or the whole board, often contain hundreds of pages of complex, detailed functional data (e.g., amount and quality of code developed, tested, and integrated or the amount of data cleaned, migrated, and integrated). In addition, there may be status reports from internal audit on nonfunctional but critical aspects such as organizational readiness, adherence to security standards, and cohesion within the teams working on the program. Advisers and experts provide reports on specific aspects or continuously throughout the program.

The board does not lack data, and each report is trying to provide context, opinion, and recommendations. However, each report is prepared from a specific perspective, using language known to the writer, and the board must interpret and integrate these often-disparate views to fully understand status and progress or to respond to a request for extended timelines, additional budget, or to reduce program scope. Independent reviews of the programs we studied concluded that there were shortcomings in the information given to the board: in some cases, information was missing; in others, there was so much data being reported that the board was not receiving the critical messages.

The Slaughter and May review of TSB Bank concluded that if the board or subcommittee had been presented with more complete and accurate information about key factors, either one would have concluded that the bank was not ready to go live on April 22, 2018. The Kelly review of the Co-operative Bank noted that the board would have had a greater chance of providing effective oversight if company executives had provided them with better information. The review found that the board papers were detailed, but that some omitted important information, failed to adequately draw out key facts, or gave an overly optimistic picture of what was really happening. A similar situation existed at the BBC. The NAO report noted that the reporting arrangements were not fit for purpose. ³⁶ It found that there was no clear and transparent reporting on progress—against the plan, cost to complete, or achieving benefits—to enable effective decision-making within the corporate governance structure.

Board packages are often long and detailed, exacerbated by the fact that most executive management teams want to be seen as acting transparently and not holding back information. Yet, the sheer volume of information can overwhelm a board member. At TSB Bank, the board did receive materials that referenced the fact that the Proteo4UK Middleware ³⁷ would need to be designed, built, and tested, a significant undertaking and critical to the success of the replatforming program. However, the program review material did not sufficiently highlight these risks. They appeared in a section of background materials to the March 2016 Program Review Paper, on two pages of a 737-page board package. There was no explanation in board papers (nor discussion at TSB Bank board meetings) of this risk, which ended up contributing to the failed migration.

Overly Focused on Technology

Framing the investment as a technology replacement investment drastically underestimates the organizational change that will be required and the resultant knock-on effects on processes, employees, and customers. Even where a program starts out as a business initiative, in many of the organizations we studied, the investment was soon recast to emphasize the software being deployed. Employees relabeled the programs as the “Salesforce project,” the “Oracle replacement program,” the “SAP rollout,” or with other tech vendor names. This repositioning was particularly evident when the program was run out of the IT unit. Moreover, the abstract nature of these programs can lead boards to assess program status and progress simply by tracking time and cost metrics, giving them the illusion of being in control. ³⁸

The BBC’s DMI was one such program, managed out of the organization’s IT department. It sought to change and standardize working practices across all of the BBC based on implementing leading-edge technology. Delivery of a single set of processes and the required change in business operations were prerequisites for the successful delivery of the benefits presented in the business case. Yet DMI reporting focused on technology risks and issues rather than the ability of DMI to drive operational change to business practices in the BBC and its wider ecosystem. In oral evidence to the House of Commons Public Accounts Committee, the Director General of the BBC noted that “[t]he DMI is a fundamental reengineering of the way the BBC makes television across not all but much of the television production activity.” ³⁹ The NAO report noted that this would require significant “cultural change.” ⁴⁰

Problems Are Seen in Isolation

There are many contributing elements to an IT program: software vendors (e.g., representing applications, middleware, cloud platforms, telephony, storage), advisers (e.g., consultants, systems integrators, change management experts), developers (internal and contracted), and end users (internal staff, external customers, and suppliers). Each will typically be reported on during progress updates. When issues and challenges are presented in isolation, a convincing narrative can be constructed where each may look surmountable, yet when combined, they can prove to be a lethal combination. Technologists are optimists, willing and able to solve issues as they arise. However, the complexity and interrelatedness of the parts may not allow boards to form a composite picture of risk and spot a struggling program early enough to avoid future expenditure of budget and effort.

In the cases that we studied, the negative program outcomes generally resulted not from just one cause but rather from an amalgam of many, often minor, issues. The CEO of TSB Bank accused the authors of the Slaughter and May report of taking a “scattergun approach,” arguing that the report “casts very little light on lessons the industry could learn to stop something like this happening again.” ⁴¹ We disagree; the report highlights just how complex such investments really are. But this is the conundrum boards face, where one or two missteps can significantly derail a program.

The culture of the organization can also influence the tone of what is presented to the board. At the Co-operative Bank, status reports on progress did highlight some of the difficulties the program was experiencing, but these were always presented with a positive spin. This practice reflected the bank’s culture of only reporting things in a favorable light.

Loss of Historical Context

Replatforming programs will likely span multiple years, sometimes being paused, re-scoped, and restarted more than once. Some of the programs we studied lasted over a decade. Turnover of boards, subcommittees, and executives can result in weak collective memory, exacerbating the episodic engagement risk that all boards face. Technology advances and competitive and regulatory changes during the implementation also present a considerable risk; current events can suddenly overcome past decisions.

At TSB Bank, although the board saw an overview of the program’s progress at almost every meeting, these meetings did not critically reflect on the program’s history or the fact that it had strayed from a number of the guiding principles, assumptions, dependencies, and milestones as set out in its plan. These omissions meant that it was easier to lose sight of how far the program had slipped or to justify why the plan was not being followed.

Skeletons Hidden in the Design Closet

Replatforming programs can be a bit like an archeological dig; you don’t know what you will find until you start. What lies beyond the computer or device screen, in the bowels of the data center, in code, in data structures, in past application and systems design decisions, and in architectural choices, will probably not be fully known at the outset of a replatforming program. This observation is not meant to denigrate the IT department; it is just a reflection of how IT systems were built in the past, the limited capability of technology, and the pressure that IT professionals are often put under to deliver something and do so quickly. Nobody ever expected that organizations might still be dependent on systems built over 30 years ago using programming languages that went out of fashion decades ago.

As the program progresses, unknown unknowns are likely to emerge that can derail the program. For example, interfaces to applications in the systems landscape that are not being replaced can be problematic. These unknowns can stifle progress, adding to timelines, cost, and risk. One global bank’s replatforming program, which was to also include a complete data center move to a more modern facility two miles away, revealed that many of the servers had never been switched off since they were first plugged in, and nobody was quite sure what would happen if they were.

Equating Go-Live with Success

The business case for replatforming programs is often justified by expectations of increased productivity, reduced costs, and higher levels of customer engagement. Changes in the software platforms are a major contributing factor, but significant levels of change inside the organization are necessary to deliver the anticipated benefits. These changes—including fixing bugs, completing missed scope, and changing processes—can take one or more years. If the program is deemed complete on the go-live date and is no longer overseen by the board, achieving the expected business outcomes can be compromised. Moreover, technical debt is a well-known phenomenon associated with technology investments, and we have not come across a board that conducted a thorough evaluation of a replatforming investment to determine the extent, if any, of technical debt built up over the duration of the program.

TSB Bank had no contingency plans if the switch to the new platform did not go as expected. ⁴² The aggregate impact of a multiple-organ failure scenario had not been considered. It took four days following migration to set up a customer war room and overhaul the customer communication strategy, and the bank did not fully return to business as usual for 232 days following the main migration event.

While these blind spots are uniquely tied to the context of a replatforming program, they can be exacerbated by the generic challenges that all boards face, which we identified above. In Table 1, the seven blind spots are connected to these challenges.

Table 1.

Connecting Blind Spots to Generic Challenges that Boards Face.

Blind Spot	Description	Relationship to Board Challenges
Misplaced Optimism	Board interprets the strategic organizational need for replatforming and management’s enthusiasm as proxies for management’s ability to deliver.	This blind spot is an extension of the support-over-scrutiny challenge that boards face. In the case of replatforming programs, boards may not be aware of the experience of the management and technology team in driving a program of the scale and scope of the proposed initiative and may not challenge this point.
Lots of Data but Little Information	Reports to the board about program status contain multiple topics and strands. Difficulties in interpreting all this data can mean that an overall picture of progress and risk is difficult to discern.	Asymmetry of information is always a challenge, and in reporting on technology-enabled business programs there are no standard frameworks to specify what should be reported or to help directors interpret reports. This is quite unlike financial reporting, where management and board members use and understand common standards and statements. Furthermore, a lack of expertise can cause board members to struggle to turn data into meaningful insight.
Overly Focused on Technology	Replatforming programs are often discussed and presented as a change to a technology application rather than a change to the organization and its processes.	This blind spot is endemic to technology-enabled business projects. Whereas boards might believe that the “Salesforce implementation,” for example, is just a technical challenge, they would never consider that a merger is just a financial challenge.
Problems Are Seen in Isolation	Because a replatforming program is multifaceted, there will be several areas being reported on at any point in time. Connections between these may be obscure and the big picture difficult to discern.	This blind spot is an example of the lack-of-expertise challenge. Because boards typically recruit very senior or retired executives, and technology changes very quickly, board members may have little understanding of the trends in new platforms. Moreover, episodic engagement can also exacerbate this blind spot.
Loss of Historical Context	Given the scale and scope of a replatforming initiative, decisions often have to be considered in the context of decisions made earlier in the program.	Good board governance practices include regular turnover of members and committees. Yet these changes can weaken organizational memory and perspective about the replatforming project. Since many programs have started and restarted over several years, an understanding of the past can provide perspective for oversight. Episodic engagement with a program at monthly or quarterly board meeting can additionally lead to gaps in knowledge about the program developing.
Skeletons Hidden in the Design Closet	A replatforming program will replace some or all of the digital underpinnings of an organization.	This blind spot is uniquely tied to the hidden multi-layered architecture of replatforming programs.
Equating Go-Live with Success	Technology programs are often monitored carefully until the go-live date. The value of technology is not in its possession; rather, realizing value is an extended process that includes tracking benefit attainment over the following years.	This blind spot is like other challenges such as in a merger or acquisition. However, although boards have agreed on a business case, the metrics that are tracked are usually budget, scope, and time, which are go-live and program performance metrics. Most boards have no process to track organizational benefits from technology-enabled business programs.

Avoiding the Blind Spots

Boards may not know what they don’t know, but actions can be taken to illuminate these blind spots, enabling organizations to reduce the risks in a digital transformation replatforming initiative. One recurring refrain from the post-implementation reviews of problematic or failed replatforming programs is that they were not set up to succeed. From the outset, the programs were never going to deliver the expected business outcomes. The bottom line: the board must assure itself that, at the very least, the investment is set up for success. While necessary, this is not sufficient.

The board will not be involved in the program on a day-to-day basis but will dip in and out over the program’s duration—which is often many years—via reports, presentations, and discussions. Therefore, it is imperative for directors not only to ask the right questions to exercise their fiduciary responsibilities and provide ongoing oversight but also to interpret what the answers they receive mean for the overall risk and progress of the program. We assume that boards that have realized the importance and risk of a replatforming investment will take steps to shore up their capacity to govern such an investment—such as adding knowledgeable board members, hiring advisers, creating subcommittees, and ensuring there is ample time within the board and committee agendas to track and scrutinize progress. But even doing all these things does not guarantee success. For example, the Co-operative Bank established a board subcommittee to oversee its replatforming program, yet its members were surprised when it failed.

From the cases we studied, interviews with board directors and technology leaders, and established research, we have identified four actions that boards can take to avoid the blind spots and increase the chance of success for replatforming investments:

• develop a shared mental model,

• ensure organizational readiness,

• agree on a reporting scorecard, and

• focus on benefits and pivot when necessary.

Recommendation 1: Develop a Shared Mental Model

An independent review of governance at the Co-operative Bank noted that, when questioned about their role in the abandoned IT replatforming program, board members replied, “No one told us. It was the fault of management. We were kept in the dark.” The report concluded that “this way of thinking reveals a deep misunderstanding of the nature of individual director responsibilities. If individuals felt unable to question and challenge management, they were not discharging their duties at the board.” ⁴³ The Kelly review of the same bank made a similar observation: “It is unreasonable to expect non-executive board members to audit information provided to them in detail. But it is their responsibility to question it. . . . It is difficult to avoid the conclusion that the board failed to interrogate the program sufficiently closely and paid inadequate attention to its obvious difficulties until it was too late.” ⁴⁴ We suggest that the inability of board members to question and challenge management is exacerbated by their lack of an accurate and shared mental model of the replatforming program, and that this shortcoming is at the root of many of the blind spots.

In describing things, states of affairs, sequences of events, the way the world is, and the social and psychological actions of daily life, mental models play a crucial and unifying function. ⁴⁵ They enable individuals to make inferences and predictions, to understand phenomena, to decide what actions to take and control their execution, and above all to experience events by proxy. ⁴⁶ A mental model of a replatforming program would identify each of the key elements of the program and would be accompanied by questions that board members should ask to determine the status of these elements. ⁴⁷

Without such an understanding of key elements, board members may ignore critical issues and resort to tracking higher-level goals such as budget, scope, and schedule. What previous research has referred to as a lack of digital savvy of board directors may be the result of an inadequate mental model of technology-enabled business change programs.

A recurring theme across the failure cases we studied was that the board did not understand the complexity of the program. With a program to construct a building, physical mock-ups and scaled models of the finished structure enable board members to visualize what is being proposed. Once the investment is approved and construction started, board members can observe progress by visiting the site. A technology-enabled business investment is abstract and amorphous; there is nothing to see or touch. Board members must therefore build a model of the program in their minds, based on their understanding of a replatforming program and their interpretation of the information they glean from presentations, reports, and discussions.

To govern an organization’s financial health, board members construct a mental model using five basic elements: assets, liabilities, equity, income, and expenses. Board members see the relationships among these elements in statements whose format is prescribed by International Financial Reporting Standards (IFRS) or another set of standards. These statements—such as the profit-and-loss statement or the balance sheet—are essentially the scaffolding of mental models for reporting the financial matters of an organization. Directors are expected to understand and interpret these key statements and use this knowledge to govern.

There is no standard framework for a mental model of a replatforming program similar to that provided for accounting and finance by the IFRS or another governing body’s standards. Apart from high-level reports on budget, schedule, and scope, there is no balance sheet equivalent to show the status of the program elements at a particular point in time. For board members to learn about and effectively oversee the program, they need to understand the programs’ major elements and the risks associated with each element. ⁴⁸ Because models exist in the minds of the individual board members, they are invisible, so it is important that these are surfaced and shared.

By developing a shared mental model of their replatforming program, a board will be able to establish the questions that need to be answered before the program begins, during the program, and before the final decision to migrate to the new platform is made. These questions, for the most part, can be established at the beginning of the program and used throughout in reports by management and advisers. The management of TSB Bank searched for a mental model to share with their board. Finding none, they created their own, which they named The Assurance Matrix, which is described and illustrated in Box 2.

Box 2.

The Assurance Matrix of TSB Bank.

This Assurance Matrix was described by the CEO of TSB Bank as “an innovative and bespoke tool that enabled me and the executives to draw together a complex and wide-ranging program into a digestible and clear decision-making tool for the TSB Board.” Illustrated in Figure 1 below, the matrix’s five rows covered the readiness of the platform applications to meet requirements, the readiness of the infrastructure to meet requirements, the readiness of the business to use the platform, the IT service provider contract, and the ability to provide the data necessary to populate the new IT system. The columns covered the performance of each row by seeking to ensure there was capability (functional and nonfunctional), an ability to support business volumes, an ability to recover should a component or part of the system fail, and compliance with all regulatory, legal, and contractual requirements. The intersections of the rows and columns were referred to as cells, and each cell was intended to function as a control to validate the performance of the individual program components against their design. A set of questions for each of the cells was developed to ensure that appropriate evidence was collected and assessed to confirm that the 20 controls had been validated. For each question, specific entry criteria and evidence requirements needed to be satisfied before it could be deemed complete. According to the CEO, the matrix sought to “draw together all of the key questions each part of the business must be able to answer to give confidence that it is prepared for the migration event and to operate post-migration.” Each executive was required to sign-off on their respective aspect of the matrix. The matrix “was intended to provide all-encompassing assurance around the program.”

Figure 1.

Structure of the TSB Assurance Matrix with illustrative questions for applications work stream and capability assessment.

This Assurance Matrix was used in the latter stages of the TSB Bank program, after the initial go-live date was aborted, and the matrix provided good guidance on the areas it covered. However, it didn’t identify the complexity of the technical architecture of the program. As one director later reported, he didn’t think that any of his fellow directors understood the concept of middleware and its significance to the replatforming program. During discussions, board members referred to the middleware as “the Spanish architecture.” Their lack of awareness and focus on the progress of building and testing this piece of middleware meant that they did not see issues mentioned in reports as describing serious risks.

The Assurance Matrix also lacked a discussion of testing and the contractor’s ability to operate the new platform after it was implemented. The Slaughter and May analysis asserted that had the board had complete and accurate information, it would have concluded that the bank was not ready to go live. Moreover, the financial regulator criticized the board for not querying the evidence underpinning the answers provided in each of the cells, and that this undermined the usefulness of the matrix.

Although replatforming is one of the foundational investments in most digital transformations, there is limited research outlining the important elements of these programs. By combining elements from the TSB Bank Assurance Matrix and interviews with board members and technology leaders, we suggest a foundational mental model framework, as shown in Figure 2. This model contains the basic elements to be considered; any specific program may have additional unique aspects that need to be understood and governed. This mental model framework considers the entire lifecycle of a replatforming program, from initial strategy through to the ongoing realization of expected outcomes and benefits. The Supplemental Appendix contains questions that boards can ask to explore the risk inherent in each element. These seek to assertain whether the board, the management team, and the organization are ready to undertake the initiative, whether the key technology and change elements are understood and being monitored, and it suggests tracking the program after go-live to obtain benefits.

Figure 2.

Mental model of a replatforming program for boards.

To create a shared mental model, workshops can be organized for board members where the investment is explained and discussed. Management, vendors, and advisers should co-create the presentations, so a single view of the program is developed and discussed. This process can take time and commitment from the board, management, advisers, and vendors. However, if the board members don’t feel that they have a proper grasp of the program, they should pause until they do: you can’t govern what you don’t understand.

Recommendation 2: Ensure Organizational Readiness

Executives can be very enthusiastic about a replatforming investment; they will likely have been suffering the consequences of the old systems, fielding many complaints and excuses stemming from a creaking platform, and can see the advantages of getting a new platform in place as quickly as possible. While the business case might be positive (at least on paper), executives’ exuberance often needs to be tempered. A recurring theme in post-implementation reports from programs that failed or encountered significant issues was that the organization did not have the capability to successfully execute a program of such scale and scope. The organization may have successfully executed technology-enabled business projects in the past, but those projects may not have been of similar scale or scope to what is envisaged. Our research has shown that in public sector environments, the arduous process to get funding approval can result in executives pushing as much scope as possible into the investment proposal so as not to have to go through the process again so soon ⁴⁹ ; this practice only increases risk.

Once the board has developed a shared mental model of the important components of the replatforming initiative and the risks each entails, it is in a good position to assess the organization’s readiness to proceed. One way to evaluate the organization’s capability is to have external advisers do a readiness assessment of the organization. Topics of interest would include the size and success of previous projects, the capability of in-house talent, and the leaders’ and vendors’ experience and previous track records. Taking these assessments seriously could lead boards to request the hiring of new program leaders, staging the deliverables, or reducing the scope of the project to something more achievable. In the TSB Bank case, it is ironic that consultants brought to the board’s attention the disastrous replatforming program that a similar institution, the Co-operative Bank, had attempted.

From data about the programs we studied, our interviews, and guidance from the 5 T model, ⁵⁰ readiness can be assessed along the following dimensions:

• Scope and value: Has the project been sized to maximize business value while minimizing risk? Are benefits clearly linked to project deliverables?

• Time: Is this the right time to put the organization through this magnitude of change? Does the executive team have the bandwidth to manage a program of this scale and scope? What projects should be deferred to make space for replatforming?

• Talent: Is the organization—the technology and the business leadership—capable and experienced? Does the organization need to add specific talent (e.g., testing, change management, data migration)? Do the vendors and advisers have enough previous experience to support the organization?

• Plan: Has the plan been created in a bottom-up manner where activities are identified and sequenced, with timelines and budgets estimated for each? Or have the go-live date and budget been imposed on the program by management? Top-down planning may have strategic merit but can contribute to failure.

Recommendation 3: Agree on a Reporting Scorecard

Because there are no accepted standards for how to present the status of a replatforming project and then to evaluate one that is in progress, the board should consider how the program will be reported before any work commences. If board members do not fully understand the program, they tend to rely on surrogate measures, primarily budget and schedule, to assess progress. These measures can give the illusion that the project is under control, and boards may assume that if these measures are on track, then everything else associated with the program must also be fine. ⁵¹ Using aggregate metrics for scope, budget, and schedule might be useful only in the highest-level reports, assuming there is a shared understanding of what terms mean. For example, everyone should know exactly what yellow (or amber) means with respect to scope (if using a traffic-light themed reporting method) or what “complete” means for a software feature.

To create a measurement framework, each element in the shared mental model needs to have its own plan, stages, and metrics identified and agreed upon by management and advisers and presented to the board. This common model, together with an agreed-upon set of metrics, prevents an adviser’s report from being just another opinion. Instead, the adviser’s reports comment on the completeness and veracity of management’s reporting, and the reports add value by raising other current and future issues.

The history of replatforming programs teaches us that such investments rarely fail because the technology didn’t work; rather, failure is usually due to continuously evolving aspects related to the organization’s culture, politics, people, or its organizational design, and how these aspects relate to each other. The board should ask for regular updates on the progress of the organizational change part of the program, using data from pulse surveys, stakeholder analyses, training progress, and user acceptance testing. If the program impacts end customers, suppliers, or members of the partner ecosystem, ensure that there are robust plans to test the impact and the acceptance—such as focus groups or alpha or beta tests—before the program is rolled out.

Once plans and metrics are in place for each element of the program, it is important to stick to any guardrails, guiderails, and safeguards that are established. If these are to be breached or amended, there must be a clear rationale for doing so. We have observed many occasions where success criteria have been modified or redefined during the program, allowing the program to proceed when it should have been halted.

When TSB Bank stopped and then replanned its replatforming program, the new plan, called the Defender Plan, spelled out various assumptions, dependencies, and risks for the program. This plan also set out fifteen guiding principles to direct and test the revised plan. The guiding principles were based on the principles behind the migration work done to date, which were supplemented with new principles developed from the learnings over the previous twenty-four months. The guiding principles, which were to help management and the board make the decision to go live on the new date, included reduced levels of parallel work streams and required achieving a comprehensive test migration carried out in real time. The guiding principles were excellent, and adhering to them could have resulted in a decision to delay the go-live date. However, when they were breached, the program team downplayed their importance, a suggestion the audit committee of the board accepted, noting “that the program was now seeking to comply with them in a different way.” ⁵²

Boards may often need to ask why key milestones are consistently being missed, what compromises are being made, and the impact of those compromises on the risk profile of the program and the readiness to go live. Status updates should include commentary explaining the significance of the facts being reported. If not, it can be unclear whether a particular change in status reflects a material increase in risk or how that change might affect the wider program.

An example from our data set saw a board subcommittee taking four steps, over a three-year period, to get a comprehensive and clear reporting system in place. The subcommittee first asked the adviser for reports on the readiness for change and the leadership aspects of the program in addition to the technology components. The subcommittee then asked the adviser to keep a running list of his recommendations and what progress had been made to address them. So that the adviser could verify management’s reports, the board required that he have access to all the progress data and the people working on the program. And finally, when the program was facing multiple hurdles, the board subcommittee asked management and the adviser to use the same metrics and formats to report on progress. Because the adviser had experience in large replatforming programs and came from an established consultancy firm, the bank’s management adopted his framework. From then on, the subcommittee could focus on the discrepancies between the reports.

Recommendation 4: Focus on Benefits and Pivot When Necessary

One might think that the sole reason for a replatforming program is that the new platform replaces the old one, paying down technical debt, improving operational efficiency and tackling the drag on innovation. This goal is fundamental but incomplete; it is not an attractive proposition for a board that is being asked to approve spending, perhaps millions of dollars, and to agree to stop other important programs to make time for this change. It also leads the board to consider this change primarily as a technical one and to not understand the importance of organizational change in making use of the new platform’s capabilities.

Programs should be designed to deliver value incrementally to avoid a situation in which the program runs into difficulties, needs to be restarted, and has not delivered any benefits. Technologists may resist a phased approach and argue that it will take longer and cost more. This argument has merit, but only if there are no unknowns in the work plan. Breaking programs into phases gives time for the organization to learn and change on its journey and can also prevent facing a total loss on a failed investment. With creative planning, most programs can be split into phases to validate the underlying technology architecture, organizational readiness for change, and to deliver value in increments. As the CIO of a European catalog company that transformed its business model and operations to become an online retailer and marketplace told us, “You don’t plan a transformation, you learn it!”

To develop a risk- and value-based phased approach, it is critical that management develop, and the board understand, the major benefits of the program. To this end, we advocate for boards mandating a benefits realization plan, showing how and when each expected benefit from the investment will be delivered, the measure(s) that will be used to assess benefit achievement, and the plan for tracking the benefits after the technology implementation is complete. ⁵³

When boards require a benefits realization plan, four important outcomes will occur:

• The business case will be less high-level and aspirational and more focused on realizable benefits and business outcomes.

• A focus on benefits avoids scope creep, which has been shown to slow down and cause failure in many programs. Any request for scope extensions can be measured against the benefit plan before being accepted.

• When a program has changed the key metrics of scope, budget, or schedule, a recalculation of the benefits plan may reveal a need to scale down or stop the program. A benefits realization plan provides the board with measurable guardrails to make tough decisions.

• As one board member, with considerable experience in technology investments, cynically said to us, “If you track benefits, it increases the likelihood that you might actually get them!”

When reports from advisers and management repeatedly contain missed deadlines, broken promises, and no clear path forward to achieving benefits, boards should not fear pivoting. Our data suggests that large replatforming programs can sometimes take two or more attempts before they achieve their objectives. ⁵⁴ The journey not going as planned happens often enough that the board needs to consider it as a possibility and govern appropriately.

An email from a TSB Bank non-executive director with extensive experience of IT programs to the bank’s CEO at the time of the major replan of the program is sobering in hindsight. “Thinking back over a career (sadly) dotted with big replans . . . an observation is that most of the successful ones involved a defining moment when board [members], executives, and suppliers took a fresh look at the approach, acted boldly, exposed and dealt ruthlessly with any sacred cows, and moved forward more solidly as a result.” ⁵⁵

When a board is faced with a pivot, our conclusions from these and other cases are fourfold:

• Ask management and advisers to discuss with the board the main elements of the replanned program to create, in effect, a new mental model.

• Ask management and advisers to report on the new plans for realizing benefits. There may be an opportunity to reduce the scope and thereby reduce the risk of the initiative.

• Ask management to explain the new implementation plan and the choice of architectural elements and to seriously consider doing the planned work in phases.

• Require that the new initiative be completely re-baselined and that the calculations be reviewed by a trusted adviser.

Many of the programs we studied were replanned on a right-to-left or a top-down basis relative to an imposed go-live date. Sometimes this planning method can be necessary (for example, to meet a regulatory deadline), and the first phase can target that date. However, the board should insist on a bottom-up or left-to-right evaluation to ensure that the re-baselined program has enough funding, resources, management attention, and time to achieve the expected benefits.

Conclusion

As many organizations embark on digital transformation, they will invariably have to replace the foundational technology applications and platforms that support their core operations and customer service. These replatforming programs frequently end in failure or significant underperformance that can stifle strategy and operations. We argue that because of the significant risks associated with these investments and the challenges in achieving expected outcomes, they require oversight from the organization’s board of directors. However, as our research has revealed, boards that govern these risky investments are often blind to the dangers that replatforming programs pose and are ineffective at preventing catastrophe.

Our research has identified seven blind spots that prevent board members from seeing and therefore governing the inherent risks associated with large replatforming initiatives. For example, replatforming is often characterized as a routine technology upgrade, leading boards to have more confidence in their leaders to deliver on the business case than is warranted, failing to understand the level of change the organization will undergo, and not establishing metrics to track specific risks such as data migration. On their own, each blind spot can have significant negative implications for a program; together they are a lethal cocktail.

We make four recommendations to alleviate these blind spots: create a shared mental model, ensure organizational readiness, agree on a reporting scorecard, and focus on benefits. These recommendations support existing research that suggests establishing robust metrics for initiatives linked to strategy. They also reflect recent advice on the need for a strategic perspective and building a shared understanding among the board, management, and advisers. They advance previous work by advocating for the creation of a holistic shared mental model that spans the lifecycle of a replatforming initiative from ideation to value delivery.

We propose the structure and elements of such a model, which were validated by senior leaders. In addition, the concept of mental models shared by the board, management, and advisers could be usefully applied in many other areas of digital operations such as cybersecurity and artificial intelligence. How, for example, should a board conceptualize cybersecurity? These topics point to a broader problem of the dearth of research on the topic of board governance of technology. Our experience is that most directors are ill-prepared to govern technology, a problematic situation given the slew of legislative and regulatory mandates that have placed technology firmly on the board agenda.

Practitioners such as advisers and technology-savvy board members will recognize that implementing our recommendations may add cost and time to the implementation plan and create some tension between management and the board. As board members ask questions about previously unexplored aspects of replatforming, the answers and ensuing discussion may result in a request to change management’s original proposals. However, board members can take comfort in knowing that research has repeatedly shown that transformational programs need to proceed slowly for lessons to be learned, pivots to be taken, and benefits to be locked in. Speed and hubris are the enemies of successful and sustainable change.

Board members play an important role in governing the foundational technology that supports their organization’s operations, products, and services, and in ensuring that any investment does not compromise the organization’s future health. As many replatforming programs will outlast the tenure of a leadership team, the board needs to help set and protect the funding designated to pay down technical debt, track the progress of investments, and ensure that expected business outcomes are achieved. We offer these insights and recommendations in the hope that boards will have a more comprehensive understanding of replatforming initiatives and will be able to confidently govern them, from the time they are being considered to the eventual attainment of benefits.

Supplemental Material

sj-docx-1-cmr-10.1177_00081256251376318 – Supplemental material for Boards of Directors and the Governance of Large IT Investments: They Don’t Know What They Don’t Know

Supplemental material, sj-docx-1-cmr-10.1177_00081256251376318 for Boards of Directors and the Governance of Large IT Investments: They Don’t Know What They Don’t Know by Joe Peppard and Blaize Horner Reich in California Management Review